Image correction method, device and equipment

Abstract

The present application provides an image correction method, device, and equipment, which includes: obtaining a pixel point to be corrected in an original image, and obtaining a first image area corresponding to the pixel point to be corrected from the original image, wherein the central pixel point of the first image area is the pixel point to be corrected; dividing the first image area into K pixel blocks, where K is a positive integer greater than 1; selecting a target pixel block from the K pixel blocks based on the pixel features corresponding to each pixel block; correcting the pixel value of the pixel point to be corrected based on the pixel features corresponding to the target pixel block to obtain the corrected pixel value of the pixel point to be corrected; and generating a corrected image based on the corrected pixel value of the pixel point to be corrected. The technical solution of the present application solves the problem of severe loss of detail information and effectively improves the detail restoration and image integrity of the image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image correction method, device and equipment. Background Art

[0002] Image acquisition devices (such as cameras) can capture images of a target scene, for example, using sensors. However, due to inherent design flaws in the sensor, limitations in the manufacturing process, or defects in the imaging system's transmission link, some abnormal pixels may appear in the image. These pixels' values ​​significantly mismatch those of surrounding pixels, disrupting the clarity and integrity of the image. These abnormal pixels are referred to as bad pixels.

[0003] For example, see Figure 1 The figure shown is an example of a bad pixel. During the process of lesion detection, due to reflections from the human body's internal structures and other reasons, there will be obvious overexposed areas on the image. There will be bad pixels at the edges of the overexposed areas. The bad pixels will affect the image quality and interfere with the doctor's observation.

[0004] Due to the existence of bad pixels, the image quality will be reduced, which will affect the image clarity, image details and image integrity, that is, the image clarity will be reduced, the image details will be lost, and the image integrity will be poor. Summary of the Invention

[0005] The present application provides an image correction method, the method comprising:

[0006] Obtaining a pixel to be corrected in an original image, and obtaining a first image region corresponding to the pixel to be corrected from the original image, wherein a central pixel of the first image region is the pixel to be corrected; wherein the pixel to be corrected is an abnormal pixel belonging to a bad pixel block;

[0007] Dividing the first image area into K pixel blocks, where K is a positive integer greater than 1;

[0008] Selecting a target pixel block from the K pixel blocks based on pixel features corresponding to each pixel block; wherein the target pixel block is the pixel block having the greatest similarity to the pixel point to be corrected;

[0009] Correcting the pixel value of the pixel to be corrected based on the pixel feature corresponding to the target pixel block to obtain a corrected pixel value of the pixel to be corrected;

[0010] A corrected image is generated based on the corrected pixel values ​​of the pixels to be corrected.

[0011] The present application provides an image correction device, comprising:

[0012] An acquisition module is configured to acquire a pixel to be corrected in an original image, obtain a first image region corresponding to the pixel to be corrected from the original image, wherein the central pixel of the first image region is the pixel to be corrected; divide the first image region into K pixel blocks, where K is a positive integer greater than 1; wherein the pixel to be corrected is an abnormal pixel belonging to a bad pixel block;

[0013] a processing module, configured to select a target pixel block from the K pixel blocks based on pixel features corresponding to each pixel block; and correct the pixel value of the pixel point to be corrected based on the pixel features corresponding to the target pixel block to obtain a corrected pixel value of the pixel point to be corrected; wherein the target pixel block is the pixel block having the greatest similarity to the pixel point to be corrected;

[0014] A generating module is used to generate a corrected image based on the corrected pixel values ​​of the pixels to be corrected.

[0015] The K pixel blocks constitute a plurality of pixel block sets, and a line connecting central pixel points of two pixel blocks in the pixel block set passes through the pixel point to be corrected; when the processing module selects a target pixel block from the K pixel blocks based on the pixel features corresponding to each pixel block, it is specifically used to: determine a gradient value corresponding to the pixel block set based on the pixel features respectively corresponding to the two pixel blocks in the pixel block set; wherein the pixel feature corresponding to the pixel block is the average value, median value, maximum value, or minimum value of the pixel values ​​of all pixel points in the pixel block; and based on the gradient value corresponding to each pixel block set, select a pixel block with a smaller pixel feature in the pixel block set corresponding to the maximum gradient value as the target pixel block.

[0016] Wherein, the processing module corrects the pixel value of the pixel to be corrected based on the pixel feature corresponding to the target pixel block, and is specifically used to obtain the corrected pixel value of the pixel to be corrected: obtain a second image area from the original image, the central pixel point of the second image area is the pixel to be corrected, and the size of the second image area is smaller than the size of the first image area; wherein, the second image area is used to determine the pixel points to be corrected, the first image area is used to determine the target pixel block, and the target pixel block is used to correct the pixel points of the second image area; for each pixel point in the second image area, the pixel value of the pixel point is corrected based on the pixel feature corresponding to the target pixel block to obtain the corrected pixel value of the pixel point; the pixel feature corresponding to the target pixel block is the average value, or median value, or maximum value, or minimum value of the pixel values ​​of all pixels in the target pixel block.

[0017] The size of the first image area is determined based on a target gain value corresponding to the original image; the target gain value is a gain value used when the original image is acquired by a sensor; and the size of the second image area is determined based on the target gain value.

[0018] Wherein, the acquisition module is further used to determine the target gain value interval to which the target gain value belongs; by querying the first mapping relationship, the target intensity control parameter corresponding to the target gain value interval is obtained; wherein, the first mapping relationship includes the correspondence between the gain value interval and the intensity control parameter, and the larger the gain value interval is, the larger the intensity control parameter corresponding to the gain value interval is; by querying the second mapping relationship, the first target size corresponding to the target intensity control parameter is obtained, and the first target size is the size of the first image area; by querying the third mapping relationship, the second target size corresponding to the target intensity control parameter is obtained, and the second target size is the size of the second image area, and the second target size is smaller than the first target size; wherein, the second mapping relationship includes the correspondence between the intensity control parameter and the first size, and the larger the intensity control parameter is, the larger the first size corresponding to the intensity control parameter is; the third mapping relationship includes the correspondence between the intensity control parameter and the second size, and the larger the intensity control parameter is, the larger the second size corresponding to the intensity control parameter is.

[0019] Among them, when the acquisition module acquires the pixel point to be corrected in the original image, it is specifically used to: for the pixel point in the original image, acquire the first image area corresponding to the pixel point from the original image, select the adjacent pixel points corresponding to the pixel point from the first image area, and determine the upper limit pixel value and lower limit pixel value corresponding to the pixel point based on the pixel values ​​of the adjacent pixel points; if the pixel value of the pixel point is between the upper limit pixel value and the lower limit pixel value, it is determined that the pixel point is not the pixel point to be corrected; if the pixel value of the pixel point is not between the upper limit pixel value and the lower limit pixel value, it is determined that the pixel point is the pixel point to be corrected, or, based on the auxiliary features corresponding to the first image area, determine that the pixel point is not the pixel point to be corrected or is the pixel point to be corrected.

[0020] Among them, if the auxiliary feature includes the pixel value of each pixel in the first image area, the acquisition module determines that the pixel is not a pixel to be corrected or is a pixel to be corrected based on the auxiliary feature corresponding to the first image area, and is specifically used for: if the pixel value of the pixel is less than a first threshold, and there is an overexposed area in the first image area, then the pixel is determined to be a pixel to be corrected; if the pixel value of the pixel is not less than the first threshold, and / or there is no overexposed area in the first image area, then the pixel is determined not to be a pixel to be corrected; wherein, if the pixel value of each pixel in the first image area is less than a second threshold, and the second threshold is greater than the first threshold, then there is no overexposed area in the first image area; if the pixel value of at least one pixel in the first image area is not less than the second threshold, then there is an overexposed area in the first image area.

[0021] The present application provides an image correction method, the method comprising:

[0022] Acquire a first image region from the original image, wherein the first image region includes pixels to be corrected and an overexposed image region, or the first image region includes pixels to be corrected and the first image region is connected to the overexposed image region;

[0023] Extracting a target pixel block from the first image area, wherein the target pixel block is any pixel block in the first image area, and the target pixel block is related to a gradient value of the pixel block and a pixel value of the pixel block;

[0024] Correcting the pixel value of the pixel to be corrected based on the pixel characteristics of the target pixel block to obtain a corrected pixel value;

[0025] A corrected image is generated based on the corrected pixel values.

[0026] The present application provides an electronic device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is configured to execute the machine-executable instructions to implement the above-mentioned example image correction method.

[0027] As can be seen from the above technical solution, in the embodiment of the present application, the first image area is divided into K pixel blocks, a target pixel block is selected from the K pixel blocks based on the pixel features corresponding to each pixel block, and the pixel values ​​of the pixels to be corrected are corrected based on the pixel features corresponding to the target pixel blocks to obtain a corrected image. By correcting the pixel values ​​of the pixels to be corrected (i.e., bad pixels), the image quality can be improved, the image clarity is improved, the image details are not missing or the missing image details are reduced, the image integrity is relatively good, the problem of serious loss of detail information is solved, and the detail restoration and image integrity of the image are effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments of the present application or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings of the embodiments of the present application.

[0029] Figure 1 is a schematic diagram of a bad pixel in one embodiment of the present application;

[0030] Figure 2 is a flow chart of an image correction method in one embodiment of the present application;

[0031] Figure 3A and Figure 3B is a schematic diagram of a bad pixel and a bad pixel block in one embodiment of the present application;

[0032] Figure 3C is a flow chart of an image correction method in one embodiment of the present application;

[0033] Figure 4 is a schematic diagram of the positional relationship of pixels in one embodiment of the present application;

[0034] Figure 5 is a schematic diagram of eight pixel blocks in one embodiment of the present application;

[0035] Figure 6A and Figure 6B This is a schematic diagram of an image in one embodiment of the present application;

[0036] Figure 7 This is a schematic structural diagram of an image correction device in one embodiment of the present application;

[0037] Figure 8 It is a hardware structure diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0038] The terms used in the embodiments of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms "a," "the," and "the" used in this application and claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items.

[0039] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" used may also be interpreted as "at the time of" or "when" or "in response to determining".

[0040] In the embodiment of the present application, an image correction method is proposed, which can be applied to any electronic device. Figure 2 FIG. 1 is a flow chart of the image correction method, which may include:

[0041] Step 201: Obtain a pixel to be corrected in the original image, and obtain a first image region corresponding to the pixel to be corrected from the original image, wherein the central pixel of the first image region is the pixel to be corrected. The pixel to be corrected may be an abnormal pixel belonging to a bad pixel block.

[0042] Step 202: Divide the first image area into K pixel blocks, where K is a positive integer greater than 1.

[0043] Step 203: Select a target pixel block from the K pixel blocks based on the pixel features corresponding to each pixel block. That is, select one pixel block from the K pixel blocks as the target pixel block corresponding to the pixel to be corrected. The target pixel block is the pixel block with the greatest similarity to the pixel to be corrected, that is, the target pixel block is the pixel block that provides the most reference information for the pixel to be corrected.

[0044] Step 204: Correct the pixel values ​​of the pixels to be corrected based on the pixel features corresponding to the target pixel block to obtain corrected pixel values ​​of the pixels to be corrected. For example, a second image region may be obtained from the original image. When correcting the pixel values ​​of the pixels to be corrected, the pixel values ​​of each pixel in the second image region are corrected based on the pixel features corresponding to the target pixel block.

[0045] Step 205: Generate a corrected image based on the corrected pixel values ​​of the pixels to be corrected.

[0046] Exemplarily, K pixel blocks constitute a plurality of pixel block sets, and for each pixel block set, the pixel block set may include two pixel blocks, and the line connecting the center pixels of the two pixel blocks in the pixel block set passes through the pixel point to be corrected. Based on this, selecting a target pixel block from the K pixel blocks based on the pixel features corresponding to each pixel block may include, but is not limited to: for each pixel block set, determining the gradient value corresponding to the pixel block set based on the pixel features corresponding to the two pixel blocks in the pixel block set; wherein the pixel feature corresponding to the pixel block may be the average, median, maximum, or minimum value of the pixel values ​​of all pixels in the pixel block; and based on the gradient value corresponding to each pixel block set, selecting the pixel block with the smaller pixel feature in the pixel block set corresponding to the maximum gradient value as the target pixel block.

[0047] The pixel block with the smallest pixel feature in the set of pixel blocks corresponding to the maximum gradient value is the pixel block with the greatest similarity to the pixel to be corrected, and this pixel block is taken as the target pixel block. Obviously, based on the gradient value and pixel features corresponding to the pixel block, the target pixel block can be found from all pixel blocks.

[0048] Exemplarily, correcting the pixel value of the pixel to be corrected based on the pixel feature corresponding to the target pixel block to obtain the corrected pixel value of the pixel to be corrected may include but is not limited to: obtaining a second image area from the original image, the central pixel of the second image area being the pixel to be corrected, and the size of the second image area may be smaller than the size of the first image area; wherein the second image area is used to determine the pixel to be corrected (that is, each pixel in the second image area is used as the pixel to be corrected), the first image area is used to determine the target pixel block, and the target pixel block is used to correct the pixel points of the second image area (all pixels in the second image area). For each pixel in the second image area, the pixel value of the pixel point is corrected based on the pixel feature corresponding to the target pixel block to obtain the corrected pixel value of the pixel point; wherein the pixel feature corresponding to the target pixel block may be the average value, median value, maximum value, or minimum value of the pixel values ​​of all pixels in the target pixel block.

[0049] Exemplarily, the size of the first image area can be determined based on a target gain value corresponding to the original image; wherein the target gain value can be a gain value used when the original image is captured by a sensor; furthermore, the size of the second image area can be determined based on the target gain value.

[0050] Exemplarily, the target gain value interval to which the target gain value belongs can be determined; by querying the first mapping relationship, the target intensity control parameter corresponding to the target gain value interval is obtained; wherein, the first mapping relationship includes a correspondence between the gain value interval and the intensity control parameter, and the larger the gain value interval is, the larger the intensity control parameter corresponding to the gain value interval is; by querying the second mapping relationship, the first target size corresponding to the target intensity control parameter is obtained, and the first target size is the size of the first image area; by querying the third mapping relationship, the second target size corresponding to the target intensity control parameter is obtained, and the second target size is the size of the second image area, and the second target size is smaller than the first target size; wherein, the second mapping relationship includes a correspondence between the intensity control parameter and the first size, and the larger the intensity control parameter is, the larger the first size corresponding to the intensity control parameter is; the third mapping relationship includes a correspondence between the intensity control parameter and the second size, and the larger the intensity control parameter is, the larger the second size corresponding to the intensity control parameter is.

[0051] Exemplarily, obtaining a pixel to be corrected in the original image may include, but is not limited to: for a pixel in the original image, obtaining a first image area corresponding to the pixel from the original image, selecting adjacent pixels corresponding to the pixel from the first image area, and determining an upper limit pixel value and a lower limit pixel value corresponding to the pixel based on the pixel values ​​of the adjacent pixels; if the pixel value of the pixel is between the upper limit pixel value and the lower limit pixel value, then determining that the pixel is not a pixel to be corrected, that is, not an abnormal pixel belonging to a bad pixel block. If the pixel value of the pixel is not between the upper limit pixel value and the lower limit pixel value, then determining that the pixel is a pixel to be corrected, that is, an abnormal pixel belonging to a bad pixel block; or, if the pixel value of the pixel is not between the upper limit pixel value and the lower limit pixel value, then determining that the pixel is not a pixel to be corrected or is a pixel to be corrected based on the auxiliary features corresponding to the first image area.

[0052] Exemplarily, if the auxiliary feature includes the pixel value of each pixel in the first image area, determining whether the pixel is a pixel to be corrected or not based on the auxiliary feature corresponding to the first image area may include, but is not limited to: if the pixel value of the pixel is less than a first threshold value, and there is an overexposed area in the first image area, then determining that the pixel is a pixel to be corrected, that is, an abnormal pixel belonging to a bad pixel block. If the pixel value of the pixel is not less than the first threshold value, and / or there is no overexposed area in the first image area, then determining that the pixel is not a pixel to be corrected, that is, not an abnormal pixel belonging to a bad pixel block.

[0053] Among them, if the pixel value of each pixel point in the first image area is less than the second threshold, and the second threshold is greater than the first threshold, then there is no overexposed area in the first image area; if the pixel value of at least one pixel point in the first image area is not less than the second threshold, then there is an overexposed area in the first image area.

[0054] As can be seen from the above technical solution, in the embodiment of the present application, the first image area is divided into K pixel blocks, a target pixel block is selected from the K pixel blocks based on the pixel features corresponding to each pixel block, and the pixel values ​​of the pixels to be corrected are corrected based on the pixel features corresponding to the target pixel blocks to obtain a corrected image. By correcting the pixel values ​​of the pixels to be corrected (i.e., bad pixels), the image quality can be improved, the image clarity is improved, the image details are not missing or the missing image details are reduced, the image integrity is relatively good, the problem of serious loss of detail information is solved, and the detail restoration and image integrity of the image are effectively improved.

[0055] The image correction method of the embodiment of the present application is described below with reference to specific embodiments.

[0056] When an image acquisition device captures an image of a target scene through a sensor, some abnormal pixels may appear on the image due to design defects of the sensor itself, manufacturing process limitations, or defects in the imaging system transmission link. The pixel values ​​of these pixels are obviously mismatched with the pixel values ​​of surrounding pixels, which destroys the clarity and integrity of the image. These abnormal pixels are called bad pixels.

[0057] Bad pixels in an image are generally categorized as static and dynamic. Static pixels are fixed in position but have inaccurate pixel values ​​due to manufacturing defects. Dynamic pixels are loose in position due to manufacturing defects or errors in the photoelectric signal conversion process. Dynamic pixels appear normal within a certain brightness range, but their pixel values ​​change abnormally when the brightness range is exceeded. The difference between dynamic pixels and surrounding pixels also changes with changes in sensor temperature and gain.

[0058] A bad pixel area formed by the adhesion of a large number of dynamic bad pixels is called a bad pixel block (also known as a dynamic bad pixel block). Bad pixel blocks are caused by design flaws in the sensor or data transmission link. They typically appear at the edges of highly reflective areas in the image. Compared to static and dynamic bad pixels, bad pixel blocks are less affected by their own pixels but more by surrounding pixels. They also share the uncertainty and randomness of dynamic bad pixels. Bad pixel blocks are larger in size, causing more severe damage to image clarity, detail, and integrity. Their main characteristics are their unstable size and a high number of invalid pixels in their neighborhood.

[0059] See also Figure 3AThe following are examples of bad pixels (static bad pixels or dynamic bad pixels), see Figure 3B The figure below shows an example of a bad pixel block. While a small number of bad pixels are present, a bad pixel block is formed by the adhesion of a large number of dynamic bad pixels. During lesion detection, due to factors such as reflections from internal structures, the image may have obvious overexposed areas. Bad pixels may be present at the edges of these overexposed areas, and a large number of dynamic bad pixels may adhere to form a bad pixel block. This bad pixel block can affect image quality and interfere with the doctor's observation.

[0060] Due to the existence of bad pixel blocks, the image quality will be reduced, which will affect the image clarity, image details and image integrity, that is, the image clarity will be reduced, the image details will be lost, and the image integrity will be poor.

[0061] In response to the above findings, an image correction method is proposed in an embodiment of the present application, which can correct the bad pixel block (i.e., the bad pixel area formed by the adhesion of a large number of dynamic bad pixels) in the original image to obtain a corrected image. The method can be applied to image acquisition devices (such as cameras, etc.), such as after the image acquisition device captures the original image, image correction is performed on the original image. The method can also be applied to back-end devices (such as servers, hosts, NVRs, storage devices, etc.), such as after the image acquisition device captures the original image, the original image is input to the back-end device, and the back-end device performs image correction on the original image.

[0062] See also Figure 3C FIG. 1 is a flow chart of an image correction method, which may include:

[0063] Step 301: Obtain an original image. The original image may be in Bayer format or other formats, such as RGB or YUV. There is no limitation on the image format. For example, a Bayer format image can simulate color sensitivity and convert grayscale information into color information using a 1 red, 2 green, 1 blue arrangement.

[0064] For example, in some application scenarios, an endoscope can be used to capture an image of a specified type of tissue (such as nerve tissue or organ tissue, etc.) inside a target object, and the image can be used as the original image.

[0065] For another example, in certain application scenarios, a camera may be used to capture an image (such as a vehicle image) of a target scene (such as a highway), and the image may be used as an original image.

[0066] For another example, in some application scenarios, a camera may be used to capture an image (such as a human body image) of a target scene (such as an access control system), and the image may be used as an original image.

[0067] Of course, the above are just a few examples, and there is no restriction on the source of the original image.

[0068] Step 302: Obtain a target gain value corresponding to the original image. This target gain value may be the gain value used by the image acquisition device when acquiring the original image through a sensor. For example, when acquiring the original image through the sensor, the image acquisition device may control the image brightness using parameters such as shutter, gain, and aperture. In this case, this gain value may be used as the target gain value corresponding to the original image.

[0069] For example, since the area of ​​the bad pixel block changes with the gain value, the target gain value corresponding to the original image can be obtained, the degree of the bad pixel block can be estimated by the target gain value, and the size of the bad pixel correction window can be estimated based on the target gain value, thereby linking the bad pixel block detection and bad pixel block correction through the target gain value. The process of bad pixel block detection and bad pixel block correction can be referred to the subsequent steps.

[0070] Step 303: Locate the bad pixel blocks in the original image.

[0071] For example, since a bad pixel block is formed by the adhesion of a large number of dynamic bad pixels, the pixel information in the neighborhood is unreliable. In order to avoid missed detection and false detection, the detection accuracy of the bad pixel block can be improved through methods such as bad pixel degree estimation, pixel constraint, and auxiliary feature judgment. For example, the following steps can be used to locate the bad pixel block in the original image (that is, to determine whether the pixel in the original image belongs to the bad pixel block):

[0072] Step 3031: bad pixel degree estimation. For example, for a pixel in the original image, a first image region corresponding to the pixel is obtained from the original image, and the first image region is a bad pixel degree estimation result.

[0073] Exemplarily, the central pixel point of the first image area is the pixel point, and the size of the first image area is m*m. The value of m is configured based on experience and is not limited to this. The value of m can be an odd number or an even number, and an odd number is used as an example below. The value of m can be between a minimum value and a maximum value. The minimum value can be configured based on experience, such as 5, 7, etc., and the maximum value can be configured based on experience, such as 9, 11, etc. For example, taking the minimum value as 7 and the maximum value as 11 as an example, m is greater than or equal to 7, and m is less than or equal to 11. When the value of m is an odd number, m can be 7, m can be 9, and m can be 11.

[0074] In one possible implementation, the size (m*m) of the first image region can be determined based on the target gain value corresponding to the original image. For example, assuming that m has K size values, all gain values ​​can be divided into K gain value intervals, with gain value interval 1 corresponding to the first size value of m, gain value interval 2 corresponding to the second size value of m, and so on, with gain value interval K corresponding to the Kth size value of m. After obtaining the target gain value corresponding to the original image, the target gain value interval to which the target gain value belongs is first determined, and the size value corresponding to the target gain value interval is used as the size of the first image region.

[0075] For example, assuming that m has three size values, namely 7, 9, and 11, all gain values ​​can be divided into three gain value intervals: gain value interval 1 (e.g., minimum gain value to gain value A) corresponds to size value 7, gain value interval 2 (e.g., gain value A to gain value B) corresponds to size value 9, and gain value interval 3 (e.g., gain value B to maximum gain value) corresponds to size value 11. After obtaining the target gain value, if the target gain value falls within gain value interval 1, size value 7 is used as the size of the first image region, i.e., the first image region is a 7*7 image region. If the target gain value falls within gain value interval 2, size value 9 is used as the size of the first image region, i.e., the first image region is a 9*9 image region. If the target gain value falls within gain value interval 3, size value 11 is used as the size of the first image region, i.e., the first image region is an 11*11 image region.

[0076] In one possible implementation, the size (m*m) of the first image region can be determined based on the target gain value corresponding to the original image. For example, assuming m has K possible size values, all gain values ​​can be divided into K gain value intervals. A first mapping relationship can be preconfigured, comprising a correspondence between gain value intervals and intensity control parameters. A larger gain value interval corresponds to a larger intensity control parameter. The first mapping relationship can be a mapping table, a mapping function, or a mapping curve, without limitation, as long as it comprises a correspondence between gain value intervals and intensity control parameters. A second mapping relationship can be preconfigured, comprising a correspondence between the intensity control parameter and the first size. A larger intensity control parameter corresponds to a larger first size. The second mapping relationship can be a mapping table, a mapping function, or a mapping curve, without limitation. Based on this, a target gain value interval to which the target gain value belongs can be determined. By querying the first mapping relationship, a target intensity control parameter corresponding to the target gain value interval can be obtained. Then, by querying the second mapping relationship, a first target size corresponding to the target intensity control parameter can be obtained. The first target size is the size of the first image region.

[0077] For example, assuming that m has three size values ​​of 7, 9, and 11, all gain values ​​are divided into gain value interval 1 (such as the minimum gain value to gain value A), gain value interval 2 (such as gain value A to gain value B), and gain value interval 3 (such as gain value B to the maximum gain value). In addition, the first mapping relationship may include the correspondence between gain value interval 1 and intensity control parameter 1, the correspondence between gain value interval 2 and intensity control parameter 2, and the correspondence between gain value interval 3 and intensity control parameter 3, where intensity control parameter 3 is greater than intensity control parameter 2, and intensity control parameter 2 is greater than intensity control parameter 1. The second mapping relationship may include the correspondence between intensity control parameter 1 and size value 7 (i.e., the first size), the correspondence between intensity control parameter 2 and size value 9, and the correspondence between intensity control parameter 3 and size value 11. After obtaining the target gain value, if the target gain value belongs to gain value interval 1, the intensity control parameter 1 corresponding to gain value interval 1 is obtained by querying the first mapping relationship. The size value 7 corresponding to the intensity control parameter 1 is obtained by querying the second mapping relationship, and the size value 7 is used as the size of the first image area, that is, the first image area is a 7*7 image area.

[0078] Of course, the above is only an example of determining the size (m*m) of the first image area based on the target gain value, and there is no limitation thereto, as long as the size (m*m) of the first image area is related to the target gain value.

[0079] Exemplarily, the intensity control parameter can be a bad pixel block degree estimation parameter, and the intensity control parameter can be denoted as α. The intensity control parameter can control the size of the first image area. Since the intensity control parameter is determined based on the target gain value, the size of the first image area is determined based on the target gain value.

[0080] Exemplarily, for a pixel point in the original image, a first image area with the pixel point as the center pixel point is obtained from the original image based on the size of the first image area, that is, an image area of ​​size m*m.

[0081] Step 3032: Analyze whether the pixel in the original image belongs to a bad pixel block based on the pixel constraint.

[0082] Exemplarily, for a pixel point in the original image, the adjacent pixel points corresponding to the pixel point can be selected from the first image area, and the upper limit pixel value and lower limit pixel value corresponding to the pixel point are determined based on the pixel values ​​of the adjacent pixel points. If the pixel value of the pixel point is between the upper limit pixel value and the lower limit pixel value, it is determined that the pixel point is not a pixel point to be corrected, that is, the pixel point is a normal pixel point that does not belong to a bad pixel block. If the pixel value of the pixel point is not between the upper limit pixel value and the lower limit pixel value, it is determined that the pixel point is a pixel point to be corrected, that is, the pixel point is an abnormal pixel point belonging to a bad pixel block, or, based on the auxiliary features corresponding to the first image area, it is further determined that the pixel point is not a pixel point to be corrected or is a pixel point to be corrected.

[0083] For example, the normal pixels in the initial image should be constrained by the value range of adjacent pixels. Based on this pixel constraint method, bad pixels can be initially eliminated. For pixel P(i,j) in the original image, pixel P(i,j) can be used as the center pixel to construct a 1*m adjacent pixel array, that is, the pixels in the same row as pixel P(i,j) in the first image area form the adjacent pixel array. Alternatively, pixel P(i,j) can be used as the center pixel to construct an m*1 adjacent pixel array, that is, the pixels in the same column as pixel P(i,j) in the first image area form the adjacent pixel array.

[0084] Taking a 1*m array of adjacent pixels as an example, the adjacent pixels corresponding to the pixel point P(i,j) can be selected from the array of adjacent pixels, and the upper limit pixel value P corresponding to the pixel point P(i,j) can be determined based on the pixel values ​​of the adjacent pixels. max and the lower pixel value P min For example, the upper pixel value P max The determination method of is shown in formula (1). Of course, formula (1) is just an example. max There is no restriction on the determination method, and the upper limit pixel value P can be determined based on the pixel values ​​of adjacent pixels. max The lower limit pixel value P min The determination method of is shown in formula (2). Of course, formula (2) is just an example. min There is no restriction on the determination method, and the lower limit pixel value P can be determined based on the pixel values ​​of adjacent pixels. mi That's it.

[0085]

[0086]

[0087] Assuming that the size of the first image area is 9*9, that is, the value of m is 9, then formula (1) can be converted to formula (3), that is, the upper limit pixel value P can be determined by formula (3) max , formula (2) can be converted to formula (4), that is, formula (4) can be used to determine the lower limit pixel value P min .

[0088] P max =MAX(2*P2-P1,2*P3-P4,P2,P3) Formula (3)

[0089] P min =MIN(2*P2-P1,2*P3-P4,P2,P3) Formula (4)

[0090] In formula (3) and formula (4), P1, P2, P3 and P4 are the pixel values ​​of the adjacent pixels corresponding to the pixel point P(i, j). The positional relationship between P1, P2, P3 and P4 and the pixel point P(i, j) can be seen in Figure 4 As shown, P1 represents the pixel value of the fourth adjacent pixel point on the left of pixel point P(i, j), P2 represents the pixel value of the second adjacent pixel point on the left of pixel point P(i, j), P3 represents the pixel value of the second adjacent pixel point on the right of pixel point P(i, j), and P4 represents the pixel value of the fourth adjacent pixel point on the right of pixel point P(i, j).

[0091] After obtaining the upper limit pixel value P max and the lower pixel value P min Afterwards, if the pixel value of pixel point P(i,j) is within the upper limit pixel value P max and the lower limit pixel value P min If the pixel value of the pixel point P(i,j) is not between the upper limit pixel value P max and the lower limit pixel value P min If the pixel point P(i, j) is a suspected bad pixel (it cannot be confirmed whether it belongs to a bad pixel block), step 3033 is executed to analyze whether the pixel point in the original image belongs to a bad pixel block based on the auxiliary feature.

[0092] Step 3033: Analyze whether the pixel in the original image belongs to a bad pixel block based on the auxiliary features.

[0093] Exemplarily, the auxiliary features may include, but are not limited to, the pixel value of each pixel in the first image area. For pixel P(i, j) in the original image, if the pixel value of pixel P(i, j) is less than a first threshold (configured based on experience) and there is an overexposed area in the first image area, then pixel P(i, j) is determined to be a pixel to be corrected, that is, pixel P(i, j) is an abnormal pixel belonging to a bad pixel block. If the pixel value of pixel P(i, j) is not less than the first threshold, and / or there is no overexposed area in the first image area, then pixel P(i, j) is determined not to be a pixel to be corrected, that is, pixel P(i, j) is a normal pixel that does not belong to a bad pixel block.

[0094] Among them, if the pixel value of each pixel point in the first image area is less than the second threshold, and the second threshold is greater than the first threshold, then there is no overexposed area in the first image area; if the pixel value of at least one pixel point in the first image area is not less than the second threshold, then there is an overexposed area in the first image area.

[0095] For example, auxiliary feature judgment, as a supplement to the pixel constraint method, uses scene characteristics and pixel features of bad pixel blocks as auxiliary features to improve the accuracy of bad pixel judgment. Since bad pixel blocks appear at the edge of highly reflective areas, that is, there are overexposed areas in the bad pixel neighborhood, and at the same time, the pixel value of the bad pixel itself is much lower than the surrounding pixel values, based on the above principle, the above features (i.e., the presence of overexposed areas in the bad pixel neighborhood and the bad pixel value itself is much lower than the surrounding pixel values) can be used as an auxiliary judgment method for the pixel constraint method. The auxiliary judgment method is used to detect whether the pixel point belongs to a bad pixel block, effectively improving the accuracy of bad pixel block detection.

[0096] The existence of an overexposed area in the neighborhood of a bad pixel means that: if the pixel value of each pixel in the first image area is less than a second threshold (which can be configured based on experience and can be a relatively large pixel value, that is, the second threshold is greater than the first threshold), then there is no overexposed area in the neighborhood of the bad pixel; if the pixel value of at least one pixel in the first image area is not less than the second threshold, then there is an overexposed area in the neighborhood of the bad pixel.

[0097] The bad pixel's own pixel value is far lower than the surrounding pixel values. This means that if the pixel value of the pixel point P(i, j) is less than a first threshold (which can be configured based on experience and can be a relatively small pixel value), it means that the pixel value of the pixel point P(i, j) itself is far lower than the surrounding pixel values; if the pixel value of the pixel point P(i, j) is not less than the first threshold, it means that the pixel value of the pixel point P(i, j) itself is not far lower than the surrounding pixel values.

[0098] At this point, step 303 is completed, and the pixel to be corrected in the original image can be located. The pixel to be corrected is an abnormal pixel belonging to the bad pixel block. After a pixel to be corrected is found, the subsequent step 304 is executed for the pixel to be corrected. For example, each pixel in the original image is traversed in sequence. If the pixel currently traversed is not a pixel to be corrected, the next pixel is traversed. If the pixel is a pixel to be corrected, the subsequent step 304 is executed for the pixel to be corrected.

[0099] Through step 3031, step 3032 and step 3033, the bad pixel block area can be effectively located and the pixels to be corrected can be found, which facilitates the subsequent correction of the bad pixel block (ie, the pixels to be corrected).

[0100] Step 304: perform bad pixel block correction on the bad pixel blocks in the original image.

[0101] For example, since bad pixels are distributed in large numbers within a certain range and stick together into blocks, there are many invalid pixels within a certain neighborhood range. The information provided by pixels close to the bad pixels is unreliable, while normal pixel blocks exist in the neighborhood area far from the correction center. The detail information contained in the normal pixel blocks is similar to the detail information lost in the bad pixel block area. Therefore, the normal pixel blocks can be migrated to the bad pixel block area by the overall migration and fusion of pixel blocks, and interpolated and fused with the original pixels in the bad pixel block area to improve pixel smoothness. Based on the above principle, in order to correct bad pixel blocks in the original image, the pixel block selection direction can be preferentially determined, the target pixel block can be selected based on the pixel block selection direction, and the pixel values ​​of the bad pixel block can be interpolated and fused based on the pixel values ​​of the target pixel block, thereby correcting the bad pixel block. Obviously, by using the pixel migration method, the pixel values ​​of pixel blocks far from the bad pixel block area can be migrated as a whole, and interpolated and fused with the original pixels, thereby obtaining a more accurate correction value.

[0102] For example, the following steps can be used to correct bad pixel blocks in the original image:

[0103] Step 3041: Determine the pixel migration direction. For example, for a pixel to be corrected in the original image (the pixel to be corrected belongs to a bad pixel block), obtain a first image region corresponding to the pixel to be corrected from the original image, divide the first image region into K pixel blocks, and select a target pixel block from the K pixel blocks based on the pixel features corresponding to each pixel block. The target pixel block is the pixel block in the pixel migration direction.

[0104] For example, after obtaining the pixel to be corrected from the original image, a first image area can be obtained, the central pixel of the first image area is the pixel to be corrected, the size of the first image area is m*m, the first image area has been obtained in step 303, and the first image area can be used in step 304.

[0105] The first image area can be divided into K pixel blocks, which form a plurality of pixel block sets, each of which includes two pixel blocks. For example, the first image area can be divided into 4 pixel blocks, 6 pixel blocks, 8 pixel blocks, or 10 pixel blocks, etc., with no restriction on the number of pixel blocks. For each pixel block set, a line connecting the center pixels of two pixel blocks within the pixel block set passes through the pixel to be corrected. Of course, the line connecting the center pixels of the two pixel blocks may not pass through the pixel to be corrected, as long as the vertical distance between the pixel to be corrected and the center pixel is less than a preset threshold, and this is not restricted.

[0106] For example, taking the first image area divided into 8 pixel blocks, see Figure 5 As shown in FIG, an example of the first image area is shown, that is, the first image area is divided into 8 pixel blocks using the eight-neighborhood method, and the central pixel point of the first image area is the pixel point to be corrected. The 8 pixel blocks are Block1, Block2, Block3, Block4, Block5, Block6, Block7, and Block8. These 8 pixel blocks can be pixel blocks of the same size or pixel blocks of different sizes. These 8 pixel blocks may have overlapping pixels (such as Block1 and Block2 have overlapping pixels, Block1 and Block4 have overlapping pixels, Block2 and Block3 have overlapping pixels, and so on). Figure 5 This overlap is not shown in the figure), and these 8 pixel blocks may not have overlapping pixels (in Figure 5 This overlap relationship is shown in FIG. ). Regarding the size of each pixel block, there is no limitation in this embodiment, as long as it is smaller than the size of the first image area. For example, if the size of the first image area is 9*9, the size of each pixel block can be 3*3 or 5*5.

[0107] See also Figure 5 As shown, Block1 and Block8 constitute pixel block set 1, Block2 and Block7 constitute pixel block set 2, Block3 and Block6 constitute pixel block set 3, and Block4 and Block5 constitute pixel block set 4. Of course, the first image area may include all pixel block sets in pixel block set 1, pixel block set 2, pixel block set 3, and pixel block set 4, and may also include some pixel block sets.

[0108] Obviously, the line connecting the center pixels of Block 1 and Block 8 in pixel block set 1 passes through the pixel to be corrected, the line connecting the center pixels of Block 2 and Block 7 in pixel block set 2 passes through the pixel to be corrected, the line connecting the center pixels of Block 3 and Block 6 in pixel block set 3 passes through the pixel to be corrected, and the line connecting the center pixels of Block 4 and Block 5 in pixel block set 4 passes through the pixel to be corrected.

[0109] Exemplarily, for each pixel block set, the gradient value corresponding to the pixel block set can be determined based on the pixel features corresponding to the two pixel blocks in the pixel block set. The pixel features corresponding to the pixel block can be the average value, median value, maximum value, or minimum value of the pixel values ​​of all the pixels in the pixel block. Of course, the above are just a few examples and there is no limitation on this pixel feature. The average value of the pixel values ​​of all the pixels in the pixel block is taken as an example. Based on this, the average value P1 of the pixel values ​​of all the pixels in Block1 is calculated, the average value P2 of the pixel values ​​of all the pixels in Block2 is calculated, the average value P3 of the pixel values ​​of all the pixels in Block3 is calculated, the average value P4 of the pixel values ​​of all the pixels in Block4 is calculated, the average value P5 of the pixel values ​​of all the pixels in Block5 is calculated, the average value P6 of the pixel values ​​of all the pixels in Block6 is calculated, the average value P7 of the pixel values ​​of all the pixels in Block7 is calculated, and the average value P8 of the pixel values ​​of all the pixels in Block8 is calculated.

[0110] Then, based on the pixel features corresponding to Block 1 and Block 8 in pixel block set 1, the gradient value corresponding to pixel block set 1 (i.e., the pixel block set in the northwest direction) is determined using the following formula: WN :Gradient WN =|P1-P8|. Based on the pixel features corresponding to Block2 and Block7 in pixel block set 2, the gradient value corresponding to pixel block set 2 (i.e., the vertical pixel block set) is determined using the following formula: V :Gradient V =|P2-P7|. Based on the pixel features corresponding to Block3 and Block6 in pixel block set 3, the gradient value corresponding to pixel block set 3 (i.e., the pixel block set in the northeast direction) is determined using the following formula: EN :Gradient EN=|P3-P6|. Based on the pixel features corresponding to Block4 and Block5 in pixel block set 4, the gradient value corresponding to pixel block set 4 (i.e., the horizontal pixel block set) is determined using the following formula: H :Gradient H =|P5-P4|.

[0111] For example, based on the gradient values ​​corresponding to each pixel block set, the pixel block set corresponding to the maximum gradient value can be selected. Thus, the pixel block with the smallest pixel feature within the pixel block set corresponding to the maximum gradient value is selected as the target pixel block. Thus, the target pixel block corresponding to the pixel to be corrected is successfully selected. The target pixel block is the pixel block with the greatest similarity to the pixel to be corrected, i.e., the pixel block with the smallest pixel feature within the pixel block set corresponding to the maximum gradient value is the pixel block with the greatest similarity to the pixel to be corrected.

[0112] For example, Gradient H It is the horizontal gradient (the gradient value corresponding to the horizontal pixel block set). V It is the vertical gradient (the gradient value corresponding to the vertical pixel block set). EN is the northeast direction gradient (the gradient value corresponding to the pixel block set in the northeast direction), WN is the northwest gradient (the gradient value corresponding to the set of pixel blocks in the northwest direction). Based on this, the maximum gradient value can be expressed as Graident max =MAX(Graident H ,Graident V ,Graident EN ,Graident WN ).

[0113] If the maximum gradient value is Graident H , then a target pixel block is selected from the horizontal pixel block set 4. When P5>P4, the target pixel block is determined to be Block4. Otherwise, the target pixel block is determined to be Block5.

[0114] If the maximum gradient value is Graident V , then a target pixel block is selected from the pixel block set 2 in the vertical direction. When P2>P7, the target pixel block is determined to be Block7, otherwise, the target pixel block is determined to be Block2.

[0115] If the maximum gradient value is Graident EN, then the target pixel block is selected from the pixel block set 3 in the northeast direction. When P3>P6, the target pixel block is determined to be Block6, otherwise, the target pixel block is determined to be Block3.

[0116] If the maximum gradient value is Graident WN , then the target pixel block is selected from the pixel block set 1 in the northwest direction. When P1>P8, the target pixel block is determined to be Block8, otherwise, the target pixel block is determined to be Block1.

[0117] At this point, the target pixel block corresponding to the pixel point to be corrected is successfully determined from all pixel blocks.

[0118] Step 3042: Pixel interpolation and fusion. For example, when correcting the pixel values ​​of the pixels to be corrected, a second image region can be obtained from the original image, and the pixel value of each pixel in the second image region can be corrected based on the pixel features corresponding to the target pixel block. In other words, the second image region can be understood as a bad pixel block, and the pixel value of each pixel in the bad pixel block can be corrected.

[0119] Among them, the second image area is used to determine the pixel points that need to be corrected, the first image area is used to determine the target pixel block (see step 3041 for the process of determining the target pixel block based on the first image area), and the target pixel block is used to correct the pixel points in the second image area, that is, the pixel value of each pixel point in the second image area is corrected based on the pixel features corresponding to the target pixel block.

[0120] Exemplarily, the central pixel of the second image region is the pixel to be corrected, and the size of the second image region is n*n, where the value of n is configured based on experience and can be less than m, without limitation. The value of n can be between a minimum value and a maximum value, where the minimum value can be configured based on experience, such as 2 or 3, and the maximum value can be configured based on experience, such as 4 or 5. For example, taking the minimum value as 3 and the maximum value as 5 as an example, then n is greater than or equal to 3 and less than or equal to 5. For example, n can be 3, n can be 4, and n can be 5.

[0121] The size (n*n) of the second image region can be determined based on the target gain value corresponding to the original image. For example, assuming n has K size values, all gain values ​​are divided into K gain value intervals, with gain value interval 1 corresponding to the first size value of n, and so on, gain value interval K corresponding to the Kth size value of n. After obtaining the target gain value corresponding to the original image, the target gain value interval to which the target gain value belongs is first determined, and the size value corresponding to the target gain value interval is used as the size of the second image region.

[0122] The size (n*n) of the second image area can be determined based on the target gain value corresponding to the original image. Exemplarily, assuming that n has K size values, all gain values ​​are divided into K gain value intervals. A first mapping relationship can be pre-configured, and the first mapping relationship includes a correspondence between a gain value interval and an intensity control parameter. A third mapping relationship can be pre-configured, and the third mapping relationship includes a correspondence between an intensity control parameter and a second size. When the intensity control parameter is larger, the second size corresponding to the intensity control parameter is larger. The third mapping relationship can be a mapping table, a mapping function, or a mapping curve, and there is no limitation on this. Based on this, the target gain value interval to which the target gain value belongs can be determined; by querying the first mapping relationship, the target intensity control parameter corresponding to the target gain value interval is obtained; then, by querying the third mapping relationship, the second target size corresponding to the target intensity control parameter is obtained, and the second target size is the size of the second image area, and the second target size of the second image area is smaller than the first target size of the first image area.

[0123] For example, assuming n has three size values, namely 3, 4, and 5, all gain values ​​are divided into gain value interval 1, gain value interval 2, and gain value interval 3. The first mapping relationship may include the correspondence between gain value interval 1 and intensity control parameter 1, the correspondence between gain value interval 2 and intensity control parameter 2, and the correspondence between gain value interval 3 and intensity control parameter 3. The third mapping relationship may include the correspondence between intensity control parameter 1 and size value 3 (i.e., the second size), the correspondence between intensity control parameter 2 and size value 4, and the correspondence between intensity control parameter 3 and size value 5. After obtaining the target gain value, if the target gain value belongs to gain value interval 1, the intensity control parameter 1 corresponding to gain value interval 1 is obtained by querying the first mapping relationship. The size value 3 corresponding to intensity control parameter 1 is obtained by querying the third mapping relationship, and size value 3 is used as the size of the second image area, i.e., the second image area is a 3*3 image area.

[0124] For a pixel to be corrected in the original image, a second image region with the pixel to be corrected as a center pixel may be acquired from the original image based on the size (n*n) of the second image region.

[0125] For example, the pixel features corresponding to the target pixel block can be determined. The pixel features corresponding to the target pixel block can be the average value, median value, maximum value, or minimum value of the pixel values ​​of all pixels in the target pixel block. Of course, the above are just a few examples and there is no limitation on this pixel feature. The median value of all pixels in the target pixel block is used as an example for explanation. Assuming that the original image is a Bayer format image, the target pixel block (denoted as Block i) may include pixel values ​​of the B channel, the Gb channel, the Gr channel, and the R channel. Based on this, the median B_val of all B channel pixel values ​​in the target pixel block can be calculated using the following formula: B_val = median(Block i (B)); The median value Gb_val of all Gb channel pixel values ​​in the target pixel block can be calculated using the following formula: Gb_val = median(Block i (Gb)); The median Gr_val of all Gr channel pixel values ​​in the target pixel block can be calculated using the following formula: Gr_val = median(Block i (Gr)); The median R_val of all R channel pixel values ​​in the target pixel block can be calculated using the following formula: R_val = median(Block i (R)).

[0126] Exemplarily, after obtaining the pixel features corresponding to the target pixel block, such as B_val, Gb_val, Gr_val, R_val, etc., the pixel value of each pixel point in the second image area can be corrected based on the pixel features corresponding to the target pixel block. For example, for each pixel point in the second image area, the corrected pixel value of the pixel point is determined based on the pixel features corresponding to the target pixel block and the pixel value of the pixel point.

[0127] For example, after determining the pixel to be corrected, a second image area corresponding to the pixel to be corrected can be obtained from the original image. The second image area can be used as a bad point block, and the pixel value of each pixel in the bad point block can be corrected, thereby correcting the bad point block using the regional correction method.

[0128] Assuming that the coordinates of the pixel to be corrected are (i, j) and the size of the second image area is n*n, the coordinate range of the second image area can be arrive Each pixel point within the coordinate range may be corrected, including the pixel point (i, j) to be corrected.

[0129] For example, for each pixel in the second image area, the corrected pixel value of the pixel can be determined using the following formula: In the above formula, Replace represents the corrected pixel value of the pixel, P represents the uncorrected pixel value of the pixel, that is, the pixel value corresponding to the pixel in the original image, val represents the pixel feature corresponding to the target pixel block (such as the median value of all pixels), 255 represents the maximum pixel value, and can also be set to other values.

[0130] For example, for each pixel in the second image area, if the pixel is a pixel of the B channel, the corrected pixel value of the pixel can be determined using the following formula: B_val represents the median value of all B channel pixel values ​​in the target pixel block. If the pixel point is a Gb channel pixel point, the corrected pixel value of the pixel point can be determined using the following formula: Gb_val represents the median value of all Gb channel pixel values ​​in the target pixel block. If the pixel point is a Gr channel pixel point, the corrected pixel value of the pixel point can be determined using the following formula: Gr_val represents the median value of all Gr channel pixel values ​​in the target pixel block. If the pixel point is a pixel point of the R channel, the corrected pixel value of the pixel point can be determined using the following formula: R_val represents the median value of all R channel pixel values ​​in the target pixel block.

[0131] At this point, the correction of the pixel to be corrected is completed, that is, the pixel to be corrected can be used as the central pixel of the bad pixel block (ie, the second image area) to complete the correction process of the bad pixel block.

[0132] After the correction of the pixel to be corrected is completed, the next pixel of the pixel to be corrected is traversed, step 303 is performed for the next pixel, and steps 303 and 304 are repeated until the traversal of the original image is completed.

[0133] Step 305: Output the corrected image. For example, for each pixel in the original image, if the pixel is corrected, the corrected image includes the corrected pixel value of the pixel; if the pixel is not corrected, the corrected image includes the original pixel value of the pixel. Figure 6A The following is an example of the original image before correction, see Figure 6B Shown is an example of the corrected image.

[0134] It can be seen from the above technical solution that in the embodiment of the present application, the first image area is divided into K pixel blocks, and a target pixel block is selected from the K pixel blocks based on the pixel features corresponding to each pixel block, and the pixel values ​​of the pixels to be corrected are corrected based on the pixel features corresponding to the target pixel blocks to obtain a corrected image. By correcting the pixel values ​​of the pixels to be corrected (i.e., bad pixels), the image quality can be improved, the image clarity is improved, the image details are not lacking or the missing image details are reduced, the image integrity is better, the problem of serious loss of detail information is solved, and the image detail restoration and image integrity are improved. The gain linkage method is used to solve the risk of expansion of the bad pixel block area caused by the change of sensor gain, and pixel migration and interpolation fusion are proposed to solve the problem of inaccurate interpolation data and solve the image grading problem caused by single data interpolation.

[0135] Based on the same application concept as the above-mentioned method, an embodiment of the present application provides an image correction method. The method may include: obtaining a first image region from an original image, wherein the first image region includes pixels to be corrected and an overexposed image region, or the first image region includes pixels to be corrected and the first image region and the overexposed image region are connected. The process of obtaining the first image region can be seen in steps 301, 302, and 303, and will not be repeated here.

[0136] A target pixel block is extracted from the first image region, wherein the target pixel block is any pixel block in the first image region, and the target pixel block is related to the gradient value of the pixel block and the pixel value of the pixel block. For example, a block pixel value of each pixel block in the first image region is obtained, wherein the block pixel value is represented by any of an average value, a median value, a maximum value, and a minimum value; gradient values ​​in the horizontal, vertical, and diagonal directions are obtained based on the block pixel value; and the pixel block corresponding to the minimum block pixel value in the direction of the maximum gradient value is used as the target pixel block, where the target pixel block is the pixel block with the greatest similarity to the pixel point to be corrected. The process of obtaining the target pixel block can be referred to in step 3041 and will not be repeated here.

[0137] Based on the pixel features of the target pixel block, the pixel values ​​of the pixels to be corrected are corrected to obtain corrected pixel values. For example, the block correction size is obtained, and the coordinate range of the area to be corrected is determined based on the block correction size and the pixels to be corrected; based on the pixel features of the target pixel block, the pixel values ​​of all pixels within the coordinate range of the area to be corrected are corrected to obtain corrected pixel values; wherein the pixel features of the target pixel block are represented by any of the average value, median value, maximum value and minimum value of each pixel in the target pixel block. wherein, the process of correcting the pixel values ​​of the pixels to be corrected can be referred to step 3042, which will not be repeated here. wherein, the coordinate range of the area to be corrected can be the second image area of ​​the above embodiment.

[0138] A corrected image is generated based on the corrected pixel values. This process can be seen in step 305 .

[0139] Based on the same application concept as the above method, an image correction device is proposed in the embodiment of the present application, see Figure 7 FIG. 1 is a schematic diagram of the structure of the image correction device, which may include:

[0140] An acquisition module 71 is configured to acquire a pixel to be corrected in an original image, obtain a first image region corresponding to the pixel to be corrected from the original image, wherein the central pixel of the first image region is the pixel to be corrected; divide the first image region into K pixel blocks, where K is a positive integer greater than 1; wherein the pixel to be corrected is an abnormal pixel belonging to a bad pixel block;

[0141] A processing module 72 is configured to select a target pixel block from the K pixel blocks based on the pixel features corresponding to each pixel block; and correct the pixel value of the pixel point to be corrected based on the pixel features corresponding to the target pixel block to obtain a corrected pixel value of the pixel point to be corrected; wherein the target pixel block is the pixel block having the greatest similarity to the pixel point to be corrected;

[0142] The generating module 73 is configured to generate a corrected image based on the corrected pixel values ​​of the pixels to be corrected.

[0143] Exemplarily, the K pixel blocks constitute a plurality of pixel block sets, and a line connecting the center pixel points of two pixel blocks in the pixel block set passes through the pixel point to be corrected; when the processing module 72 selects a target pixel block from the K pixel blocks based on the pixel features corresponding to each pixel block, it is specifically used to: determine the gradient value corresponding to the pixel block set based on the pixel features corresponding to the two pixel blocks in the pixel block set; wherein the pixel feature corresponding to the pixel block is the average value, median value, maximum value, or minimum value of the pixel values ​​of all pixel points in the pixel block; based on the gradient value corresponding to each pixel block set, select the pixel block with a smaller pixel feature in the pixel block set corresponding to the maximum gradient value as the target pixel block.

[0144] Exemplarily, the processing module 72 corrects the pixel value of the pixel to be corrected based on the pixel feature corresponding to the target pixel block, and is specifically used to obtain the corrected pixel value of the pixel to be corrected: obtaining a second image area from the original image, the central pixel point of the second image area is the pixel to be corrected, and the size of the second image area is smaller than the size of the first image area; wherein the second image area is used to determine the pixel points to be corrected, the first image area is used to determine the target pixel block, and the target pixel block is used to correct the pixel points of the second image area; for each pixel point in the second image area, the pixel value of the pixel point is corrected based on the pixel feature corresponding to the target pixel block to obtain the corrected pixel value of the pixel point; the pixel feature corresponding to the target pixel block is the average value, or median value, or maximum value, or minimum value of the pixel values ​​of all pixels in the target pixel block.

[0145] Exemplarily, the size of the first image area is determined based on a target gain value corresponding to the original image; wherein the target gain value is a gain value used when the original image is captured by a sensor; and the size of the second image area is determined based on the target gain value.

[0146] Exemplarily, the acquisition module 71 is further used to determine the target gain value interval to which the target gain value belongs; by querying the first mapping relationship, the target intensity control parameter corresponding to the target gain value interval is obtained; wherein, the first mapping relationship includes the correspondence between the gain value interval and the intensity control parameter, and the larger the gain value interval is, the larger the intensity control parameter corresponding to the gain value interval is; by querying the second mapping relationship, the first target size corresponding to the target intensity control parameter is obtained, and the first target size is the size of the first image area; by querying the third mapping relationship, the second target size corresponding to the target intensity control parameter is obtained, and the second target size is the size of the second image area, and the second target size is smaller than the first target size; wherein, the second mapping relationship includes the correspondence between the intensity control parameter and the first size, and the larger the intensity control parameter is, the larger the first size corresponding to the intensity control parameter is; the third mapping relationship includes the correspondence between the intensity control parameter and the second size, and the larger the intensity control parameter is, the larger the second size corresponding to the intensity control parameter is.

[0147] Exemplarily, when the acquisition module 71 acquires the pixel point to be corrected in the original image, it is specifically used to: for the pixel point in the original image, acquire the first image area corresponding to the pixel point from the original image, select the adjacent pixel point corresponding to the pixel point from the first image area, and determine the upper limit pixel value and lower limit pixel value corresponding to the pixel point based on the pixel values ​​of the adjacent pixel points; if the pixel value of the pixel point is between the upper limit pixel value and the lower limit pixel value, it is determined that the pixel point is not the pixel point to be corrected; if the pixel value of the pixel point is not between the upper limit pixel value and the lower limit pixel value, it is determined that the pixel point is the pixel point to be corrected, or, based on the auxiliary features corresponding to the first image area, determine that the pixel point is not the pixel point to be corrected or is the pixel point to be corrected.

[0148] Exemplarily, if the auxiliary feature includes the pixel value of each pixel in the first image area, the acquisition module 71 is specifically used to determine whether the pixel is a pixel to be corrected or not based on the auxiliary feature corresponding to the first image area: if the pixel value of the pixel is less than a first threshold, and there is an overexposed area in the first image area, then the pixel is determined to be a pixel to be corrected; if the pixel value of the pixel is not less than the first threshold, and / or there is no overexposed area in the first image area, then the pixel is determined not to be a pixel to be corrected; wherein, if the pixel value of each pixel in the first image area is less than the second threshold, and the second threshold is greater than the first threshold, then there is no overexposed area in the first image area; if the pixel value of at least one pixel in the first image area is not less than the second threshold, then there is an overexposed area in the first image area.

[0149] Based on the same application concept as the above method, an image correction device is proposed in an embodiment of the present application, including: an acquisition module, used to acquire a first image area from an original image, the first image area includes pixels to be corrected and an overexposed image area, or the first image area includes pixels to be corrected, and the first image area is connected to the overexposed image area; an extraction module, used to extract a target pixel block from the first image area, the target pixel block is any pixel block in the first image area, and the target pixel block is related to the gradient value of the pixel block and the pixel value of the pixel block; a correction module, used to correct the pixel value of the pixel point to be corrected based on the pixel characteristics of the target pixel block to obtain a corrected pixel value; a generation module, used to generate a corrected image based on the corrected pixel value.

[0150] Exemplarily, when the extraction module extracts the target pixel block from the first image area, it is specifically used to: obtain the block pixel value of each pixel block in the first image area, wherein the block pixel value is represented by any one of an average value, a median value, a maximum value and a minimum value; obtain the gradient values ​​in the horizontal direction, the vertical direction and the diagonal direction respectively according to the block pixel value; and take the pixel block corresponding to the minimum block pixel value in the direction of the maximum gradient value as the target pixel block.

[0151] Exemplarily, the correction module corrects the pixel values ​​of the pixel points to be corrected based on the pixel characteristics of the target pixel block, and when obtaining the corrected pixel values, is specifically used to: obtain a block correction size, and determine a coordinate range of the area to be corrected based on the block correction size and the pixel points to be corrected; correct the pixel values ​​of all pixels within the coordinate range of the area to be corrected based on the pixel characteristics of the target pixel block, and obtain the corrected pixel values; wherein the pixel characteristics of the target pixel block are represented by any one of the average value, median value, maximum value and minimum value of each pixel point in the target pixel block.

[0152] Based on the same application concept as the above method, an electronic device is proposed in the embodiment of the present application, see Figure 8 As shown, the electronic device includes: a processor 81 and a machine-readable storage medium 82, wherein the machine-readable storage medium 82 stores machine-executable instructions that can be executed by the processor 81; the processor 81 is used to execute the machine-executable instructions to implement the image correction method disclosed in the above example of this application.

[0153] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the image correction method disclosed in the above example of the present application can be implemented.

[0154] The machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0155] The systems, devices, modules, or units described in the above embodiments may be implemented by a computer entity or by a product having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0156] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0157] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0158] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0159] Furthermore, these computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0161] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. An image correction method, characterized in that: The method comprises: Obtaining a pixel to be corrected in an original image, and obtaining a first image region corresponding to the pixel to be corrected from the original image, wherein a central pixel of the first image region is the pixel to be corrected; wherein the pixel to be corrected is an abnormal pixel belonging to a bad pixel block; Dividing the first image area into K pixel blocks, where K is a positive integer greater than 1; Selecting a target pixel block from the K pixel blocks based on pixel features corresponding to each pixel block; wherein the target pixel block is the pixel block having the greatest similarity to the pixel point to be corrected; Correcting the pixel value of the pixel to be corrected based on the pixel feature corresponding to the target pixel block to obtain a corrected pixel value of the pixel to be corrected; generating a corrected image based on the corrected pixel values ​​of the pixels to be corrected; The K pixel blocks form a plurality of pixel block sets, and a line connecting central pixel points of two pixel blocks in the pixel block set passes through the pixel point to be corrected; and selecting a target pixel block from the K pixel blocks based on the pixel features corresponding to each pixel block includes: Determine a gradient value corresponding to the pixel block set based on pixel features corresponding to two pixel blocks in the pixel block set; wherein the pixel feature corresponding to the pixel block is an average value, a median value, a maximum value, or a minimum value of pixel values ​​of all pixels in the pixel block; Based on the gradient value corresponding to each pixel block set, a pixel block with a smaller pixel feature in the pixel block set corresponding to the maximum gradient value is selected as the target pixel block.

2. The method according to claim 1, characterized in that Correcting the pixel value of the pixel to be corrected based on the pixel feature corresponding to the target pixel block to obtain the corrected pixel value of the pixel to be corrected includes: Acquire a second image area from the original image, wherein a central pixel point of the second image area is the pixel point to be corrected, and a size of the second image area is smaller than a size of the first image area; wherein the second image area is used to determine the pixel points to be corrected, and the first image area is used to determine a target pixel block, and the target pixel block is used to correct the pixel points of the second image area; For each pixel point in the second image area, correcting the pixel value of the pixel point based on the pixel feature corresponding to the target pixel block to obtain a corrected pixel value of the pixel point; The pixel feature corresponding to the target pixel block is the average value, median value, maximum value, or minimum value of the pixel values ​​of all pixels in the target pixel block.

3. The method according to claim 2, characterized in that The size of the first image area is determined based on a target gain value corresponding to the original image; wherein the target gain value is a gain value used when the original image is acquired by a sensor; The size of the second image area is determined based on the target gain value.

4. The method according to claim 3, characterized in that The method further comprises: determining a target gain value interval to which the target gain value belongs; Obtaining a target intensity control parameter corresponding to the target gain value interval by querying a first mapping relationship; wherein the first mapping relationship includes a correspondence between the gain value interval and the intensity control parameter, and the larger the gain value interval, the larger the intensity control parameter corresponding to the gain value interval; By querying the second mapping relationship, a first target size corresponding to the target intensity control parameter is obtained, where the first target size is the size of the first image area; and by querying the third mapping relationship, a second target size corresponding to the target intensity control parameter is obtained, where the second target size is the size of the second image area, and the second target size is smaller than the first target size. The second mapping relationship includes a correspondence between an intensity control parameter and a first size, and the larger the intensity control parameter is, the larger the first size corresponding to the intensity control parameter is; The third mapping relationship includes a correspondence between the intensity control parameter and the second size. When the intensity control parameter is larger, the second size corresponding to the intensity control parameter is larger.

5. The method according to any one of claims 1 to 4, characterized in that The step of obtaining the pixel points to be corrected in the original image includes: For a pixel point in the original image, obtain a first image region corresponding to the pixel point from the original image, select adjacent pixels corresponding to the pixel point from the first image region, and determine an upper limit pixel value and a lower limit pixel value corresponding to the pixel point based on pixel values ​​of the adjacent pixels; If the pixel value of the pixel point is between the upper limit pixel value and the lower limit pixel value, it is determined that the pixel point is not the pixel point to be corrected; if the pixel value of the pixel point is not between the upper limit pixel value and the lower limit pixel value, it is determined that the pixel point is the pixel point to be corrected, or, based on the auxiliary features corresponding to the first image area, it is determined that the pixel point is not the pixel point to be corrected or is the pixel point to be corrected.

6. The method according to claim 5, characterized in that If the auxiliary feature includes a pixel value of each pixel in the first image area, determining, based on the auxiliary feature corresponding to the first image area, that the pixel is not a pixel to be corrected or is a pixel to be corrected includes: If the pixel value of the pixel point is less than a first threshold value and there is an overexposed area in the first image area, the pixel point is determined to be a pixel point to be corrected; if the pixel value of the pixel point is not less than the first threshold value and / or there is no overexposed area in the first image area, the pixel point is determined not to be a pixel point to be corrected; Among them, if the pixel value of each pixel point in the first image area is less than the second threshold, and the second threshold is greater than the first threshold, then there is no overexposed area in the first image area; if the pixel value of at least one pixel point in the first image area is not less than the second threshold, then there is an overexposed area in the first image area.

7. An image correction device, characterized in that: The device comprises: An acquisition module is configured to acquire a pixel to be corrected in an original image, obtain a first image region corresponding to the pixel to be corrected from the original image, wherein the central pixel of the first image region is the pixel to be corrected; divide the first image region into K pixel blocks, where K is a positive integer greater than 1; wherein the pixel to be corrected is an abnormal pixel belonging to a bad pixel block; a processing module, configured to select a target pixel block from the K pixel blocks based on pixel features corresponding to each pixel block; and correct the pixel value of the pixel point to be corrected based on the pixel features corresponding to the target pixel block to obtain a corrected pixel value of the pixel point to be corrected; wherein the target pixel block is the pixel block having the greatest similarity to the pixel point to be corrected; A generating module, configured to generate a corrected image based on the corrected pixel values ​​of the pixels to be corrected; Among them, K pixel blocks constitute a plurality of pixel block sets, and a line connecting the center pixel points of two pixel blocks in the pixel block set passes through the pixel point to be corrected; when the processing module selects a target pixel block from the K pixel blocks based on the pixel features corresponding to each pixel block, it is specifically used to: determine a gradient value corresponding to the pixel block set based on the pixel features respectively corresponding to the two pixel blocks in the pixel block set; wherein the pixel feature corresponding to the pixel block is the average value, median value, maximum value, or minimum value of the pixel values ​​of all pixel points in the pixel block; based on the gradient value corresponding to each pixel block set, select a pixel block with a smaller pixel feature in the pixel block set corresponding to the maximum gradient value as the target pixel block.

8. An image correction method, characterized in that: The method comprises: Acquire a first image region from the original image, wherein the first image region includes pixels to be corrected and an overexposed image region, or the first image region includes pixels to be corrected and the first image region is connected to the overexposed image region; Extracting a target pixel block from the first image area, wherein the target pixel block is any pixel block in the first image area, and the target pixel block is related to a gradient value of the pixel block and a pixel value of the pixel block; Correcting the pixel value of the pixel to be corrected based on the pixel characteristics of the target pixel block to obtain a corrected pixel value; generating a corrected image based on the corrected pixel values; The step of extracting a target pixel block from the first image area includes: Obtaining a block pixel value of each pixel block in the first image region, wherein the block pixel value is represented by any one of an average value, a median value, a maximum value, and a minimum value; Obtaining gradient values ​​in the horizontal direction, the vertical direction, and the diagonal direction respectively according to the block pixel value; The pixel block corresponding to the minimum block pixel value in the direction of the maximum gradient value is used as the target pixel block.

9. The method according to claim 8, characterized in that Correcting the pixel value of the pixel to be corrected based on the pixel feature of the target pixel block to obtain the corrected pixel value includes: Acquire a block correction size, and determine a coordinate range of a to-be-corrected area based on the block correction size and the to-be-corrected pixel points; Based on the pixel characteristics of the target pixel block, the pixel values ​​of all pixels within the coordinate range of the to-be-corrected area are corrected to obtain corrected pixel values; wherein the pixel characteristics of the target pixel block are represented by any one of the average value, median value, maximum value and minimum value of each pixel point in the target pixel block.

10. An electronic device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions capable of being executed by the processor; The processor is used to execute machine-executable instructions to implement the method described in any one of claims 1-6; or, the processor is used to execute machine-executable instructions to implement the method described in any one of claims 8-9.

Images