Image processing method, device and storage medium for suppressing false information

Abstract

The present application relates to an image processing method, device and storage medium for suppressing false information. The method comprises the following steps: obtaining a first image to be processed, determining a target position in the first image to be interpolated with a first channel, and determining an operation area with the target position as the center; the number of pixels of the first channel in the first image is not less than the number of pixels of the second channel; calculating first direction information of the original second channel pixels in the target position based on the operation area; inserting a first interpolation pixel at the target position according to the first direction information; inserting direction information of the first channel pixels after the first interpolation pixel according to the first direction information of the second channel pixels, and calculating second direction information of the target position; updating the first interpolation pixel according to the second direction information; interpolating the second channel based on the first channel pixels obtained after updating the first interpolation pixel to obtain a second image; suppressing false information in the process of de-mosaicing, and improving image clarity without increasing the amount of calculation.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method, device and storage medium for suppressing false information. Background Art

[0002] With the continuous improvement of medical surgery technology, the target part is often observed with the help of shooting and display systems during surgery. The quality of medical images and videos has become one of the concerns in medical systems. Medical equipment usually uses CMOS and other camera devices to collect Bayer-arranged image data, referred to as Bayer data. Bayer format data is composed of incomplete RGB data arranged at intervals. It is necessary to use the existing pixel values ​​in the surrounding information to interpolate and complete the RGB three-channel data. This interpolation and completion process is also called de-mosaicing.

[0003] In the process of demosaicing, the original Bayer image includes red pixel values, blue pixel values ​​and green pixel values. The two missing color components of each pixel are obtained by interpolation (for example, the blue component and red component of the green pixel are interpolated), so that each pixel includes red component, green component and blue component, thereby converting the Bayer image into an RGB image. However, after the demosaicing process, additional pseudo information that does not exist in the original image will be introduced, such as false color, moiré, bad pixels, etc., thereby affecting the clarity of the image. In the related art, for the pseudo information of the image, one method is to judge and remove it after the interpolation is completed, but this will add additional judgment and elimination processes. One method is to find similar blocks in the neighborhood for weighting and replacement, but the calculation amount of this solution is also large.

[0004] At present, no effective solution has been proposed for the problem that removing false information from images requires a large amount of computation in related technologies. Summary of the invention

[0005] Based on this, it is necessary to provide an image processing method, device and storage medium that can suppress false information without increasing the amount of calculation in order to solve the above technical problems.

[0006] In a first aspect, the present application provides an image processing method for suppressing false information, comprising:

[0007] Acquire a first image to be processed, determine a target position in the first image where first channel interpolation is to be performed, and determine an operation area with the target position as the center; wherein the number of pixels in the first channel in the first image is not less than the number of pixels in the second channel;

[0008] Calculate first direction information of original second channel pixels in the target position based on the operation area;

[0009] inserting a first interpolation pixel at the target position according to the first direction information;

[0010] Calculate the second direction information of the target position according to the first direction information of the second channel pixel and the direction information of the first channel pixel after inserting the first interpolation pixel;

[0011] updating the first interpolation pixel according to the second direction information;

[0012] Based on the first channel pixels obtained after updating the first interpolation pixels, the second channel is interpolated to obtain a second image.

[0013] In one embodiment, calculating the first direction information of the original second channel pixel in the target position based on the operation area includes:

[0014] In the horizontal direction of the rows of the operation area, a first mean value of the absolute values ​​of the differences between adjacent pixels of the first channel is calculated, and in the horizontal direction of the rows of the operation area, a second mean value of the absolute values ​​of the differences between adjacent pixels of the second channel is calculated;

[0015] Adding the first mean value and the second mean value to obtain first horizontal direction information;

[0016] Calculating a third mean of absolute values ​​of differences between adjacent pixels of the first channel in a vertical direction of a column of the operation area, and calculating a fourth mean of absolute values ​​of differences between adjacent pixels of the second channel in a vertical direction of a column of the operation area;

[0017] Adding the third mean value and the fourth mean value to obtain first vertical direction information;

[0018] The first direction information is obtained according to the first horizontal direction information and the first vertical direction information.

[0019] In one embodiment, inserting a first interpolation pixel at the target position according to the first direction information includes:

[0020] comparing the first horizontal direction information with the first vertical direction information;

[0021] If the first horizontal direction information is less than the first vertical direction information, calculating the value of the first interpolated pixel along the row direction of the target position;

[0022] If the horizontal direction information of the first direction information is greater than the vertical direction information, calculating the value of the first interpolation pixel along the column direction of the target position;

[0023] If the horizontal direction information of the first direction information is equal to the vertical direction information, the value of the first interpolation pixel is calculated along the row direction and the column direction of the target position.

[0024] In one embodiment, calculating the second direction information of the target position according to the first direction information of the second channel pixel and the direction information of the first channel pixel after inserting the first interpolated pixel includes:

[0025] respectively calculating horizontal direction information of the second channel pixels and horizontal direction information of the first channel pixels;

[0026] Adding the horizontal direction information of the second channel pixel and the horizontal direction information of the first channel pixel to obtain second horizontal direction information;

[0027] respectively calculating vertical direction information of the second channel pixels and vertical direction information of the first channel pixels;

[0028] Adding the vertical direction information of the second channel pixel and the vertical direction information of the first channel pixel to obtain second vertical direction information;

[0029] The second direction information is obtained according to the second horizontal direction information and the second vertical direction information.

[0030] In one embodiment, respectively calculating the horizontal direction information of the second channel pixel and the horizontal direction information of the first channel pixel includes:

[0031] On the center row where the target position is located, the first horizontal direction information of the second channel pixels having the same channel attribute as the original pixel at the target position is added to obtain a first value;

[0032] On upper and lower neighboring rows of the central row, adding the mean values ​​of the first horizontal direction information of the second channel pixels having a channel attribute different from that of the original pixel to obtain a second value;

[0033] Adding the first value and the second value to obtain horizontal direction information of the second channel pixel;

[0034] On the central row and the upper and lower neighboring rows, the absolute values ​​of the differences between adjacent pixels of the first channel are calculated respectively, and the absolute values ​​of the differences between adjacent pixels of the first channel are added to obtain the horizontal direction information of the pixels of the first channel.

[0035] In one embodiment, respectively calculating the vertical direction information of the second channel pixel and the vertical direction information of the first channel pixel includes:

[0036] In the center column where the target position is located, the first vertical direction information of the second channel pixels having the same channel attribute as the original pixel at the target position is added to obtain a third value;

[0037] In left and right neighboring columns of the central column, average values ​​of the first vertical direction information of the second channel pixels having a channel attribute different from that of the original pixel are added to obtain a fourth value;

[0038] Adding the third value and the fourth value to obtain vertical direction information of the second channel pixel;

[0039] On the central column and the left and right neighboring columns, the absolute values ​​of the differences between adjacent pixels of the first channel are calculated respectively, and the absolute values ​​of the differences between adjacent pixels of the first channel are added to obtain the vertical direction information of the pixels of the first channel.

[0040] In one of the embodiments, after updating the first interpolation pixel according to the second direction information, the method further includes:

[0041] The following steps are executed in a loop until it is determined that the direction information of the current first interpolation pixel is consistent with the direction information of the second channel pixel:

[0042] Determining whether the updated direction information of the first interpolation pixel is consistent with the direction information of the second channel pixel;

[0043] If the updated direction information of the first interpolated pixel is inconsistent with the direction information of the second channel pixel, calculating new second direction information according to the first direction information of the second channel pixel and the direction information of the first channel pixel after the first interpolated pixel is updated;

[0044] The current first interpolation pixel is updated according to the new second direction information.

[0045] In one embodiment, determining whether the updated direction information of the first interpolation pixel is consistent with the direction information of the second channel pixel includes:

[0046] Calculating, based on the operation area, an average of absolute values ​​of differences between adjacent updated first interpolation pixels to obtain updated direction information of the first interpolation pixels;

[0047] Obtaining direction information of pixels of the second channel according to the second mean and the fourth mean;

[0048] Subtracting the updated direction information of the first interpolation pixel from the direction information of the second channel pixel to obtain a difference;

[0049] If the absolute value of the difference is less than a preset value, it is determined that the updated direction information of the first interpolation pixel is consistent with the direction information of the second channel pixel;

[0050] If the absolute value of the difference is not less than a preset value, it is determined that the updated direction information of the first interpolation pixel is inconsistent with the direction information of the second channel pixel.

[0051] In one embodiment, updating the first interpolation pixel according to the second direction information includes:

[0052] Determining pixel update information of the target position;

[0053] Determine pixel update information of a non-target position in the operation area according to the second direction information;

[0054] The first interpolation pixel is updated according to the pixel update information of the target position and the pixel update information of the non-target position to obtain a first interpolation pixel update value.

[0055] In one embodiment, after updating the first interpolated pixel according to the pixel update information of the target position and the pixel update information of the non-target position to obtain the first interpolated pixel update value, the method further includes:

[0056] comparing the first interpolated pixel update value of the target position with the first interpolated pixel update value adjacent to the target position;

[0057] If the first interpolation pixel update value of the target position is greater than the maximum value of the first interpolation pixel update values ​​adjacent to the target position, inserting the maximum value of the first interpolation pixel update values ​​adjacent to the target position into the target position;

[0058] If the first interpolation pixel update value of the target position is smaller than a minimum value of the first interpolation pixel update values ​​adjacent to the target position, the minimum value of the first interpolation pixel update values ​​adjacent to the target position is inserted into the target position.

[0059] In one embodiment, interpolating the second channel based on the first channel pixels obtained after updating the first interpolated pixels to obtain the second image includes:

[0060] Determine a second channel pixel that is adjacent to the position to be interpolated and has the same channel attribute as the second interpolation pixel to be inserted;

[0061] The second interpolation pixel is calculated based on the determined second channel pixel, the original pixel at the position to be interpolated, and the first interpolation pixel adjacent to the position to be interpolated and updated, and the second interpolation pixel is inserted into the position to be interpolated.

[0062] In a second aspect, the present application provides an image processing device for suppressing false information, the device comprising:

[0063] an acquisition module, used for acquiring a first image to be processed, determining a target position in the first image for first channel interpolation, and determining an operation area with the target position as the center; wherein the number of pixels in the first channel in the first image is not less than the number of pixels in the second channel;

[0064] A first direction information calculation module, used for calculating the first direction information of the original second channel pixel in the target position based on the operation area;

[0065] A first interpolation module, configured to insert a first interpolation pixel at the target position according to the first direction information;

[0066] A second direction information calculation module, used to calculate the second direction information of the target position according to the first direction information of the second channel pixel and the direction information of the first channel pixel after inserting the first interpolation pixel;

[0067] An updating module, configured to update the first interpolation pixel according to the second direction information;

[0068] The second interpolation module is used to interpolate the second channel based on the first channel pixels obtained after updating the first interpolation pixels to obtain a second image.

[0069] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0070] The above-mentioned image processing method, device and storage medium for suppressing false information determine the channel interpolation priority according to the amount of channel information in the RGB channel. For the priority interpolation channel, the initial first interpolation pixel is obtained by introducing the first direction information into the first channel interpolation process, and the false color, moiré and bad pixel problems are preliminarily eliminated. The first direction information is filtered according to the first channel pixel with full resolution to obtain the second direction information, and then the first interpolation pixel is updated based on the second direction, so that the interpolation result can be more consistent with the surrounding texture direction information, further reducing moiré and bad pixels; finally, the interpolation of the second channel is completed based on the updated first interpolation pixel. In this way, without increasing the calculation area, the false information caused by de-mosaicing is effectively suppressed, and the false information is suppressed in the process of de-mosaicing, and the clarity of the image is improved without increasing the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A hardware structure block diagram of a terminal of an image processing method for suppressing false information in an embodiment;

[0072] Figure 2 is a flowchart of an image processing method for suppressing false information in one embodiment;

[0073] Figure 3 Schematic diagram of input and output of de-mosaicing in one embodiment;

[0074] Figure 4 is a schematic diagram of interpolation for demosaicing in one embodiment;

[0075] Figure 5 is a schematic diagram of a first image in one embodiment;

[0076] Figure 6 is a schematic diagram of a first image in another embodiment;

[0077] Figure 7 is a schematic diagram of a first image in another embodiment;

[0078] Figure 8 is a flowchart of a first interpolation pixel updating method according to an embodiment;

[0079] Fig. 9 is a flowchart of an image processing method for suppressing false information in another embodiment;

[0080] Fig.10 FIG. 4 is a structural block diagram of an image processing device for suppressing false information in an embodiment. DETAILED DESCRIPTION

[0081] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0082] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0083] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure block diagram of a terminal of an image processing method for suppressing false information according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 101 and memory 102 for storing data, wherein processor 101 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 103 and an input and output device 104 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0084] The memory 102 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the image processing method for suppressing false information in this embodiment. The processor 101 executes various functional applications and data processing by running the computer program stored in the memory 102, that is, to implement the above method. The memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 102 may further include a memory remotely arranged relative to the processor 101, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0085] The transmission device 103 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 103 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 103 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0086] The related art provides a method for removing false colors, which divides the image into different areas according to the information entropy of the image area, and performs different false color removal methods for different areas, thereby achieving the purpose of removing false colors. However, this method performs false color removal after demosaicing, which increases the image processing flow and calculation amount.

[0087] Based on the analysis of the above situation, in this embodiment, an image processing method for suppressing false information is provided to solve image quality problems such as false color, moiré, bad pixels, etc. that occur in the de-mosaic processing in the ISP (Image Signal Processor) process. Figure 2 is a flow chart of the image processing method for suppressing false information, such as Figure 2 As shown, the process includes the following steps:

[0088] Step S101, obtaining a first image to be processed, determining a target position in the first image for first channel interpolation, and determining an operation area with the target position as the center; wherein the number of pixels in the first channel in the first image is not less than the number of pixels in the second channel.

[0089] Figure 3 This is a schematic diagram of the input and output of demosaicing, such as Figure 3As shown, the de-mosaicing process is to convert the input Bayer format data into the output data in RGB format. The conversion process is to supplement the missing data information, that is, to interpolate the R, G, and B channel information respectively. Bayer images can have four formats: RGGB, BGGR, GRGB, and GBGR, depending on the order of pixel position arrangement. This embodiment can input a Bayer image in any format and output an RGB image. For ease of understanding, this embodiment is described using the RGGB format. For the first image in RGGB format, the first channel is the G channel, and the second channel is the R or B channel (referred to as the R / B channel). It can be found that the number of pixels in the G channel is twice the number of pixels in the R / B channel.

[0090] Figure 4 is a schematic diagram of interpolation for demosaicing, such as Figure 4 As shown, in this process, the missing g data is interpolated in the G channel, the missing r data is interpolated in the R channel, and the missing b data is interpolated in the B channel. For the RGGB format, the G channel needs to be newly expanded by 1 times the amount of original data, and the R and B channels need to be newly expanded by 2 times the amount of original data. In this process, since the amount of information in the R and B channels is small, and the spectral response intensity of the R and B channels is not as strong as the spectral response intensity of the G channel, if the R / B channels are interpolated first and the G channel is interpolated later, the R / B channels are very prone to interpolation errors and color mutations, regional interference and other problems, and affect the processing of the subsequent ISP process, and ultimately appear on the display end. Image quality problems such as false colors, moiré patterns, and bad pixels that do not conform to the laws of reality. To solve the above problems, this embodiment interpolates the G channel first and then the R / B channels.

[0091] Figure 5 is a schematic diagram of the first image in this embodiment, such as Figure 5 As shown, red represents R channel pixels, green represents G channel pixels, blue represents B channel pixels, and the numbers represent the numbers of the corresponding channel pixels. Assuming that R0 is currently determined to be the target position for G channel interpolation, the N×N neighborhood information around R0 can be obtained to obtain a small operation area, where N can be 5, 7, 9, etc. As an example, N is 5.

[0092] Step S102: calculating first direction information of original second channel pixels in the target position based on the operation area.

[0093] The first direction information of the original second channel pixel is the first direction information of R0. When calculating the first direction information, the direction information of the G channel and the R / B channel around R0 can be used in the operation area to calculate the first direction information of R0. The first direction information includes the first horizontal direction information and the first vertical direction information.

[0094] Step S103: inserting a first interpolation pixel at the target position according to the first direction information.

[0095] According to the first direction information of R0 and the pixels around R0, the initial first interpolation pixel (g pixel) is calculated. The same operation is performed for other target positions to be interpolated for the first channel. The G channel is interpolated once according to the first direction information to fill in all the missing data in the G channel and obtain the full-resolution G pixel.

[0096] Step S104 , calculating second direction information of the target position according to the first direction information of the second channel pixel and the direction information of the first channel pixel after inserting the first interpolation pixel.

[0097] When calculating the second direction information, the first direction information of the R / B channel and the direction information of the g pixel in the previous step can be used in the operation area to calculate the second direction information of the target position, wherein the second direction information includes the second horizontal direction information and the second vertical direction information.

[0098] Step S105: update the first interpolated pixel according to the second direction information.

[0099] According to the second direction information of the target position and the pixels in the area around the target position, a new first interpolation pixel (g' pixel) is calculated. It should be noted that the causes of moiré and bad pixels include high-frequency noise interference. The second direction information is equivalent to the low-frequency information obtained after filtering the first direction information. The low-frequency information is subsequently used to update the first interpolation pixel, which can make the interpolation result more consistent with the surrounding texture direction information, thereby reducing moiré and bad pixels.

[0100] Step S106, interpolating the second channel based on the first channel pixels obtained after updating the first interpolated pixels to obtain a second image.

[0101] Interpolation of the second channel is to insert r pixels into the R channel and b pixels into the B channel to complete the missing data in the R / B channel. The color difference interpolation method can be used to interpolate the r and b pixels at the G channel position, and the pixels at the surrounding R / B channel positions are used to calculate.

[0102] In the above steps S101 to 106, the channel interpolation priority is determined according to the amount of channel information in the RGB channel. For the priority interpolation channel, the initial first interpolation pixel is obtained by introducing the first direction information into the first channel interpolation process, and the false color, moiré and bad pixel problems are preliminarily eliminated. The first direction information is filtered according to the first channel pixel with full resolution to obtain the second direction information, and then the first interpolation pixel is updated based on the second direction, so that the interpolation result can be more consistent with the surrounding texture direction information, further reducing moiré and bad pixels; finally, the interpolation of the second channel is completed based on the updated first interpolation pixel. In this way, without increasing the calculation area, the pseudo information caused by de-mosaicing is effectively suppressed, and the pseudo information is suppressed in the process of de-mosaicing, and the clarity of the image is improved without increasing the amount of calculation.

[0103] In one embodiment, the above step S102, calculating the first direction information of the original second channel pixel in the target position based on the operation area, can be implemented by the following method:

[0104] The first direction information may be obtained by adding the first horizontal direction information and the first vertical direction information.

[0105] (1) First horizontal direction information: In the horizontal direction of the rows of the operation area, calculate the first mean of the absolute values ​​of the differences between adjacent first channel pixels, and, in the horizontal direction of the rows of the operation area, calculate the second mean of the absolute values ​​of the differences between adjacent second channel pixels; add the first mean and the second mean to obtain the first horizontal direction information. Figure 5 , the specific calculation formula is as follows:

[0106]

[0107] Among them, H1 represents the first horizontal direction information, h1 represents the first mean value, and h2 represents the second mean value.

[0108] (2) First vertical direction information: In the vertical direction of the columns of the operation area, calculate the third mean of the absolute values ​​of the differences between adjacent first channel pixels, and, in the vertical direction of the columns of the operation area, calculate the fourth mean of the absolute values ​​of the differences between adjacent second channel pixels; add the third mean and the fourth mean to obtain the first vertical direction information. Figure 5 , the specific calculation formula is as follows:

[0109]

[0110] Among them, V1 represents the first vertical direction information, v1 represents the third mean, and v2 represents the fourth mean.

[0111] This embodiment combines the direction information of the first channel and the second channel when calculating the first direction information, so that the initial first direction information is more accurate.

[0112] In one embodiment, the above step S103, inserting the first interpolation pixel at the target position according to the first direction information, can be implemented by the following method:

[0113] The first horizontal direction information and the first vertical direction information are compared, and a corresponding interpolation method is selected according to the comparison result. Figure 6 is a schematic diagram of the first image in this embodiment, such as Figure 6 As shown, the results are divided into the following three cases:

[0114] (1) If the first horizontal direction information is less than the first vertical direction information, the value of the first interpolated pixel is calculated along the row direction of the target position. Exemplarily, the first interpolated pixel (g pixel) is the sum of the following two parts:

[0115]

[0116] (2) If the horizontal direction information of the first direction information is greater than the vertical direction information, the value of the first interpolated pixel is calculated along the column direction of the target position. Exemplarily, the first interpolated pixel (g pixel) is the sum of the following two parts:

[0117]

[0118] (3) If the horizontal direction information of the first direction information is equal to the vertical direction information, the value of the first interpolated pixel is calculated along the row direction and the column direction of the target position. Exemplarily, the first interpolated pixel (g pixel) is the sum of the following two parts:

[0119]

[0120] In this embodiment, the edge and texture of the image are taken into consideration during interpolation. By judging the size of the first horizontal direction information and the first vertical direction information, interpolation is performed preferentially along the smaller direction information. This can play the role of interpolation along the image texture and edge direction, and ultimately achieve a better effect of removing moiré patterns.

[0121] In one embodiment, the above step S104, calculating the second direction information of the target position according to the first direction information of the second channel pixel and the direction information of the first channel pixel after inserting the first interpolated pixel, can be implemented by the following method:

[0122] The second direction information may be obtained by adding the second horizontal direction information and the second vertical direction information.

[0123] (1) Second horizontal direction information: Calculate the horizontal direction information of the second channel pixel and the horizontal direction information of the first channel pixel respectively; add the horizontal direction information of the second channel pixel and the horizontal direction information of the first channel pixel to obtain the second horizontal direction information.

[0124] Specifically, on the center row where the target position is located, the first horizontal direction information of the second channel pixels having the same channel attribute as the original pixel of the target position is added to obtain a first value; on the upper and lower neighboring rows of the center row, the mean values ​​of the first horizontal direction information of the second channel pixels having different channel attributes from the original pixel are added to obtain a second value; the first value and the second value are added to obtain the horizontal direction information of the second channel pixels; on the center row and the upper and lower neighboring rows, the absolute values ​​of the differences between adjacent first channel pixels are calculated respectively, and the absolute values ​​of the differences between adjacent first channel pixels are added to obtain the horizontal direction information of the first channel pixels. Figure 5 , the specific calculation formula is as follows:

[0125]

[0126] Among them, H2 represents the second horizontal direction information, H RB Represents the horizontal direction information of the second channel pixel, H g Represents the horizontal direction information of the first channel pixel, H R0 +H R4 +H R5 represents the first value, (H B1 +H B3 ) / 2+(H B2 +H B4 ) / 2 represents the second value.

[0127] (2) Second vertical direction information: The vertical direction information of the second channel pixel and the vertical direction information of the first channel pixel are calculated respectively; the vertical direction information of the second channel pixel and the vertical direction information of the first channel pixel are added to obtain the second vertical direction information.

[0128] Specifically, in the center column where the target position is located, the first vertical direction information of the second channel pixels having the same channel attribute as the original pixel of the target position is added to obtain a third value; in the left and right neighboring columns of the center column, the mean values ​​of the first vertical direction information of the second channel pixels having different channel attributes from the original pixel are added to obtain a fourth value; the third value and the fourth value are added to obtain the vertical direction information of the second channel pixel; in the center column and the left and right neighboring columns, the absolute values ​​of the differences between adjacent first channel pixels are calculated respectively, and the absolute values ​​of the differences between adjacent first channel pixels are added to obtain the vertical direction information of the first channel pixel. Reference Figure 5 , the specific calculation formula is as follows:

[0129]

[0130] Wherein, V2 represents the second vertical direction information, V RB Represents the vertical information of the second channel pixel, V g Represents the vertical information of the first channel pixel, V R0 +V R2 +V R7 Represents the third value, (V B1 +V B2 ) / 2+(V B3 +V B4 ) / 2 represents the fourth value.

[0131] In one embodiment, the above step S105, updating the first interpolation pixel according to the second direction information, can be implemented by the following method:

[0132] Determine pixel update information of the target position; determine pixel update information of the non-target position in the operation area according to the second direction information; update the first interpolated pixel according to the pixel update information of the target position and the pixel update information of the non-target position to obtain a first interpolated pixel update value.

[0133] Figure 7 is a schematic diagram of the first image in this embodiment, such as Figure 7 As shown, the updating process includes updating pixels at target locations and updating pixels at non-target locations.

[0134] Pixel update at target location:

[0135]

[0136] Pixel updates at non-target locations:

[0137]

[0138] The final updated result is the weighted sum of the above two parts:

[0139]

[0140] Wherein, α represents an update weight parameter, and its value range is [0, 1]. Exemplarily, in this embodiment, the value of α is 0.3.

[0141] In one embodiment, after updating the first interpolated pixel according to the pixel update information of the target position and the pixel update information of the non-target position to obtain the first interpolated pixel update value, the method further includes:

[0142] The first interpolation pixel update value of the target position is compared with the first interpolation pixel update value adjacent to the target position; if the first interpolation pixel update value of the target position is greater than the maximum value of the first interpolation pixel update value adjacent to the target position, the maximum value of the first interpolation pixel update value adjacent to the target position is inserted into the target position; if the first interpolation pixel update value of the target position is less than the minimum value of the first interpolation pixel update value adjacent to the target position, the minimum value of the first interpolation pixel update value adjacent to the target position is inserted into the target position.

[0143] After obtaining the g' pixels at all positions, this embodiment performs clipping processing based on the g pixels above, below, left, and right of each g' pixel. R0 For example, calculate the surrounding g' G4 、g' G6 、g' G7 、g' G9 The maximum and minimum values ​​of g' R0 If it is greater than the maximum value, replace g' with the maximum value. R0 ; if g' R0 If it is less than the minimum value, replace g' with the minimum value. R0 ; if g' R0 When updating the G channel, this embodiment generates a weight value according to the second direction information, updates the center position and the non-center position, and uses the surrounding information for clipping, effectively suppressing the generation of false colors, moiré patterns, and bad pixels, and improving image clarity.

[0144] In one embodiment, Figure 8 A flowchart of a first interpolation pixel updating method is provided, such as Figure 8 As shown, in step S105, after the first interpolation pixel is updated according to the second direction information, the process further includes the following steps:

[0145] Step S201, determining whether the updated direction information of the first interpolation pixel is consistent with the direction information of the second channel pixel; if not, executing step S202; if consistent, exiting the current process.

[0146] The method comprises calculating the mean of the absolute values ​​of the differences between the adjacent updated first interpolation pixels based on the operation area to obtain the direction information of the updated first interpolation pixels; obtaining the direction information of the second channel pixels based on the second mean and the fourth mean; subtracting the direction information of the updated first interpolation pixels from the direction information of the second channel pixels to obtain the difference; if the absolute value of the difference is less than a preset value, determining that the direction information of the updated first interpolation pixels is consistent with that of the second channel pixels; and if the absolute value of the difference is not less than the preset value, determining that the direction information of the updated first interpolation pixels is inconsistent with that of the second channel pixels.

[0147] Specifically, in the N×N neighborhood, it is determined whether the updated direction information of the first interpolation pixel is consistent with that of the second channel pixel, which is equivalent to comparing whether the direction information (including horizontal direction information and vertical direction information) of the g' channel and the direction information (including horizontal direction information and vertical direction information) of the R / B channel are close. If they are close, it is determined that the directions are consistent; if they are not close, it is determined that the directions are inconsistent, and steps S203 and S204 are repeated and then determined again. Optionally, if the number of repetitions reaches a certain number (for example, 3 times), the direction consistency determination is exited.

[0148] The direction information of the second channel pixel is obtained by adding the second mean and the fourth mean, which will not be described in detail in this embodiment. The specific calculation formula of the updated direction information of the first interpolation pixel (the direction information of the g' channel) is as follows:

[0149]

[0150] Among them, F is the difference between the directional information of the g' channel and the R / B channel, h2g' is the horizontal directional information of the g' channel, and v2g' is the vertical directional information of the g' channel.

[0151] The calculation formula of h2g' is as follows:

[0152]

[0153] The calculation formula of v2g' is as follows:

[0154]

[0155] Step S202 , calculating new second direction information according to the first direction information of the second channel pixel and the direction information of the first channel pixel after the first interpolation pixel is updated.

[0156] Step S203, updating the current first interpolation pixel according to the new second direction information; and returning to step S201.

[0157] In this embodiment, the directional consistency judgment is performed using the directional information of the channel to be interpolated and the interpolation result, so as to decide whether it is necessary to further repeatedly update the second directional information and the first interpolation pixel, so as to make the first interpolation pixel consistent with the original direction to the greatest extent, and further make the interpolation result more consistent with the surrounding texture directional information, thereby better reducing moiré and bad pixels.

[0158] In one embodiment, the above step S106, interpolating the second channel based on the first channel pixels obtained after updating the first interpolated pixels to obtain the second image, can be implemented by the following method:

[0159] A second channel pixel adjacent to the position to be interpolated and having the same channel attribute as the second interpolation pixel to be inserted is determined; a second interpolation pixel is calculated based on the determined second channel pixel, the original pixel at the position to be interpolated, and the first interpolation pixel adjacent to the position to be interpolated and updated, and the second interpolation pixel is inserted into the position to be interpolated.

[0160] In this embodiment, a color difference interpolation method can be used. When interpolating the r pixel and the b pixel at the G channel position, the pixels at the surrounding R / B channel positions are used to calculate.

[0161] refer to Figure 6 When interpolating the r pixel of the G channel, the neighboring R pixel and the g' pixel at the corresponding position are used for processing. The specific interpolation calculation formulas are given below.

[0162] Interpolation r G6 Pixels:

[0163]

[0164] Interpolation r G9 Pixels:

[0165]

[0166] When interpolating the b pixel of the G channel, the neighboring B pixel and the g' pixel at the corresponding position are used for processing. The specific interpolation calculation formulas are given below.

[0167] Interpolation b G6 Pixels:

[0168]

[0169] Interpolation b G9 Pixels:

[0170]

[0171] When interpolating the b pixel of the R channel, use the neighboring B pixels and the g' pixels at the corresponding position. R0 Pixels:

[0172]

[0173] When interpolating the r pixel of the B channel, use the neighboring R pixels and the g' pixels at the corresponding position. B1 Pixels:

[0174]

[0175] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0176] In one embodiment, Fig. 9 A flowchart of another image processing method for suppressing false information is provided, such as Fig. 9 As shown, the process includes the following steps:

[0177] Step S301, calculating first direction information corresponding to each pixel position in the Bayer image;

[0178] Step S302, performing a first interpolation on the G channel according to the first direction information to obtain full-resolution g channel information;

[0179] Step S303, combining the full-resolution g channel information and the original R / B information to calculate the second direction information;

[0180] Step S304, updating the full-resolution g channel information after the first interpolation according to the second direction information to obtain new g' channel information;

[0181] Step S305, determining whether the directions of the g' channel information and the R / B information are consistent; if not, returning to step S303; if consistent, proceeding to the next step;

[0182] Step S306: interpolate the full-resolution r channel and b channel information respectively according to the g' channel information to obtain the final ro, go, and bo channel information.

[0183] This embodiment suppresses the generation of problem pixels such as false color and moiré during the de-mosaicing process. The initial g channel information is obtained by introducing the first direction information into the G channel interpolation process. The second direction information and direction consistency judgment are calculated according to the g channel, and the g channel is iteratively updated to make it consistent with the original R / B channel direction. Without increasing the calculation area, the false color, moiré, bad pixel and other problems caused by de-mosaicing are effectively suppressed, and the image clarity is improved. Specifically, when calculating the first direction information, the direction information of the G channel and the non-G channel are combined to make the initial vertical and horizontal direction information more accurate. When initially interpolating the G channel, pixels at different direction positions are used for interpolation according to the size of the vertical and horizontal direction information. When calculating the second direction information, the first direction information of the R / B channel and the initial g channel are combined, and the interpolation result of the G channel is updated using the low-frequency direction information, so that the result is more consistent with the surrounding texture direction information. When updating the G channel, a weight value is generated according to the second direction information, and the update is divided into the center position and the non-center position, and the surrounding information is used for limiting, which effectively suppresses the generation of false color, moiré, and bad pixel, and improves the image clarity. When judging the direction consistency, the direction information of the channel to be interpolated and the interpolation result is used to determine whether the second direction information and the g channel need to be further repeatedly updated, so as to make the interpolated g channel consistent with the original R / B channel direction to the greatest extent.

[0184] In one embodiment, an image processing device for suppressing false information is provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made are not repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0185] Fig.10 is a structural block diagram of the image processing device for suppressing false information in this embodiment. Fig.10 As shown, the device comprises:

[0186] An acquisition module is used to acquire a first image to be processed, determine a target position in the first image where first channel interpolation is to be performed, and determine an operation area with the target position as the center; wherein the number of pixels in the first channel in the first image is not less than the number of pixels in the second channel;

[0187] A first direction information calculation module, used to calculate the first direction information of the original second channel pixel in the target position based on the operation area;

[0188] A first interpolation module, used for inserting a first interpolation pixel at a target position according to first direction information;

[0189] A second direction information calculation module, used to calculate the second direction information of the target position according to the first direction information of the second channel pixel and the direction information of the first channel pixel after inserting the first interpolation pixel;

[0190] An updating module, used for updating the first interpolation pixel according to the second direction information;

[0191] The second interpolation module is used to interpolate the second channel based on the first channel pixels obtained after updating the first interpolation pixels to obtain a second image.

[0192] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment. The above modules can be functional modules or program modules, which can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0193] In addition, in combination with the image processing method for suppressing false information provided in the above embodiments, a storage medium may be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any image processing method for suppressing false information in the above embodiments is implemented.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0195] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0196] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0197] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method for suppressing false information, characterized in that: include: Acquire a first image to be processed, determine a target position in the first image where first channel interpolation is to be performed, and determine an operation area with the target position as the center; wherein the number of pixels in the first channel in the first image is not less than the number of pixels in the second channel; Calculate first direction information of original second channel pixels in the target position based on the operation area; inserting a first interpolation pixel at the target position according to the first direction information; Calculate second direction information of the target position according to the first direction information of the second channel pixel and the direction information of the first channel pixel after inserting the first interpolation pixel; the second direction information is low-frequency information of the first direction information, and the second direction information includes second horizontal direction information and second vertical direction information; updating the first interpolation pixel according to the second direction information; interpolating the second channel based on the first channel pixels obtained after updating the first interpolation pixels to obtain a second image; Among them, calculating the first direction information of the original second channel pixel in the target position based on the operation area includes: within the operation area, calculating the first horizontal direction information and the first vertical direction information based on the direction information of the surrounding channel pixels of the second channel pixel, and obtaining the first direction information from the first horizontal direction information and the first vertical direction information.

2. The image processing method for suppressing false information according to claim 1, characterized in that: Calculating first direction information of original second channel pixels in the target position based on the operation area includes: In the horizontal direction of the rows of the operation area, a first mean value of the absolute values ​​of the differences between adjacent pixels of the first channel is calculated, and in the horizontal direction of the rows of the operation area, a second mean value of the absolute values ​​of the differences between adjacent pixels of the second channel is calculated; Adding the first mean value and the second mean value to obtain the first horizontal direction information; Calculating a third mean of absolute values ​​of differences between adjacent pixels of the first channel in a vertical direction of a column of the operation area, and calculating a fourth mean of absolute values ​​of differences between adjacent pixels of the second channel in a vertical direction of a column of the operation area; Adding the third mean value and the fourth mean value to obtain the first vertical direction information; The first direction information is obtained according to the first horizontal direction information and the first vertical direction information.

3. The image processing method for suppressing false information according to claim 2, characterized in that: Inserting a first interpolation pixel at the target position according to the first direction information includes: comparing the first horizontal direction information with the first vertical direction information; If the first horizontal direction information is less than the first vertical direction information, calculating the value of the first interpolated pixel along the row direction of the target position; If the horizontal direction information of the first direction information is greater than the vertical direction information, calculating the value of the first interpolation pixel along the column direction of the target position; If the horizontal direction information of the first direction information is equal to the vertical direction information, the value of the first interpolation pixel is calculated along the row direction and the column direction of the target position.

4. The image processing method for suppressing false information according to claim 2, characterized in that: Calculating the second direction information of the target position according to the first direction information of the second channel pixel and the direction information of the first channel pixel after inserting the first interpolation pixel, includes: respectively calculating horizontal direction information of the second channel pixels and horizontal direction information of the first channel pixels; Adding the horizontal direction information of the second channel pixel and the horizontal direction information of the first channel pixel to obtain the second horizontal direction information; respectively calculating vertical direction information of the second channel pixels and vertical direction information of the first channel pixels; Adding the vertical direction information of the second channel pixel and the vertical direction information of the first channel pixel to obtain the second vertical direction information; The second direction information is obtained according to the second horizontal direction information and the second vertical direction information.

5. The image processing method for suppressing false information according to claim 4, characterized in that: Calculating the horizontal direction information of the second channel pixel and the horizontal direction information of the first channel pixel respectively, including: On the center row where the target position is located, the first horizontal direction information of the second channel pixels having the same channel attribute as the original pixel at the target position is added to obtain a first value; On upper and lower neighboring rows of the central row, adding the mean values ​​of the first horizontal direction information of the second channel pixels having a channel attribute different from that of the original pixel to obtain a second value; Adding the first value and the second value to obtain horizontal direction information of the second channel pixel; On the central row and the upper and lower neighboring rows, the absolute values ​​of the differences between adjacent pixels of the first channel are calculated respectively, and the absolute values ​​of the differences between adjacent pixels of the first channel are added to obtain the horizontal direction information of the pixels of the first channel.

6. The image processing method for suppressing false information according to claim 4, characterized in that: Respectively calculating the vertical direction information of the second channel pixel and the vertical direction information of the first channel pixel, including: In the center column where the target position is located, the first vertical direction information of the second channel pixels having the same channel attribute as the original pixel at the target position is added to obtain a third value; In left and right neighboring columns of the central column, average values ​​of the first vertical direction information of the second channel pixels having a channel attribute different from that of the original pixel are added to obtain a fourth value; Adding the third value and the fourth value to obtain vertical direction information of the second channel pixel; On the central column and the left and right neighboring columns, the absolute values ​​of the differences between adjacent pixels of the first channel are calculated respectively, and the absolute values ​​of the differences between adjacent pixels of the first channel are added to obtain the vertical direction information of the pixels of the first channel.

7. The image processing method for suppressing false information according to claim 4, characterized in that: After updating the first interpolation pixel according to the second direction information, the method further includes: The following steps are executed in a loop until it is determined that the direction information of the current first interpolation pixel is consistent with the direction information of the second channel pixel: Determining whether the updated direction information of the first interpolation pixel is consistent with the direction information of the second channel pixel; If the updated direction information of the first interpolated pixel is inconsistent with the direction information of the second channel pixel, calculating new second direction information according to the first direction information of the second channel pixel and the direction information of the first channel pixel after the first interpolated pixel is updated; The current first interpolation pixel is updated according to the new second direction information.

8. The image processing method for suppressing false information according to claim 7, characterized in that: Determining whether the updated direction information of the first interpolation pixel is consistent with the direction information of the second channel pixel includes: Calculating, based on the operation area, an average of absolute values ​​of differences between adjacent updated first interpolation pixels to obtain updated direction information of the first interpolation pixels; Obtaining direction information of pixels of the second channel according to the second mean and the fourth mean; Subtracting the updated direction information of the first interpolation pixel from the direction information of the second channel pixel to obtain a difference; If the absolute value of the difference is less than a preset value, it is determined that the updated direction information of the first interpolation pixel is consistent with the direction information of the second channel pixel; If the absolute value of the difference is not less than a preset value, it is determined that the updated direction information of the first interpolation pixel is inconsistent with the direction information of the second channel pixel.

9. The image processing method for suppressing false information according to any one of claims 1 to 8, characterized in that: Updating the first interpolation pixel according to the second direction information includes: Determining pixel update information of the target position; Determine pixel update information of a non-target position in the operation area according to the second direction information; The first interpolation pixel is updated according to the pixel update information of the target position and the pixel update information of the non-target position to obtain a first interpolation pixel update value.

10. The image processing method for suppressing false information according to claim 9, characterized in that: After updating the first interpolated pixel according to the pixel update information of the target position and the pixel update information of the non-target position to obtain a first interpolated pixel update value, the method further includes: comparing the first interpolated pixel update value of the target position with the first interpolated pixel update value adjacent to the target position; If the first interpolation pixel update value of the target position is greater than the maximum value of the first interpolation pixel update values ​​adjacent to the target position, inserting the maximum value of the first interpolation pixel update values ​​adjacent to the target position into the target position; If the first interpolation pixel update value of the target position is smaller than a minimum value of the first interpolation pixel update values ​​adjacent to the target position, the minimum value of the first interpolation pixel update values ​​adjacent to the target position is inserted into the target position.

11. The image processing method for suppressing false information according to claim 1, characterized in that: Interpolating the second channel based on the first channel pixels obtained after updating the first interpolation pixels to obtain a second image includes: Determine a second channel pixel that is adjacent to the position to be interpolated and has the same channel attribute as the second interpolation pixel to be inserted; The second interpolation pixel is calculated based on the determined second channel pixel, the original pixel at the position to be interpolated, and the first interpolation pixel adjacent to the position to be interpolated and updated, and the second interpolation pixel is inserted into the position to be interpolated.

12. An image processing device for suppressing false information, characterized in that: The device comprises: an acquisition module, used for acquiring a first image to be processed, determining a target position in the first image for first channel interpolation, and determining an operation area with the target position as the center; wherein the number of pixels in the first channel in the first image is not less than the number of pixels in the second channel; A first direction information calculation module, used for calculating the first direction information of the original second channel pixel in the target position based on the operation area; A first interpolation module, configured to insert a first interpolation pixel at the target position according to the first direction information; a second direction information calculation module, configured to calculate second direction information of the target position according to the first direction information of the second channel pixel and the direction information of the first channel pixel after inserting the first interpolation pixel; the second direction information is low-frequency information of the first direction information, and the second direction information includes second horizontal direction information and second vertical direction information; An updating module, configured to update the first interpolation pixel according to the second direction information; A second interpolation module, configured to interpolate the second channel based on the first channel pixels obtained after updating the first interpolation pixels, so as to obtain a second image; Among them, calculating the first direction information of the original second channel pixel in the target position based on the operation area includes: within the operation area, calculating the first horizontal direction information and the first vertical direction information based on the direction information of the surrounding channel pixels of the second channel pixel, and obtaining the first direction information from the first horizontal direction information and the first vertical direction information.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images