Image Processing Method, Apparatus, and Storage Medium for Suppressing False Information

Abstract

The present application relates to an image processing method, apparatus, and storage medium for suppressing false information. By obtaining a first image to be processed, determining a target position in the first image where first-channel interpolation is to be performed, and determining an operation area centered on the target position; wherein the number of pixels in the first channel of the first image is not less than the number of pixels in the second channel; calculating first-direction information of the original second-channel pixels in the target position based on the operation area; filtering the first-direction information to obtain second-direction information; inserting first interpolation pixels at the target position according to the first-direction information; updating the first interpolation pixels according to the second-direction information; and performing interpolation on the second channel based on the first-channel pixels obtained after updating the first interpolation pixels to obtain a second image; it realizes suppressing false information during the demosaicing process and improves the clarity of the image without increasing the computational amount.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to an image processing method, apparatus, and storage medium for suppressing false information. Background Art

[0002] With the continuous improvement of medical surgery technology, a shooting and display system is often used during surgery to observe the target site, and the quality of medical images and videos has become one of the concerns in the medical system. Medical devices usually use imaging devices such as CMOS to collect image data arranged in Bayer pattern, abbreviated as Bayer data. The data in Bayer format is composed of incomplete RGB data arranged at intervals, and it is necessary to use the pixel values existing in the surrounding information to perform interpolation and completion to obtain complete RGB three-channel data. This process of interpolation and completion is also called demosaicing.

[0003] During the demosaicing process, the original Bayer image includes red pixel values, blue pixel values, and green pixel values. By interpolation, the two missing color components of each pixel point are obtained (for example, the blue component and red component of the green pixel point are interpolated) so that each pixel point includes a red component, a green component, and a blue component, thereby converting the Bayer image into an RGB image. However, after the demosaicing process, false information that did not originally exist in the original image, such as false colors, moiré patterns, and bad pixels, will be introduced additionally, thus affecting the clarity of the image. In related technologies, for problems such as false colors, moiré patterns, and bad pixels generated during the demosaicing process, judgment and removal are performed after the interpolation is completed, which increases the additional judgment and elimination processes and increases the computational amount.

[0004] Currently, for the problem that a large amount of computation is required to remove false information in an image in related technologies, no effective solution has been proposed yet. Summary of the Invention

[0005] Based on this, it is necessary to provide an image processing method, apparatus, and storage medium for suppressing false information that can avoid increasing the computational amount for the above technical problems.

[0006] In a first aspect, the present application provides an image processing method for suppressing false information, including:

[0007] Obtain a first image to be processed, determine a target position in the first image where first-channel interpolation is to be performed, and determine an operation area centered on the target position; wherein, the number of pixels in the first channel of the first image is not less than the number of pixels in the second channel;

[0008] Calculate first-direction information of the original second-channel pixels in the target position based on the operation area;

[0009] Filter the first direction information to obtain second direction information;

[0010] Insert a first interpolated pixel at the target position according to the first direction information;

[0011] Update the first interpolated pixel according to the second direction information;

[0012] Interpolate the second channel based on the first channel pixel obtained after updating the first interpolated pixel to obtain a second image.

[0013] In one embodiment, calculating the first direction information of the original second channel pixel in the target position based on the operation area includes:

[0014] On the row horizontal direction of the operation area, calculate a first mean value of the absolute values of the differences between adjacent first channel pixels, and on the row horizontal direction of the operation area, calculate a second mean value of the absolute values of the differences between adjacent second channel pixels;

[0015] Add the first mean value and the second mean value to obtain first horizontal direction information;

[0016] On the column vertical direction of the operation area, calculate a third mean value of the absolute values of the differences between adjacent first channel pixels, and on the column vertical direction of the operation area, calculate a fourth mean value of the absolute values of the differences between adjacent second channel pixels;

[0017] Add the third mean value and the fourth mean value to obtain first vertical direction information;

[0018] Obtain the first direction information according to the first horizontal direction information and the first vertical direction information.

[0019] In one embodiment, inserting a first interpolated pixel at the target position according to the first direction information includes:

[0020] Compare the first horizontal direction information and the first vertical direction information;

[0021] If the first horizontal direction information is less than the first vertical direction information, calculate the value of the first interpolated pixel along the row direction of the target position;

[0022] If the horizontal direction information of the first direction information is greater than the vertical direction information, calculate the value of the first interpolated pixel along the column direction of the target position;

[0023] If the horizontal direction information of the first direction information is equal to the vertical direction information, calculate the value of the first interpolated pixel along the row direction and column direction of the target position.

[0024] In one embodiment, filtering the first direction information to obtain second direction information includes:

[0025] Divide the operation area into multiple sub-areas;

[0026] In each sub-area, identify a first pixel having the same channel attribute as the original pixel of the target position, and a second pixel having a channel attribute different from that of the original pixel; wherein, the second pixel does not include the first channel pixel;

[0027] In a specific direction corresponding to the sub-area, calculate the sum of the direction information of the first pixel and calculate the mean value of the direction information of the second pixel;

[0028] Add the sum of the direction information of the first interpolated pixel to the mean value of the direction information of the second pixel to obtain the second direction information of each sub-area.

[0029] In one embodiment, the second direction information includes: second horizontal left direction information, second horizontal right direction information, second vertical up direction information, and second vertical down direction information.

[0030] In one embodiment, the sub-areas include the left half sub-area, right half sub-area, upper half sub-area, and lower half sub-area of the operation area. Filtering the first direction information to obtain second direction information includes:

[0031] For the left half sub-area: On the central row where the target position is located, add the first horizontal direction information of the first pixel to obtain a first value; on the neighboring rows adjacent to the central row, add the mean value of the first horizontal direction information of the second pixel to obtain a second value; add the first value and the second value to obtain the second horizontal left direction information;

[0032] For the right half sub-area: On the central row, add the first horizontal direction information of the first pixel to obtain a third value; on the neighboring rows adjacent to the central row, add the mean value of the first horizontal direction information of the second pixel to obtain a fourth value; add the third value and the fourth value to obtain the second horizontal right direction information;

[0033] For the upper sub-region: On the central column where the target position is located, add the first vertical direction information of the first pixels to obtain a fifth value; on the neighborhood columns adjacent to the central column, add the average value of the first vertical direction information of the second pixels to obtain a sixth value; add the fifth value and the sixth value to obtain the second vertical upward direction information;

[0034] For the lower sub-region: On the central column, add the first vertical direction information of the first pixels to obtain a seventh value; on the neighborhood columns adjacent to the central column, add the average value of the first vertical direction information of the second pixels to obtain an eighth value; add the seventh value and the eighth value to obtain the second vertical downward direction information.

[0035] In one embodiment, updating the first interpolated pixel according to the second direction information includes:

[0036] Determine the pixel update information of the target position;

[0037] Determine the pixel update information of non-target positions in the operation region according to the second direction information;

[0038] Update the first interpolated pixel according to the pixel update information of the target position and the pixel update information of the non-target positions to obtain a first interpolated pixel update value.

[0039] In one embodiment, after updating the first interpolated pixel according to the pixel update information of the target position and the pixel update information of the non-target positions to obtain a first interpolated pixel update value, the method further includes:

[0040] Compare the first interpolated pixel update value of the target position with the first interpolated pixel update values adjacent to the target position;

[0041] If the first interpolated pixel update value of the target position is greater than the maximum value of the first interpolated pixel update values adjacent to the target position, insert the maximum value of the first interpolated pixel update values adjacent to the target position into the target position;

[0042] If the first interpolated pixel update value of the target position is less than the minimum value of the first interpolated pixel update values adjacent to the target position, insert the minimum value of the first interpolated pixel update values adjacent to the target position into the target position.

[0043] In one embodiment, based on the first channel pixels obtained after updating the first interpolated pixel, interpolating the second channel to obtain a second image includes:

[0044] Determine a second-channel pixel that is adjacent to the position to be interpolated and has the same channel attribute as the second interpolated pixel to be inserted;

[0045] Calculate the second interpolated pixel based on the determined second-channel pixel, the original pixel at the position to be interpolated, and the first interpolated pixel that is adjacent to the position to be interpolated and has been updated, and insert the second interpolated pixel into the position to be interpolated.

[0046] In a second aspect, the present application provides an image processing device for suppressing false information, the device including:

[0047] An acquisition module, configured to acquire a first image to be processed, determine a target position in the first image where first-channel interpolation is to be performed, and determine an operation area centered on the target position; wherein, the number of pixels in the first channel of the first image is not less than the number of pixels in the second channel;

[0048] A first direction information calculation module, configured to calculate first direction information of the original second-channel pixel in the target position based on the operation area;

[0049] A second direction information calculation module, configured to filter the first direction information to obtain second direction information;

[0050] A first interpolation module, configured to insert a first interpolated pixel at the target position according to the first direction information;

[0051] An update module, configured to update the first interpolated pixel according to the second direction information;

[0052] A second interpolation module, configured to perform interpolation on the second channel based on the first-channel pixel obtained after updating the first interpolated pixel to obtain a second image.

[0053] In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0054] The above image processing method, device, and storage medium for suppressing false information determine the channel interpolation priority order according to the amount of channel information in the RGB channels. For the channels with priority interpolation, initial first interpolation pixels are obtained by introducing first direction information into the first channel interpolation process, preliminarily eliminating false colors, moiré patterns, and bad pixel problems. Then, the first direction information is filtered to obtain second direction information, and the first interpolation pixels are updated based on the second direction, which can make the interpolation result more conform to the surrounding texture direction information, further reducing moiré patterns and bad pixels; finally, based on the updated first interpolation pixels, the interpolation of the second channel is completed. In this way, without increasing the calculation area range, false information caused by demosaicing is effectively suppressed, and false information is suppressed during the demosaicing process, improving the image clarity without increasing the calculation amount. Description of the Drawings

[0055] Figure 1 It is a block diagram of the hardware structure of a terminal for an image processing method for suppressing false information in an embodiment;

[0056] Figure 2 It is a flowchart of an image processing method for suppressing false information in an embodiment;

[0057] Figure 3 It is an input / output schematic diagram of demosaicing in an embodiment;

[0058] Figure 4 It is an interpolation schematic diagram of demosaicing in an embodiment;

[0059] Figure 5 It is a schematic diagram of a first image in an embodiment;

[0060] Figure 6 It is a schematic diagram of a first image in another embodiment;

[0061] Figure 7 It is a schematic diagram of a first image in another embodiment;

[0062] Figure 8 It is a flowchart of an image processing method for suppressing false information in another embodiment;

[0063] Figure 9 It is a block diagram of the structure of an image processing device for suppressing false information in an embodiment. Detailed Embodiments

[0064] In order to make the objectives, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0065] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by those with ordinary skills in the technical field to which this application belongs. In this application, words such as "a", "one", "a kind of", "the", "these" and the like do not indicate a limitation in quantity, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The terms "connected", "coupled" and the like involved in this application do not limit to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are in an "or" relationship. The terms "first", "second", "third" and the like involved in this application only distinguish similar objects and do not represent a specific sorting of the objects.

[0066] The method embodiment provided in this embodiment can be executed on a terminal, a computer or a similar computing device. For example, when running on a terminal, Figure 1 is a hardware structure block diagram of a terminal for an image processing method for suppressing false information according to an embodiment of the present application. As Figure 1 shown, the terminal may include one or more ( Figure 1 only one is shown in the figure) processors 101 and a memory 102 for storing data. Among them, the processor 101 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may further include a transmission device 103 for communication functions and an input / output device 104. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above terminal. For example, the terminal may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown.

[0067] The memory 102 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the image processing method for suppressing pseudo information in this embodiment. The processor 101 executes various functional applications and data processing by running the computer program stored in the memory 102, that is, implements the above method. The memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 102 may further include a memory remotely disposed relative to the processor 101, and these remote memories can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.

[0068] The transmission device 103 is used to receive or send data via a network. The above network includes the wireless network provided by the communication provider of the terminal. In one instance, the transmission device 103 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 103 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0069] The related technology provides the following image pseudo information removal methods:

[0070] One is to divide the image into different regions through the information entropy of the image region, and perform different pseudo-color removal methods for different regions, so as to achieve the purpose of pseudo-color removal. This method performs pseudo-color removal after demosaicing processing, increasing the image processing flow and computational complexity.

[0071] The other is to screen out the pixel points that meet the preset moiré color and direction characteristics in the image to be processed as moiré pixel points, statistically interpolate the direction of the pixel point area, and perform re-interpolation, so as to suppress the generation of moiré. This solution also judges whether the pixel after interpolation and completion is a moiré pixel after demosaicing processing, and then performs re-interpolation processing.

[0072] Based on the analysis of the above situation, in this embodiment, an image processing method for suppressing pseudo information is provided to solve image quality problems such as pseudo-color, moiré, and bad points that occur in the demosaicing process of the ISP (Image Signal Processor). Figure 2 It is a flowchart of the image processing method for suppressing pseudo information, as Figure 2 shown, and this process includes the following steps:

[0073] Step S101: Obtain a first image to be processed, determine a target position in the first image where first-channel interpolation is to be performed, and determine an operation area centered on the target position; wherein, the number of pixels in the first channel of the first image is not less than the number of pixels in the second channel.

[0074] Figure 3 is a schematic diagram of the input and output of demosaicing. As Figure 3 shown, demosaicing is to convert the input Bayer-format data into RGB-format output data. The conversion process is to supplement the missing data information, that is, to interpolate the R, G, and B channel information respectively. According to the different arrangement orders of pixel positions, Bayer images can have 4 formats: RGGB, BGGR, GRGB, and GBGR. This embodiment can input a Bayer image in any format and output an RGB image. For the sake of easy understanding, this embodiment uses the RGGB format for description. For the first image in the RGGB format, the first channel is the G channel, and the second channel is the R or B channel (abbreviated as the R / B channel). It can be found that the number of pixels in the G channel is twice the number of pixels in the R / B channel.

[0075] Figure 4 is a schematic diagram of interpolation for demosaicing. As Figure 4 shown, in this process, missing g data is interpolated in the G channel, missing r data is interpolated in the R channel, and missing b data is interpolated in the B channel. For the RGGB format, the G channel needs to be newly expanded by 1 time the original data volume, and the R and B channels need to be newly expanded by 2 times the original data volume. In this process, since the amount of information in the R and B channels is less, and the spectral response intensity of the R and B channels is not as large as that of the G channel, therefore, if the R / B channels are interpolated first and then the G channel, the R / B channels are very likely to have problems such as interpolation errors, color mutations, and regional interferences, and affect the processing of the subsequent ISP process, and finally there will be image quality problems such as false colors, moiré patterns, and dead pixels that do not conform to the real laws at the display end. To solve the above problems, this embodiment interpolates the G channel first and then the R / B channels.

[0076] Figure 5 is a schematic diagram of the first image in this embodiment. As Figure 5 shown, where red represents the pixels of the R channel, green represents the pixels of the G channel, blue represents the pixels of the B channel, and the numbers represent the numbers of the pixels in the corresponding channels. Assume that currently R0 is determined as the target position where G-channel interpolation is to be performed, then the N×N neighborhood information around R0 can be taken to obtain a small operation area. N can be 5, 7, 9, etc. As an example, N is taken as 5.

[0077] Step S102: Calculate the first-direction information of the original second-channel pixels at the target position based on the operation area.

[0078] The first direction information of the original second-channel pixels is the first direction information of R0. When calculating the first direction information, within the operation area, the direction information of the G channel and the R / B channel around R0 can be used to calculate the first direction information of R0. Among them, the first direction information includes the first horizontal direction information and the first vertical direction information.

[0079] Step S103: Filter the first direction information to obtain the second direction information.

[0080] When calculating the second direction information, within the operation area, the first direction information of the R / B channel can be filtered several times (for example, 2 times) to calculate the second direction information of the target position. Among them, the second direction information includes the second horizontal direction information and the second vertical direction information.

[0081] Step S104: Insert the first interpolation pixel at the target position according to the first direction information.

[0082] According to the first direction information of R0 and the pixels around R0, the initial first interpolation pixel (g pixel) is calculated. The same operation is performed for other target positions to be interpolated in the first channel. Interpolate the G channel once according to the first direction information to fill in all the missing data in the G channel and obtain the full-resolution G pixel.

[0083] Step S105: Update the first interpolation pixel according to the second direction information.

[0084] According to the second direction information of the target position and the pixels in the surrounding area of the target position, a new first interpolation pixel (g' pixel) is calculated. It should be noted that the causes of moiré and dead pixels include high-frequency noise interference. And the second direction information is the low-frequency information obtained by filtering the first direction information. Subsequently, using this low-frequency information to update the first interpolation pixel can make the interpolation result more conform to the surrounding texture direction information, thereby reducing moiré and dead pixels.

[0085] Step S106: Based on the first-channel pixels obtained after updating the first interpolation pixel, interpolate the second channel to obtain the second image.

[0086] Interpolating the second channel means inserting r pixels into the R channel and b pixels into the B channel to fill in all the missing data in the R / B channel. The chromatic aberration interpolation method can be used. When interpolating the r pixels and b pixels at the G-channel position, they are calculated using the pixels at the surrounding R / B-channel positions.

[0087] In the above steps S101 to 106, the channel interpolation priority order is determined according to the amount of channel information in the RGB channels. For the channels with priority interpolation, the initial first interpolation pixels are obtained by introducing the first direction information into the first channel interpolation process, preliminarily eliminating the problems of false colors, moiré patterns, and bad pixels. Then, after filtering the first direction information, the second direction information is obtained, and the first interpolation pixels are updated based on the second direction, which can make the interpolation result more conform to the surrounding texture direction information, further reducing moiré patterns and bad pixels; finally, based on the updated first interpolation pixels, the interpolation of the second channel is completed. In this way, without increasing the calculation area range, the pseudo information caused by demosaicing is effectively suppressed, and the suppression of pseudo information is realized during the demosaicing process, and the clarity of the image is improved without increasing the calculation amount.

[0088] In one embodiment, in the above step S102, calculating the first direction information of the original second channel pixels in the target position based on the operation area can be achieved by the following method:

[0089] (1) The first horizontal direction information: On the row horizontal direction of the operation area, calculate the first mean value of the absolute values of the differences between adjacent first channel pixels, and on the row horizontal direction of the operation area, calculate the second mean value of the absolute values of the differences between adjacent second channel pixels; add the first mean value and the second mean value to obtain the first horizontal direction information. Refer to Figure 5 , and the specific calculation formula is as follows:

[0090]

[0091] Among them, H1 represents the first horizontal direction information, h1 represents the first mean value, and h2 represents the second mean value.

[0092] (2) The first vertical direction information: On the column vertical direction of the operation area, calculate the third mean value of the absolute values of the differences between adjacent first channel pixels, and on the column vertical direction of the operation area, calculate the fourth mean value of the absolute values of the differences between adjacent second channel pixels; add the third mean value and the fourth mean value to obtain the first vertical direction information. Refer to Figure 5 , and the specific calculation formula is as follows:

[0093]

[0094] Among them, V1 represents the first vertical direction information, v1 represents the third mean value, and v2 represents the fourth mean value.

[0095] In this embodiment, when calculating the first direction information, the direction information of the first channel and the second channel is combined, making the initial first direction information more accurate.

[0096] In one embodiment, in the above step S103, filtering the first direction information to obtain the second direction information can be achieved by the following method:

[0097] Divide the operation area into multiple sub-areas; within each sub-area, identify the first pixels having the same channel attribute as the original pixels at the target position, and the second pixels having different channel attributes from the original pixels; wherein, the second pixels do not include the first-channel pixels; in the specific direction corresponding to the sub-area, calculate the sum of the direction information of the first pixels, and calculate the mean value of the direction information of the second pixels; add the sum of the direction information of the first pixels and the mean value of the direction information of the second pixels to obtain the second direction information of each sub-area.

[0098] In this embodiment, the second direction information includes: the second horizontal left direction information, the second horizontal right direction information, the second vertical up direction information, and the second vertical down direction information. Referring to Figure 5 , the sub-areas include the left half sub-area, the right half sub-area, the upper half sub-area, and the lower half sub-area of the operation area. Filtering the first direction information to obtain the second direction information can be achieved by the following method:

[0099] (1) For the left half sub-area: On the central row where the target position is located, add the first horizontal direction information of the first pixels to obtain the first value; on the neighboring rows adjacent to the central row, add the mean values of the first horizontal direction information of the second pixels to obtain the second value; add the first value and the second value to obtain the second horizontal left direction information. The calculation formula is as follows:

[0100]

[0101] (2) For the right half sub-area: On the central row, add the first horizontal direction information of the first pixels to obtain the third value; on the neighboring rows adjacent to the central row, add the mean values of the first horizontal direction information of the second pixels to obtain the fourth value; add the third value and the fourth value to obtain the second horizontal right direction information. The calculation formula is as follows:

[0102]

[0103] (3) For the upper half sub-area: On the central column where the target position is located, add the first vertical direction information of the first pixels to obtain the fifth value; on the neighboring columns adjacent to the central column, add the mean values of the first vertical direction information of the second pixels to obtain the sixth value; add the fifth value and the sixth value to obtain the second vertical up direction information. The calculation formula is as follows:

[0104]

[0105] For the lower sub-region: on the central column, add the first vertical direction information of the first pixel to obtain a seventh value; on the neighboring columns adjacent to the central column, add the average value of the first vertical direction information of the second pixels to obtain an eighth value; add the seventh value and the eighth value to obtain the second vertical downward direction information. The calculation formula is as follows:

[0106]

[0107] In this embodiment, when calculating the second direction information, on the basis of the first direction information, the first horizontal direction information therein is further subdivided into second horizontal left direction information and second horizontal right direction information, and the first vertical direction information therein is further subdivided into second vertical upward direction information and second vertical downward direction information, making the direction information more accurate, thereby refining the interpolation result of the first channel. Further suppress the pseudo information.

[0108] In one embodiment, the above step S104, inserting the first interpolation pixel at the target position according to the first direction information, can be implemented by the following method:

[0109] Compare the first horizontal direction information and the first vertical direction information, and select the corresponding interpolation method according to the comparison result of the two. Figure 6 Schematic diagram of the first image in this embodiment, as Figure 6 shown, the results are divided into the following 3 cases:

[0110] (1) If the first horizontal direction information is less than the first vertical direction information, calculate the value of the first interpolation pixel along the row direction of the target position. Exemplarily, the first interpolation pixel (g pixel) is the sum of the following 2 parts:

[0111]

[0112] (2) If the horizontal direction information of the first direction information is greater than the vertical direction information, calculate the value of the first interpolation pixel along the column direction of the target position. Exemplarily, the first interpolation pixel (g pixel) is the sum of the following 2 parts:

[0113]

[0114] (3) If the horizontal direction information of the first direction information is equal to the vertical direction information, calculate the value of the first interpolation pixel along the row direction and the column direction of the target position. Exemplarily, the first interpolation pixel (g pixel) is the sum of the following 2 parts:

[0115]

[0116] In this embodiment, considering the edges and textures of the image during interpolation, by judging the magnitudes of the first horizontal direction information and the first vertical direction information, interpolation is preferentially performed along the smaller direction information, which can play a role in interpolating along the texture and edge directions of the image, and finally achieve a better effect of removing moiré patterns.

[0117] In one embodiment, the above step S105 of updating the first interpolated pixel according to the second direction information can be implemented by the following method:

[0118] Determine the pixel update information at the target position; determine the pixel update information at non-target positions in the operation area according to the second direction information; update the first interpolated pixel according to the pixel update information at the target position and the pixel update information at non-target positions to obtain the updated value of the first interpolated pixel.

[0119] Figure 7 It is a schematic diagram of the first image in this embodiment, as Figure 7 shown, the update process includes pixel update at the target position and pixel update at non-target positions.

[0120] Pixel update at the target position:

[0121]

[0122] Pixel update at non-target positions:

[0123]

[0124] The final update result is the weighted sum of the above two parts:

[0125]

[0126] Among them, α represents the update weight parameter, and its value range is [0, 1]. Exemplarily, in this embodiment, the value of α is 0.3.

[0127] In one embodiment, after updating the first interpolated pixel according to the pixel update information at the target position and the pixel update information at non-target positions to obtain the updated value of the first interpolated pixel, the method further includes:

[0128] Compare the updated value of the first interpolated pixel at the target position with the updated values of the first interpolated pixels adjacent to the target position; if the updated value of the first interpolated pixel at the target position is greater than the maximum value of the updated values of the first interpolated pixels adjacent to the target position, insert the maximum value of the updated values of the first interpolated pixels adjacent to the target position into the target position; if the updated value of the first interpolated pixel at the target position is less than the minimum value of the updated values of the first interpolated pixels adjacent to the target position, insert the minimum value of the updated values of the first interpolated pixels adjacent to the target position into the target position.

[0129] After obtaining the g' pixels at all positions in this embodiment, clipping processing is performed according to the g pixels above, below, left, and right of each g' pixel. Taking the clipped pixel g' R0 as an example, calculate the maximum and minimum values of the surrounding g' G4 , g' G6 , g' G7 , g' G9 ; if g' R0 is greater than the maximum value, replace g' R0 with the maximum value; if g' R0 is less than the minimum value, replace g' R0 with the minimum value; if g' R0 is between the maximum and minimum values, no processing is performed. In this embodiment, when updating the G channel, weight values are generated according to the second direction information, and the update is performed separately for the central position and non-central positions, and clipping is performed using the surrounding information, effectively suppressing the generation of false colors, moiré patterns, and dead pixels, and improving the image clarity.

[0130] In one embodiment, the above step S106, based on the first channel pixels obtained after updating the first interpolated pixels, interpolates the second channel to obtain a second image, which can be implemented by the following method:

[0131] Determine the second channel pixels adjacent to the position to be interpolated and having the same channel attribute as the second interpolated pixel to be inserted; calculate the second interpolated pixel according to the determined second channel pixels, the original pixel at the position to be interpolated, and the updated first interpolated pixels adjacent to the position to be interpolated, and insert the second interpolated pixel into the position to be interpolated.

[0132] In this embodiment, the chromatic aberration interpolation method can be used. When interpolating the r pixel and b pixel at the G channel position, they are calculated using the pixels at the surrounding R / B channel positions.

[0133] Refer to Figure 6 . When interpolating the r pixel of the G channel, use the neighboring R pixel and the corresponding g' pixel for processing. The following respectively give the specific interpolation calculation formulas.

[0134] Interpolate the r G6 pixel:

[0135]

[0136] Interpolate the r G9 pixel:

[0137]

[0138] When interpolating the b pixel of the G channel, use the neighboring B pixel and the corresponding g' pixel for processing. The following respectively give the specific interpolation calculation formulas.

[0139] Interpolation b G6 Pixel:

[0140]

[0141] Interpolation b G9 Pixel:

[0142]

[0143] When interpolating the b pixel of the R channel, use the nearest neighbor B pixel and the g' pixel at the corresponding position for processing. Interpolation b R0 Pixel:

[0144]

[0145] When interpolating the r pixel of the B channel, use the nearest neighbor R pixel and the g' pixel at the corresponding position for processing. Interpolation r B1 Pixel:

[0146]

[0147] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the indications of the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0148] In one embodiment, Figure 8 A flowchart of another image processing method for suppressing false information is provided, as Figure 8 shown, and this process includes the following steps:

[0149] Step S201, calculate the first direction information corresponding to each pixel position in the Bayer image;

[0150] Step S202, calculate the second direction information based on the first direction information;

[0151] Step S203, perform the first interpolation on the G channel according to the first direction information to obtain the full-resolution g channel information;

[0152] Step S204: Update the full-resolution g-channel information after the first interpolation according to the second direction information to obtain new g'-channel information;

[0153] Step S205: Interpolate the full-resolution r-channel and b-channel information respectively according to the g'-channel information to obtain the final ro, go, and bo channel information.

[0154] In this embodiment, during the demosaicing process, the generation of problem pixels such as false colors and moiré is suppressed. By introducing the first direction information and the second direction information into the G-channel interpolation process, and according to the different direction information in the neighborhood area of the point to be interpolated, the interpolation of the G-channel is refined. Without increasing the calculation area range, the problems of false colors, moiré, and bad pixels caused by demosaicing are effectively suppressed, and the clarity of the image is improved. Specifically, when calculating the first direction information, the direction information of the G-channel and the non-G-channel is combined to make the initial vertical and horizontal direction information more accurate. When calculating the second direction information, the first horizontal direction information is subdivided into left horizontal and right horizontal, and the first vertical direction information is subdivided into upper vertical and lower vertical, making the direction information more precise. When initially interpolating the G-channel, according to the different magnitudes of the first vertical and horizontal direction information, the pixel values at different direction positions are used for interpolation respectively. When updating the G-channel, according to the weight values generated by the refined second direction information, the update is divided into the center position and the non-center position, and the surrounding information is used for limiting, effectively suppressing the generation of false colors, moiré, and bad pixels, and improving the image clarity.

[0155] In one embodiment, an image processing device for suppressing false information is provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated here. The following terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0156] Figure 9 is the structural block diagram of the image processing device for suppressing false information in this embodiment, as Figure 9 shown, the device includes:

[0157] An acquisition module, configured to acquire a first image to be processed, determine a target position in the first image where the first-channel interpolation is to be performed, and determine an operation area centered on the target position; wherein, the number of pixels of the first channel in the first image is not less than the number of pixels of the second channel;

[0158] A first direction information calculation module, configured to calculate the first direction information of the original second-channel pixels in the target position based on the operation area;

[0159] A second direction information calculation module, configured to filter the first direction information to obtain second direction information;

[0160] A first interpolation module, configured to insert first interpolation pixels at a target position according to the first direction information;

[0161] An update module, configured to update the first interpolation pixels according to the second direction information;

[0162] A second interpolation module, configured to perform interpolation on a second channel based on first channel pixels obtained after updating the first interpolation pixels to obtain a second image.

[0163] It should be noted that specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein. The above-mentioned modules may be functional modules or program modules, and may be implemented by software or hardware. For modules implemented by hardware, the above-mentioned modules may be located in the same processor; or the above-mentioned modules may also be located in different processors in any combination form.

[0164] In addition, in combination with the image processing method for suppressing false information provided in the above embodiments, a storage medium may also be provided in this embodiment to implement it. A computer program is stored on the storage medium; when the computer program is executed by a processor, any one of the image processing methods for suppressing false information in the above embodiments is implemented.

[0165] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0166] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0167] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0168] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An image processing method for suppressing false information, characterized in that: include: Acquire a first image to be processed, determine a target position in the first image where first channel interpolation is to be performed, and determine an operation area with the target position as the center; wherein the number of pixels in the first channel in the first image is not less than the number of pixels in the second channel; Calculate first direction information of original second channel pixels in the target position based on the operation area; filtering the first direction information to obtain second direction information; inserting a first interpolation pixel at the target position according to the first direction information; updating the first interpolation pixel according to the second direction information; interpolating the second channel based on the first channel pixels obtained after updating the first interpolation pixels to obtain a second image; The step of calculating the first direction information of the original second channel pixel in the target position based on the operation area includes: In the horizontal direction of the rows of the operation area, a first mean value of the absolute values ​​of the differences between adjacent pixels of the first channel is calculated, and in the horizontal direction of the rows of the operation area, a second mean value of the absolute values ​​of the differences between adjacent pixels of the second channel is calculated; Adding the first mean value and the second mean value to obtain first horizontal direction information; Calculating a third mean of absolute values ​​of differences between adjacent pixels of the first channel in a vertical direction of a column of the operation area, and calculating a fourth mean of absolute values ​​of differences between adjacent pixels of the second channel in a vertical direction of a column of the operation area; Adding the third mean value and the fourth mean value to obtain first vertical direction information; The first direction information is obtained according to the first horizontal direction information and the first vertical direction information.

2. The image processing method for suppressing false information according to claim 1, characterized in that: Inserting a first interpolation pixel at the target position according to the first direction information includes: comparing the first horizontal direction information with the first vertical direction information; If the first horizontal direction information is less than the first vertical direction information, calculating the value of the first interpolated pixel along the row direction of the target position; If the horizontal direction information of the first direction information is greater than the vertical direction information, calculating the value of the first interpolation pixel along the column direction of the target position; If the horizontal direction information of the first direction information is equal to the vertical direction information, the value of the first interpolation pixel is calculated along the row direction and the column direction of the target position.

3. The image processing method for suppressing false information according to claim 1, characterized in that: Filtering the first direction information to obtain second direction information includes: Dividing the operating area into a plurality of sub-areas; In each sub-region, a first pixel having the same channel attribute as an original pixel at the target position and a second pixel having a different channel attribute from the original pixel are identified; wherein the second pixel does not include the first channel pixel; In a specific direction corresponding to the sub-region, calculating a sum of the direction information of the first pixels, and calculating a mean of the direction information of the second pixels; The sum of the direction information of the first pixels and the mean of the direction information of the second pixels are added to obtain the second direction information of each sub-region.

4. The image processing method for suppressing false information according to any one of claims 1 to 3, characterized in that: The second direction information includes: second horizontal left direction information, second horizontal right direction information, second vertical upper direction information, and second vertical lower direction information.

5. The image processing method for suppressing false information according to claim 4, characterized in that: The sub-regions of the operation region include a left half sub-region, a right half sub-region, an upper half sub-region, and a lower half sub-region, and filtering the first direction information to obtain second direction information includes: For the left half sub-region: on the central row where the target position is located, the first horizontal direction information of the first pixel is added to obtain a first value; on the neighboring row adjacent to the central row, the mean value of the first horizontal direction information of the second pixel is added to obtain a second value; the first value and the second value are added to obtain the second horizontal left direction information; For the right half sub-region: on the central row, add the first horizontal direction information of the first pixels to obtain a third value; on a neighboring row adjacent to the central row, add the mean values ​​of the first horizontal direction information of the second pixels to obtain a fourth value; add the third value and the fourth value to obtain the second horizontal right direction information; For the upper sub-region: on the central column where the target position is located, the first vertical direction information of the first pixels is added to obtain a fifth value; on the neighboring column adjacent to the central column, the mean values ​​of the first vertical direction information of the second pixels are added to obtain a sixth value; and the fifth value and the sixth value are added to obtain the second vertical upper direction information; For the lower half sub-area: on the central column, add the first vertical direction information of the first pixels to obtain a seventh value; on the neighboring column adjacent to the central column, add the mean values ​​of the first vertical direction information of the second pixels to obtain an eighth value; add the seventh value and the eighth value to obtain the second vertical downward direction information.

6. The image processing method for suppressing false information according to claim 1, characterized in that: Updating the first interpolation pixel according to the second direction information includes: Determining pixel update information of the target position; Determine pixel update information of a non-target position in the operation area according to the second direction information; The first interpolation pixel is updated according to the pixel update information of the target position and the pixel update information of the non-target position to obtain a first interpolation pixel update value.

7. The image processing method for suppressing false information according to claim 6, characterized in that: After updating the first interpolated pixel according to the pixel update information of the target position and the pixel update information of the non-target position to obtain a first interpolated pixel update value, the method further includes: comparing the first interpolated pixel update value of the target position with the first interpolated pixel update value adjacent to the target position; If the first interpolation pixel update value of the target position is greater than the maximum value of the first interpolation pixel update values ​​adjacent to the target position, inserting the maximum value of the first interpolation pixel update values ​​adjacent to the target position into the target position; If the first interpolation pixel update value of the target position is smaller than a minimum value of the first interpolation pixel update values ​​adjacent to the target position, the minimum value of the first interpolation pixel update values ​​adjacent to the target position is inserted into the target position.

8. The image processing method for suppressing false information according to claim 1, characterized in that: Interpolating the second channel based on the first channel pixels obtained after updating the first interpolation pixels to obtain a second image includes: Determine a second channel pixel that is adjacent to the position to be interpolated and has the same channel attribute as the second interpolation pixel to be inserted; The second interpolation pixel is calculated based on the determined second channel pixel, the original pixel at the position to be interpolated, and the first interpolation pixel adjacent to the position to be interpolated and updated, and the second interpolation pixel is inserted into the position to be interpolated.

9. An image processing device for suppressing false information, characterized in that: The device comprises: an acquisition module, used for acquiring a first image to be processed, determining a target position in the first image for first channel interpolation, and determining an operation area with the target position as the center; wherein the number of pixels in the first channel in the first image is not less than the number of pixels in the second channel; A first direction information calculation module, used for calculating the first direction information of the original second channel pixel in the target position based on the operation area; A second direction information calculation module, used for filtering the first direction information to obtain second direction information; A first interpolation module, configured to insert a first interpolation pixel at the target position according to the first direction information; An updating module, configured to update the first interpolation pixel according to the second direction information; A second interpolation module, configured to interpolate the second channel based on the first channel pixels obtained after updating the first interpolation pixels, so as to obtain a second image; The step of calculating the first direction information of the original second channel pixel in the target position based on the operation area includes: In the horizontal direction of the rows of the operation area, a first mean value of the absolute values ​​of the differences between adjacent pixels of the first channel is calculated, and in the horizontal direction of the rows of the operation area, a second mean value of the absolute values ​​of the differences between adjacent pixels of the second channel is calculated; Adding the first mean value and the second mean value to obtain first horizontal direction information; Calculating a third mean of absolute values ​​of differences between adjacent pixels of the first channel in a vertical direction of a column of the operation area, and calculating a fourth mean of absolute values ​​of differences between adjacent pixels of the second channel in a vertical direction of a column of the operation area; Adding the third mean value and the fourth mean value to obtain first vertical direction information; The first direction information is obtained according to the first horizontal direction information and the first vertical direction information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images