Image purple edge removing method and device and electronic equipment

Abstract

The invention discloses an image purple edge removing method and device and electronic equipment, and belongs to the technical field of image processing. The method comprises the following steps: carrying out normalized cross-correlation calculation on first channel image data of an R channel and second channel image data of a G channel in first RGB image data to obtain a first local offset, and carrying out normalized cross-correlation calculation on third channel image data of a B channel in the first RGB image data and the third channel image data to obtain a second local offset; obtaining a second local offset; and correcting the color value of the R channel of the pixel point to be corrected according to the first local offset, and correcting the color value of the B channel of the pixel point to be corrected according to the second local offset to obtain second RGB image data.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a method, apparatus, and electronic device for removing purple fringing from images. Background Technology

[0002] During digital image acquisition and processing, due to factors such as defects in the optical system, sensor characteristics, or shooting conditions, the brightness contrast of the scene being photographed is large. Color spots are prone to appear at the boundary between the highlight and low-light areas. These color spots are usually called purple fringes.

[0003] Currently, existing methods for removing purple fringes use a fixed offset to compensate for the acquired RGB image data. This results in the same offset compensation being applied to the pixels corresponding to the purple fringes in all RGB image data. Consequently, the offset compensation for the pixels corresponding to the purple fringes in some RGB image data is insufficient, while the offset compensation for the pixels corresponding to the purple fringes in other RGB image data is excessive, leading to poor removal results for the purple fringes in RGB image data. Summary of the Invention

[0004] The purpose of this application is to provide an image purple fringing removal method, apparatus, electronic device, and storage medium. For each frame of an image, the determined first local offset and second local offset are different. Therefore, purple fringing removal can be performed in a targeted manner using the first local offset and the second local offset for different frames of images, thereby improving the accuracy of purple fringing removal.

[0005] In a first aspect, embodiments of this application provide a method for removing purple fringing from an image, the method comprising:

[0006] A normalized cross-correlation calculation is performed on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain a first local offset. A normalized cross-correlation calculation is also performed on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain a second local offset.

[0007] Based on the first local offset, the color value of the R channel of the pixel to be corrected is corrected, and based on the second local offset, the color value of the B channel of the pixel to be corrected is corrected to obtain the second RGB image data.

[0008] Secondly, embodiments of this application provide an information processing apparatus, which includes:

[0009] The processing module is used to perform normalized cross-correlation calculation on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain a first local offset, and to perform normalized cross-correlation calculation on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain a second local offset.

[0010] The correction module is used to correct the color value of the R channel of the pixel to be corrected according to the first local offset, and to correct the color value of the B channel of the pixel to be corrected according to the second local offset, so as to obtain the second RGB image data.

[0011] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can run on the processor, and the programs or instructions, when executed by the processor, implement the method as described in the first aspect.

[0012] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described in the first aspect.

[0013] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0014] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0015] In this embodiment, by performing normalized cross-correlation calculations on the first channel image data of the R channel and the third channel image data of the B channel in the first RGB image data, respectively, with the second channel image data of the G channel, a first local offset of the first channel image data relative to the second channel image data and a second local offset of the third channel image data relative to the second channel image data can be obtained. Then, the color value of the R channel of the pixel to be corrected can be corrected according to the first local offset, and the color value of the B channel of the pixel to be corrected can be corrected according to the second local offset, to obtain the second RGB image data. Thus, for each frame of image, the image data of the R channel, B channel, and G channel are different, and the first local offset and the second local offset are also different. Therefore, the purple fringing removal can be performed in a targeted manner using the first local offset and the second local offset for different frames of image, rather than using a fixed offset to compensate for the pixels in the traditional way, thus improving the accuracy of purple fringing removal. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an image purple fringing removal method provided in some embodiments of this application;

[0017] Figure 2 This is a flowchart illustrating an image purple fringing removal method provided in some embodiments of this application;

[0018] Figure 3 This is a schematic diagram of the structure of an image purple fringing removal device provided in some embodiments of this application;

[0019] Figure 4 These are schematic diagrams of the structure of electronic devices provided in some embodiments of this application;

[0020] Figure 5 These are schematic diagrams of the hardware structure of electronic devices provided in some embodiments of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0022] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or N objects. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0023] The terminology used in the implementation section of this application is only for explaining specific embodiments of this application and is not intended to limit this application. The terminology involved in the embodiments of this application is explained below.

[0024] Purple fringing: A phenomenon in which purple spots appear at the edges of objects in high-backlight conditions during the image acquisition process.

[0025] RGB: The RGB color mode is an industry color standard that uses variations in the three color channels—red (R), green (G), and blue (B)—and their superposition to obtain a variety of colors. RGB represents the colors of the three channels: red, green, and blue.

[0026] An icon is a highly simplified, abstract, stylized, and easily recognizable graphic symbol used to efficiently convey information, identify functions, or represent entities in digital interfaces or physical environments. It conveys specific information, concepts, functions, brands, or objects through simple graphics or images, primarily used for interface interaction, navigation, and information identification.

[0027] Interface: Refers to the graphical interactive layer seen by users through the screen of an electronic device. Also known as the "user interface" (UI), it is the medium through which applications or operating systems interact and exchange information with users, converting the internal form of information into a form acceptable to the user. The user interface is source code written in specific computer languages ​​such as Java and XML. This source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. The most common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements displayed on the screen of an electronic device, such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and web widgets.

[0028] Application: A computer program developed to perform one or more specific tasks and running on an operating system. Applications run in user mode, can interact with the user, and have a visual user interface.

[0029] Photo preview interface: This is the real-time view that the user sees on the device screen before taking a photo. It is a visual interactive area that is rendered in real time after the image data captured by the camera sensor is processed. It can be a GUI, which can display visible interface elements such as buttons, navigation bars, and widgets.

[0030] Normalized cross-correlation (NCC) is a mathematical method used to measure the similarity between two signals, typically one-dimensional or two-dimensional signals, such as an image. The NCC value ranges from -1 to 1. NCC = 1 indicates that the template T and image window I are identical, exhibiting the highest positive correlation. NCC = 0 indicates that the template T and image window I are completely uncorrelated. NCC = -1 indicates that the template T and image window I are completely opposite, meaning they are negatively correlated.

[0031] The technical solutions of this application can be applied to scenarios where purple-edged image data is removed from image data during image capture, or where purple-edged region data is removed from video frames during video recording.

[0032] The image purple fringing removal method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0033] Figure 1 This is a schematic flowchart of an image purple fringing removal method provided in an embodiment of this application. The subject executing the image purple fringing removal method can be an electronic device, which can be, but is not limited to, a personal computer (PC), a smartphone, a tablet computer, or a personal digital assistant (PDA).

[0034] like Figure 1 As shown, the image purple fringing removal method provided in this application embodiment may include steps 110-120.

[0035] Step 110: Perform normalized cross-correlation calculation on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain the first local offset, and perform normalized cross-correlation calculation on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain the second local offset.

[0036] The first RGB image data can be RGB format image data of the scene to be photographed, acquired when the scene is photographed. The scene to be photographed can be a roof, as in the example above, or a tree, as in the example above.

[0037] In some embodiments of this application, the user clicks the camera application icon on the electronic device, and the camera application starts running, displaying a photo preview interface. During the operation of the camera application, the contact image sensor (CIS) in the camera continuously acquires RAW format image data, and then sends the continuously acquired RAW format image data to the image signal processor (ISP) for image processing such as de-mosaicing, white balance calibration, noise reduction, sharpening, and dynamic range optimization to obtain RGB format image data, which is the first RGB image data.

[0038] It should be noted that after obtaining RGB format image data, the color space of the RGB format image data can be converted to YUV format image data. This YUV format image data is then sent to an image data cache space for caching. The display module of the electronic device can retrieve the cached YUV format image data from the image data cache space in real time, render and display the YUV format image data in real time, thus displaying the preview image in real time on the photo preview interface. When the user clicks the shooting control in the shooting interface, the preview image displayed in real time can be imaged to obtain the captured image. Alternatively, when the user clicks the video recording control in the shooting interface, the preview image displayed in real time can be recorded as a video to obtain the recorded video. Therefore, the solution in this embodiment is executed before the preview image is displayed in real time on the photo preview interface, or it can be executed before imaged from the preview image to obtain the captured image, or it can be executed before video recording from the real-time displayed preview image to obtain the recorded video. This embodiment does not limit the implementation of this solution.

[0039] RGB image data is a three-dimensional data set, which takes the following form:

[0040]

[0041] In the above three-dimensional data set, m and n are the length and width of the photosensitive unit in the CIS, respectively. The length and width of the photosensitive unit represent the resolution of the CIS. If the resolution of the CIS is 4000*3000, then m and n are 4000 and 3000, respectively. The number of pixels in the final image of the RGB image data acquired by the CIS is 4000*3000 = 12 million.

[0042] In the aforementioned three-dimensional data set, each element represents the intensity values ​​of the R, B, and G channels corresponding to that pixel. Specifically, in the three-dimensional data set (R11, G11, B11), R11, G11, and B11 respectively represent the color values ​​of the R, B, and G channels of the first pixel in the RGB image data. For RGB image data, I is typically used... C (x,y) represents the pixel value of the pixel at coordinates (x,y), where x and y are the length and width of the photosensitive unit, respectively, and C∈{R,G,B} represents three channels.

[0043] The aforementioned first channel image data can be the image data of the R channel in the first RGB image data. Taking the data structure of the above three-dimensional data group as the first RGB image data as an example, the form of the first channel image data is as follows:

[0044]

[0045] The second channel image data can be the image data of the G channel in the first RGB image data. Taking the above three-dimensional data set as the data structure of the first RGB image data as an example, the form of the second channel image data is as follows:

[0046]

[0047] The third channel image data can be the B channel image data from the first RGB image data. Taking the above three-dimensional data set as the first RGB image data as an example, the form of the third channel image data is as follows:

[0048]

[0049] The first local offset can be the offset of the first channel image data of the R channel relative to the first channel image data of the G channel. The second local offset can be the offset of the first channel image data of the B channel relative to the first channel image data of the G channel.

[0050] It should be noted that the first channel image data, the second channel image data, and the third channel image data mentioned above are all two-dimensional data sets.

[0051] It should be noted that, when performing step 110 above, one can first perform a normalized cross-correlation calculation on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain the first local offset, and then perform a normalized cross-correlation calculation on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain the second local offset. Alternatively, one can first perform a normalized cross-correlation calculation on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain the second local offset, and then perform a normalized cross-correlation calculation on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain the first local offset. Alternatively, the normalized cross-correlation calculation of the first R channel image data and the second G channel image data in the first RGB image data can be performed simultaneously to obtain a first local offset, and the normalized cross-correlation calculation of the third B channel image data and the third channel image data in the first RGB image data can be performed to obtain a second local offset. Specifically, the execution order of the two operations of performing the normalized cross-correlation calculation of the first R channel image data and the second G channel image data in the first RGB image data to obtain the first local offset and the normalized cross-correlation calculation of the third B channel image data and the third channel image data in the first RGB image data to obtain the second local offset can be set according to user needs and is not limited in this embodiment.

[0052] In some embodiments of this application, each first element in the first channel image data is used to indicate a pixel, and a first element is the color value of the pixel indicated by the first element in the R channel. The aforementioned first element refers to an element in the first channel image data, that is, R11, R12, etc. in the aforementioned first channel data are all first elements. Each first element is used to indicate a pixel, that is, R11 is used to indicate a pixel, and R12 is also used to indicate a pixel. For example, R11 is used to indicate the first pixel, and R12 is used to indicate the second pixel. R11 is the color value of the first pixel in the R channel, and R12 is the color value of the second pixel in the R channel.

[0053] Correspondingly, in the second channel image data, each second element indicates a pixel, and a second element is the color value of the pixel indicated by that second element in the G channel. In the third channel image data, each third element indicates a pixel, and a third element is the color value of the pixel indicated by that third element in the B channel. The aforementioned second elements are elements in the second channel image data, and the third elements are elements in the third channel image data.

[0054] In the first, second, and third channel image data, the pixels indicated by the first element, the second element, and the third element at the same position are the same pixel. That is, the pixels corresponding to elements at the same position in the first, second, and third channel image data and the first RGB image data are the same pixel. For example, (R11, G11, B11) in the first RGB image data, R11 in the first channel image data, G11 in the second channel image data, and B11 in the third channel image data all refer to the first pixel in the first RGB image data. Here, R11 is the color value of the R channel of the first pixel in the first RGB image data, B11 is the color value of the B channel of the first pixel in the first RGB image data, G11 is the color value of the G channel of the first pixel in the first RGB image data, and (R11, G11, B11) is the pixel value of the first pixel in the first RGB image data, which is determined based on the color values ​​of the R, G, and B channels of that pixel.

[0055] To improve the accuracy of purple fringing removal, step 110 may specifically include:

[0056] The color value of the R channel of each pixel is cross-correlated with the color value of the G channel of each pixel in a normalized manner to obtain the first normalized cross-correlation value of each pixel.

[0057] The color value of the B channel of each pixel is cross-correlated with the color value of the G channel of each pixel in a normalized manner to obtain the second normalized cross-correlation value of each pixel.

[0058] The R-channel color value of the pixel corresponding to the maximum value of the first normalized cross-correlation value of each pixel is determined as the first local offset.

[0059] The B-channel color value of the pixel corresponding to the maximum value of the second normalized cross-correlation value of each pixel is determined as the second local offset.

[0060] Specifically, for each pixel corresponding to each RGB image data in the first RGB image data, the first normalized cross-correlation value can be the normalized cross-correlation value obtained after performing normalized cross-correlation calculation on the color value of the pixel in the R channel and the color value of the pixel in the G channel.

[0061] For each pixel corresponding to each RGB image data in the first RGB image data, the second normalized cross-correlation value can be the normalized cross-correlation value obtained by performing normalized cross-correlation calculation on the color value of the pixel in the B channel and the color value of the pixel in the G channel.

[0062] In some embodiments of this application, for each pixel corresponding to each RGB image data in the first RGB image data, the normalized cross-correlation of the pixel's color value in the R channel and its color value in the G channel is calculated according to the following formula (1) to obtain the first normalized cross-correlation value of the pixel:

[0063]

[0064] In the above formula (1), I G This represents the color value of a pixel in the G channel. I is the average of all second elements in channel G. R The color value of the pixel in the R channel. Let u,v be the average of all first elements in channel R, where u,v ∈ [-d]. max ,d max ], d max The maximum estimated offset can be set according to the parameters of the camera lens, or it can be a default value that is directly defined and then adjusted according to the actual effect. In this embodiment, no limitation is made.

[0065] In some embodiments of this application, for each pixel corresponding to each RGB image data in the first RGB image data, the normalized cross-correlation value of the pixel in the B channel and the pixel in the G channel are calculated according to the following formula (2) to obtain the second normalized cross-correlation value of the pixel:

[0066]

[0067] In the above formula (2), I G This represents the color value of a pixel in the G channel. I is the average of all second elements in channel G. B This represents the color value of the pixel in the B channel. Let u,v be the average of all third elements in channel B, where u,v ∈ [-d]. max ,d max ], d max This represents the maximum estimated offset.

[0068] After obtaining the first normalized cross-correlation value corresponding to each pixel in the first RGB image data according to the above formula (1), the NCC value can be selected from all the first normalized cross-correlation values ​​corresponding to the pixels. R The R-channel color value of the pixel corresponding to the maximum value of (u,v) is determined as the first local offset of the R-channel relative to the G-channel.

[0069] Similarly, after obtaining the second normalized cross-correlation value corresponding to each pixel in the first RGB image data according to the above formula (2), the NCC value can be selected from all the second normalized cross-correlation values ​​corresponding to the pixels. B The B channel color value of the pixel corresponding to the maximum value of (u,v) is determined as the second local offset of the B channel relative to the G channel.

[0070] It should be noted that in the embodiments of this application, the G color channel is used as the reference channel. That is, when the R color channel and B color channel are both normalized and cross-correlated with the G color channel, it is assumed that the green channel generally contains more details. This is because the human retina is most sensitive to green, so the G color channel can be used as the reference channel.

[0071] In the embodiments of this application, since the image data of the R channel, B channel, and G channel are different for each frame of image, for each pixel corresponding to each RGB image data in the first RGB image data, the first local offset of the R channel relative to the G channel and the second local offset of the B channel relative to the G channel are determined by performing normalized cross-correlation calculation on the color value of the pixel in the R channel and the color value of the pixel in the G channel, and by performing normalized cross-correlation calculation on the color value of the pixel in the B channel and the color value of the pixel in the G channel. The first local offset and the second local offset determined in this way are also different. That is, for each frame of image, when its content changes, the detected first local offset and the second local offset also change dynamically. Therefore, the first local offset and the second local offset can be used to remove purple fringing in a targeted manner for different frames of image, rather than using a fixed offset to compensate for pixels in the traditional way, which improves the accuracy of purple fringing removal in captured images or video streams.

[0072] Step 120: Based on the first local offset, correct the color value of the R channel of the pixel to be corrected, and based on the second local offset, correct the color value of the B channel of the pixel to be corrected, to obtain the second RGB image data.

[0073] Among them, the pixel to be corrected can be a pixel that needs to be corrected, and the pixel to be corrected can be a pixel that has purple fringing.

[0074] The second RGB image data can be RGB format image data obtained by correcting the color values ​​of the R channel and B channel of the pixel to be corrected, based on the corrected color values ​​of the R channel, the corrected color values ​​of the B channel, and the color values ​​of the G channel.

[0075] It should be noted that when executing step 120, the color value of the R channel of the pixel to be corrected can be corrected first based on the first local offset, and then the color value of the B channel of the pixel to be corrected can be corrected based on the second local offset. Alternatively, the color value of the B channel of the pixel to be corrected can be corrected first based on the second local offset, and then the color value of the R channel of the pixel to be corrected can be corrected based on the first local offset. Alternatively, the color values ​​of the R channel and B channel of the pixel to be corrected can be corrected simultaneously based on the first local offset and the second local offset. The specific execution order of correcting the color value of the R channel of the pixel to be corrected based on the first local offset and the color value of the B channel of the pixel to be corrected based on the second local offset can be set according to user needs and is not limited in this embodiment.

[0076] In some embodiments of this application, prior to step 120, the method described above may further include:

[0077] Edge detection is performed on the second channel image data to obtain the edge intensity set of the G channel for each pixel;

[0078] The pixels to be corrected are determined based on the edge intensity set of the G channel.

[0079] The edge intensity set can be the set of edge intensity values ​​of each pixel obtained after edge detection of the second channel image data. That is, the edge intensity set of the G channel includes the edge intensity value of the G channel for each pixel. The edge intensity set of the G channel is a two-dimensional matrix, and each element in the edge intensity set represents the spatial intensity change of the pixel at the corresponding position in the G channel image data.

[0080] In some embodiments of this application, edge detection can be performed on the second channel image data to obtain the edge intensity set of the G channel for each pixel, and then the pixel to be corrected can be determined based on the edge intensity set of the G channel.

[0081] In the embodiments of this application, the pixels to be corrected are determined by the edge intensity set of the G channel, rather than by using a purple fringing detection method with a fixed threshold. Thus, the G channel image data is different for each frame of the image, and the determined pixels to be corrected are also different. That is, for each frame of the image, when the content changes, the detected edge intensity of the G channel is also different, and therefore the detected pixels to be corrected are also dynamically changing. In other words, for each frame of the image, when the content changes, the area for purple fringing repair is also dynamically changing. This allows for targeted determination of the pixels to be corrected for each frame of the image, improving the accuracy of the determination.

[0082] In some embodiments of this application, the step of performing edge detection on the second channel image data to obtain the edge intensity set of the G channel for each pixel may specifically include:

[0083] Calculate the horizontal gradient of the second channel image data to obtain the horizontal gradient matrix;

[0084] The vertical gradient matrix is ​​obtained by calculating the vertical gradient of the second channel image data.

[0085] For each pixel indicated by each element in the second channel image data, the values ​​corresponding to the pixel in the vertical gradient matrix and the values ​​corresponding to the pixel in the horizontal gradient matrix are combined and calculated to obtain the edge intensity value of the pixel.

[0086] Based on the edge intensity value of the pixel indicated by each element in the second channel image data, the edge intensity set of the G channel for each pixel is obtained.

[0087] The horizontal gradient matrix can be a matrix obtained by calculating the horizontal gradient of the second channel image data. Each element in the horizontal gradient matrix represents the intensity change of the corresponding pixel in the G channel image data in the horizontal direction.

[0088] The vertical gradient matrix can be obtained by calculating the vertical gradient of the second channel image data. Each element in the vertical gradient matrix represents the intensity change of the corresponding pixel in the G channel image data in the vertical direction.

[0089] In some embodiments of this application, the horizontal gradient of the second channel image data can be calculated. For example, the horizontal Sobel filter convolution algorithm can be used to calculate the horizontal gradient of the second channel image data using the following formula (3). The intensity change of the corresponding pixel in the G channel image data in the horizontal direction can be obtained. The intensity change of the corresponding pixel in the G channel image data in the horizontal direction is combined to form a two-dimensional matrix, which is the horizontal gradient matrix.

[0090]

[0091] In the above formula (3), G(x,y) refers to the image data value at position (x,y) in the second channel image data of channel G. x (x,y) represents the calculation in the x-direction of the second channel image data of channel G.

[0092] Correspondingly, the vertical gradient can be calculated on the second channel image data. For example, the vertical Sobel filter convolution algorithm can be used to calculate the vertical gradient on the second channel image data using the following formula (4). The intensity change of the corresponding pixel in the vertical direction in the image data of the G channel can be obtained. The intensity change of the corresponding pixel in the vertical direction in the image data of the G channel can be combined to form a two-dimensional matrix, which is the vertical gradient matrix.

[0093]

[0094] In the above formula (4), G y (x,y) represents the calculation in the y-direction of the second channel image data of channel G.

[0095] For each pixel indicated by each element in the second channel image data, the values ​​corresponding to the pixel in the vertical gradient matrix and the values ​​corresponding to the pixel in the horizontal gradient matrix can be combined to calculate the edge intensity value of the pixel. That is, after obtaining the intensity change value in the horizontal direction and the intensity change value in the vertical direction of each pixel, the intensity change value in the horizontal direction and the intensity change value in the vertical direction of each pixel can be combined to calculate the edge intensity value of each pixel. Specifically, the intensity change value in the horizontal direction and the intensity change value in the vertical direction of each pixel can be combined according to the following formula (5) to calculate the edge intensity value of each pixel.

[0096]

[0097] Then, the edge intensity values ​​of the pixels indicated by each element in the second channel image data are combined together to obtain the edge intensity set E corresponding to the G channel. G.

[0098] In the embodiments of this application, a horizontal gradient matrix is ​​obtained by performing horizontal gradient calculation on the second channel image data, and a vertical gradient matrix is ​​obtained by performing vertical gradient calculation on the second channel image data. The resulting horizontal and vertical gradient matrices are two-dimensional matrices, just like the second channel image data, thus maintaining the spatial layout of the first RGB image data. Then, for each pixel indicated by each element in the second channel image data, the values ​​corresponding to the pixel in the vertical gradient matrix and the values ​​corresponding to the pixel in the horizontal gradient matrix are combined to calculate the edge intensity of the pixel. Then, based on the edge intensity value of each pixel indicated by each element in the second channel image data, the edge intensity set of the G channel of each pixel is also the same as the structure of the second channel image data, which is a two-dimensional array. This structure preserves complete spatial information and provides a data foundation for accurately determining the pixels to be corrected.

[0099] In some embodiments of this application, to improve the accuracy of purple fringing removal, the step of determining the pixels to be corrected based on the edge intensity set of the G channel may specifically include:

[0100] Calculate the mean and standard deviation of all edge intensity values ​​in the edge intensity set of the G channel to obtain the mean and standard deviation of the edge intensity.

[0101] The maximum edge intensity of the pixel to be corrected is determined based on the mean edge intensity and the standard deviation of edge intensity.

[0102] The pixels with edge intensity values ​​greater than the maximum edge intensity value in the edge intensity set of the G channel are identified as the pixels to be corrected.

[0103] The maximum edge strength value can be the maximum edge strength value of a given pixel to be corrected.

[0104] In some embodiments of this application, the average edge intensity value μ is obtained by averaging all edge intensity values ​​in the edge intensity set of the G channel according to the following formula (6). E :

[0105]

[0106] In the above formula (6), N is the number of elements in the edge intensity set of the G channel.

[0107] According to the following formula (7), based on the mean edge intensity, the standard deviation of the edge intensity σ can be obtained by calculating the standard deviation of all edge intensity values ​​in the edge intensity set of the G channel. E :

[0108]

[0109] Then, based on the mean edge strength and standard deviation of edge strength, the maximum edge strength T of the pixel to be corrected can be determined according to the following formula (8). high :

[0110] T high =μ E +2σ E (8)

[0111] Since purple fringing typically occurs at pixels with high edge intensity, pixels with edge intensity values ​​greater than the maximum edge intensity value in the edge intensity set corresponding to the G channel can be identified as pixels to be corrected. That is, there must be at least one pixel to be corrected.

[0112] In the embodiments of this application, the mean and standard deviation of edge intensity values ​​in the edge intensity set of the G channel are calculated to obtain the mean and standard deviation of edge intensity. Then, based on the mean and standard deviation of edge intensity, the maximum edge intensity of the pixel to be corrected is determined. Then, the pixel corresponding to the edge intensity greater than the maximum edge intensity in the edge intensity set of the G channel is determined as the pixel to be corrected. Since the edge intensity set corresponding to the G channel changes when the content of each frame of the image changes, the maximum edge intensity of each frame is dynamically changing. Thus, the pixel to be corrected is also dynamically changing. In this way, the pixel to be corrected is different for each frame, and purple fringing removal can be performed on each frame in a targeted manner, rather than the traditional purple fringing removal of fixed pixels, thus improving the accuracy of purple fringing removal.

[0113] In some embodiments of this application, to further improve the accuracy of purple fringing removal, step 120 may specifically include:

[0114] Based on the first local offset, the color value of the pixel to be corrected in the R channel is corrected to obtain the first corrected color value;

[0115] Based on the second local offset, the color value of the pixel to be corrected in the B channel is corrected to obtain the second corrected color value;

[0116] The first corrected color value, the second corrected color value, and the second channel image data are merged to obtain the second RGB image data.

[0117] The first corrected color value can be the color value obtained after correcting the color value of the pixel to be corrected in the R channel based on the first local offset.

[0118] The second corrected color value can be the color value obtained after correcting the color value of the pixel to be corrected in the B channel based on the second local offset.

[0119] In some embodiments of this application, since the number of pixels to be corrected is at least one, for each pixel to be corrected, its color value in the R channel can be corrected using the first local offset according to the following formula (9) to obtain its corresponding first corrected color value.

[0120]

[0121] In the above formula (9), (Δx) R ,Δy R (x, y) represents the first local offset, and (x, y) represents the coordinates of the pixel to be corrected.

[0122] Similarly, for each pixel in the pixel to be corrected, its color value in the B channel can be corrected using the second local offset according to the following formula (10) to obtain its corresponding second corrected color value.

[0123]

[0124] In the above formula (10), (Δx) B ,Δy B (x,y) represents the second local offset, and (x,y) represents the coordinates of the pixel to be corrected.

[0125] Thus, by merging the color values ​​based on the first corrected color value, the second corrected color value, and the second channel image data of the G channel, that is, merging the first corrected color value, the second corrected color value, and the second channel image data of the G channel for each pixel, and merging them into the RGB format image data of the above three-dimensional structure, the second RGB image data can be obtained.

[0126] In the embodiments of this application, since the determined first local offset, second local offset, and pixel to be corrected are different for each frame of image, the color value of the pixel to be corrected in the R channel can be corrected specifically based on the first local offset of the R channel relative to the G channel, and the color value of the pixel to be corrected in the B channel can be corrected based on the second local offset of the B channel relative to the G channel, instead of using a fixed offset to compensate for a fixed pixel for the entire first RGB image data, thus improving the accuracy of purple fringing removal.

[0127] In some embodiments of this application, after averaging and calculating the standard deviation of all edge intensity values ​​in the edge intensity set of the G channel to obtain the mean edge intensity and the standard deviation of the edge intensity, the method described above may further include:

[0128] Based on the average edge intensity, determine the minimum edge intensity of the pixel to be corrected;

[0129] The step of merging the first corrected color value, the second corrected color value, and the second channel image data to obtain the second RGB image data may specifically include:

[0130] The smoothing weight factor of the pixel to be corrected is determined based on the edge intensity set, the minimum edge intensity, and the maximum edge intensity.

[0131] The first color value of the pixel to be corrected in the R channel is determined based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the first channel image data, and the first corrected color value of the pixel to be corrected.

[0132] The second color value of the pixel to be corrected in the B channel is determined based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the third channel image data, and the second corrected color value of the pixel to be corrected.

[0133] The first color value, the second color value, and the second channel image data of the pixel to be corrected are merged to obtain the second RGB image data.

[0134] The minimum edge strength value can be the minimum edge strength of a given pixel to be corrected. The maximum and minimum edge strength values ​​define the range of pixels to be corrected subsequently.

[0135] For each pixel in the pixel to be corrected, its corresponding smoothing weight factor can be obtained based on the edge intensity value, the minimum edge intensity value and the maximum edge intensity value in the edge intensity set of the G channel. The smoothing weight factor is to make the transition between the first corrected color value of the pixel to be corrected in the R channel and the color value of the surrounding pixels natural, and to make the transition between the second corrected color value of the pixel to be corrected in the B channel and the color value of the surrounding pixels natural.

[0136] For each pixel in the pixel to be corrected, the first color value can be the color value of the pixel to be corrected in the R channel, which is determined based on the smoothing weight factor, the color value of the pixel to be corrected in the first channel image data, and the first corrected color value of the pixel to be corrected. In the R channel, the first color value of the pixel to be corrected transitions naturally with the color values ​​of the surrounding pixels.

[0137] The second color value can be the color value of the pixel to be corrected in the B channel, determined based on the smoothing weight factor, the color value of the pixel to be corrected in the third channel image data, and the second corrected color value of the pixel to be corrected. In the B channel, the second color value of the pixel to be corrected transitions naturally with the color values ​​of the surrounding pixels.

[0138] In some embodiments of this application, the minimum edge intensity T of the pixel to be corrected can be determined according to the mean and the following formula (11). low :

[0139] T low =a·μ E (11)

[0140] In the above formula (9), a is a constant with a value of [0,1]. This constant can be set according to the user's needs, for example, a can be 0.3.

[0141] Then, based on the minimum and maximum edge strength values, and the edge strength set, the smoothing weight factor of the pixel to be corrected can be obtained according to the following formula (12). Since there is at least one pixel to be corrected, for each pixel to be corrected, the smoothing weight factor α corresponding to that pixel can be obtained according to the following formula (12) based on the minimum and maximum edge strength values, and the edge strength value of that pixel in the edge strength set:

[0142]

[0143] In the above formula (12), E G (x,y) represents the coordinates of any pixel among the pixels to be corrected.

[0144] For each pixel in the pixel to be corrected, after obtaining the smoothing correction factor of the pixel, the first color value of the pixel in the R channel can be obtained according to the following formula (13) based on the smoothing weight factor of the pixel, the color value of the pixel in the first channel image data, and the first corrected color value of the pixel.

[0145]

[0146] In the above formula (13), This is the color value of the pixel in the first channel image data.

[0147] Similarly, for each pixel in the pixel to be corrected, after obtaining the smoothing correction factor of that pixel, the second color value of that pixel in the B channel can be obtained according to the following formula (14) based on the smoothing weight factor of that pixel, the color value of that pixel in the third channel image data, and the second corrected color value of that pixel.

[0148]

[0149] In the above formula (14), This is the color value of the pixel in the third channel image data.

[0150] After calculating each pixel in the pixel to be corrected according to the above formulas (13) and (14), the first color value and the second color value of each pixel in the pixel to be corrected can be obtained. Then, based on the first color value, the second color value of each pixel in the pixel to be corrected, the color value of the pixel to be corrected in the G channel, and the color values ​​of other pixels before the pixel to be corrected in each color channel, the color values ​​of the pixels are merged, and the second RGB image data can be obtained.

[0151] In the embodiments of this application, by calculating the smoothing weight factor of the pixel to be corrected, the first corrected color value and the second corrected color value are smoothly transitioned according to the smoothing weight factor. This makes the color value of the pixel to be corrected in the R channel and B channel of the second RGB image data transition naturally with the color value of the surrounding pixels of the pixel to be corrected. In turn, the color transition of each pixel in the image after removing purple fringing obtained based on the second RGB image data is natural, thereby improving the quality of the image after removing purple fringing.

[0152] In some embodiments of this application, since purple fringing generally appears at the edges of an image, such as the gap edge between adjacent leaves in the example above, the intensity value of these edges is generally greater than the minimum edge intensity value mentioned above. Therefore, when performing step 110, RGB image data with an intensity greater than the minimum edge intensity value can be selected from the first RGB image data for calculation. That is, the first channel image data of the R channel and the third channel image data of the B channel of the RGB image data with an intensity greater than the minimum edge intensity value in the first RGB image data are respectively subjected to normalized cross-correlation calculation with the second channel image data of the G channel to obtain the first local offset of the R channel relative to the G channel and the second local offset of the B channel relative to the G channel. In this way, it is not necessary to calculate all image data of the R channel and all image data in the B channel, which reduces the amount of data calculation and improves the efficiency of data calculation.

[0153] To better understand the solutions of the embodiments of this application, another implementation of the image purple fringing removal method is also provided in the embodiments of this application.

[0154] Figure 2 This is a schematic flowchart illustrating an image purple fringing removal method according to an exemplary embodiment. Figure 2 As shown, the image purple fringing removal method may include steps 201-206.

[0155] Step 201: Obtain RAW image data.

[0156] In step 201, with the camera application of the electronic device running, the contact image sensor (CIS) in the camera begins to continuously acquire RAW format image data.

[0157] Step 202: Convert the RAW image data to RGB format to obtain the first RGB image data.

[0158] In step 202, the continuously acquired RAW format image data is sent to the ISP for image processing such as de-mosaicing, white balance calibration, noise reduction, sharpening, and dynamic range optimization, and then RGB format image data is obtained, which is the first RGB image data in step 110 of the above embodiment.

[0159] Step 203: Perform normalized cross-correlation calculation on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain the first local offset, and perform normalized cross-correlation calculation on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain the second local offset.

[0160] Step 203 is the same as step 110 above, and will not be repeated here.

[0161] Step 204: Determine the pixels to be corrected.

[0162] In step 204, when determining the pixel to be corrected, it can be achieved by performing edge detection on the second channel image data to obtain the edge intensity set of the G channel for each pixel, and then determining the pixel to be corrected based on the edge intensity set of the G channel, as described in the above embodiment.

[0163] Step 205: Correct the color value of the pixel to be corrected in the R channel according to the first local offset to obtain the first corrected color value, and correct the color value of the pixel to be corrected in the B channel according to the second local offset to obtain the second corrected color value.

[0164] In step 205, when correcting the pixel to be corrected, the color value of the R channel of the pixel to be corrected can be corrected according to the first local offset to obtain the first corrected color value, as in the above embodiment, and the color value of the B channel of the pixel to be corrected can be corrected according to the second local offset to obtain the second corrected color value.

[0165] Step 206: Based on the smoothing weight factor of the pixel to be corrected, perform pixel smoothing on the first corrected color value and the second corrected color value to obtain the second RGB image data.

[0166] In step 206, the smoothing weight factor of the pixel to be corrected can be obtained first according to the process in the above embodiment. Specifically, as in the above embodiment, the average and standard deviation of all edge intensity values ​​in the edge intensity set of the G channel can be calculated to obtain the mean edge intensity and the standard deviation edge intensity. Then, based on the mean edge intensity and the standard deviation edge intensity, the maximum edge intensity of the pixel to be corrected can be obtained. Based on the mean edge intensity, the minimum edge intensity of the pixel to be corrected can be determined. Based on the edge intensity set of the G channel, the minimum edge intensity, and the maximum edge intensity, the smoothing weight factor of the pixel to be corrected can be determined.

[0167] Then, based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the first channel image data, and the first corrected color value of the pixel to be corrected, the first color value of the pixel to be corrected in the R channel is determined.

[0168] The second color value of the pixel to be corrected in the B channel is determined based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the third channel image data, and the second corrected color value of the pixel to be corrected.

[0169] Then, the first and second target color values ​​of the pixels to be corrected, along with the second channel image data, are merged to obtain the second RGB image data.

[0170] The image purple fringing removal method provided in this application can be executed by an image purple fringing removal device. This application uses an image purple fringing removal device to perform the image purple fringing removal method as an example to illustrate the image purple fringing removal device provided in this application.

[0171] Figure 3 This is a schematic diagram illustrating the structure of an image purple fringing removal device according to an exemplary embodiment. Figure 3 As shown, the image purple fringing removal device 300 may include:

[0172] The processing module 310 is used to perform normalized cross-correlation calculation on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain a first local offset, and to perform normalized cross-correlation calculation on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain a second local offset.

[0173] The correction module 320 is used to correct the color value of the R channel of the pixel to be corrected according to the first local offset, and to correct the color value of the B channel of the pixel to be corrected according to the second local offset, so as to obtain the second RGB image data.

[0174] In this embodiment, by performing normalized cross-correlation calculations on the first channel image data of the R channel and the third channel image data of the B channel in the first RGB image data, respectively, with the second channel image data of the G channel, a first local offset of the first channel image data relative to the second channel image data and a second local offset of the third channel image data relative to the second channel image data can be obtained. Then, the color value of the R channel of the pixel to be corrected can be corrected according to the first local offset, and the color value of the B channel of the pixel to be corrected can be corrected according to the second local offset, to obtain the second RGB image data. Thus, for each frame of image, the image data of the R channel, B channel, and G channel are different, and the first local offset and the second local offset are also different. Therefore, the purple fringing removal can be performed in a targeted manner using the first local offset and the second local offset for different frames of image, rather than using a fixed offset to compensate for the pixels in the traditional way, thus improving the accuracy of purple fringing removal.

[0175] In some embodiments of this application, the processing module 310 is specifically used for:

[0176] The color value of the R channel of each pixel is cross-correlated with the color value of the G channel of each pixel in a normalized manner to obtain the first normalized cross-correlation value of each pixel.

[0177] The color value of the B channel of each pixel is cross-correlated with the color value of the G channel of each pixel in a normalized manner to obtain the second normalized cross-correlation value of each pixel.

[0178] The R channel color value of the pixel corresponding to the maximum value of the first normalized cross-correlation value of each pixel is determined as the first local offset.

[0179] The B-channel color value of the pixel corresponding to the maximum value of the second normalized cross-correlation value of each pixel is determined as the second local offset.

[0180] In some embodiments of this application, the apparatus may further include:

[0181] An edge detection module is used to perform edge detection on the second channel image data before correcting the color value of the R channel of the pixel to be corrected according to the first local offset and correcting the color value of the B channel of the pixel to be corrected according to the second local offset to obtain the second RGB image data, thereby obtaining an edge intensity set of the G channel for each pixel; the edge intensity set includes the edge intensity value of the G channel for each pixel.

[0182] The processing module 310 is also used to determine the pixel to be corrected based on the edge intensity set of the G channel.

[0183] In some embodiments of this application, the processing module 310 is specifically used for:

[0184] The mean and standard deviation of all edge intensity values ​​in the edge intensity set of the G channel are calculated to obtain the mean edge intensity and the standard deviation of edge intensity.

[0185] The maximum edge intensity of the pixel to be corrected is determined based on the mean edge intensity and the standard deviation of edge intensity.

[0186] The pixels with edge intensity values ​​greater than the maximum edge intensity value in the edge intensity set of the G channel are identified as the pixels to be corrected.

[0187] In some embodiments of this application, the correction module 320 is specifically used for:

[0188] Based on the first local offset, the color value of the pixel to be corrected in the R channel is corrected to obtain the first corrected color value;

[0189] Based on the second local offset, the color value of the pixel to be corrected in the B channel is corrected to obtain the second corrected color value;

[0190] The first corrected color value, the second corrected color value, and the second channel image data are merged to obtain the second RGB image data.

[0191] In some embodiments of this application, the processing module 310 is further configured to, after calculating the average and standard deviation of the edge intensity values ​​in the edge intensity set to obtain the mean edge intensity and the standard deviation of the edge intensity, determine the minimum edge intensity value of the pixel to be corrected based on the mean edge intensity;

[0192] The calibration module 320 is specifically used for:

[0193] The smoothing weight factor of the pixel to be corrected is determined based on the set of edge intensities, the minimum edge intensity, and the maximum edge intensity.

[0194] Based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the first channel image data, and the first corrected color value of the pixel to be corrected, the first color value of the pixel to be corrected in the R channel is determined.

[0195] Based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the third channel image data, and the second corrected color value of the pixel to be corrected, the second color value of the pixel to be corrected in the B channel is determined.

[0196] The first color value, the second color value, and the second channel image data of the pixel to be corrected are merged to obtain the second RGB image data.

[0197] The image purple fringing removal device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0198] The image purple fringing removal device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0199] The image purple fringing removal device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0200] Optionally, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401 and a memory 402. The memory 402 stores a program or instructions that can run on the processor 401. When the program or instructions are executed by the processor 401, they implement the various steps of the above-described image purple edge removal method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0201] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0202] Figure 5 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0203] The electronic device 500 includes, but is not limited to, components such as: radio frequency unit 501, network module 502, audio output unit 503, input unit 504, sensor 505, display unit 506, user input unit 507, interface unit 508, memory 509, and processor 510.

[0204] Those skilled in the art will understand that the electronic device 500 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 510 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0205] The processor 510 is configured to perform normalized cross-correlation calculation on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain a first local offset, and perform normalized cross-correlation calculation on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain a second local offset; based on the first local offset, the color value of the R channel of the pixel to be corrected is corrected, and based on the second local offset, the color value of the B channel of the pixel to be corrected is corrected to obtain the second RGB image data.

[0206] Thus, by performing normalized cross-correlation calculations on the first channel image data of the R channel and the third channel image data of the B channel in the first RGB image data, respectively, with the second channel image data of the G channel, a first local offset of the first channel image data relative to the second channel image data, and a second local offset of the third channel image data relative to the second channel image data, can be obtained. Then, the color value of the R channel of the pixel to be corrected can be corrected according to the first local offset, and the color value of the B channel of the pixel to be corrected can be corrected according to the second local offset, to obtain the second RGB image data. In this way, the image data of the R channel, B channel, and G channel are different for each frame of the image, and the first local offset and the second local offset are also different. Therefore, the purple fringing removal can be performed in a targeted manner using the first local offset and the second local offset for different frames of the image, rather than using a fixed offset to compensate for the pixels in the traditional way, thus improving the accuracy of purple fringing removal.

[0207] Optionally, the processor 510 is further configured to perform normalized cross-correlation calculation on the color value of the R channel of each pixel and the color value of the G channel of each pixel to obtain a first normalized cross-correlation value for each pixel; perform normalized cross-correlation calculation on the color value of the B channel of each pixel and the color value of the G channel of each pixel to obtain a second normalized cross-correlation value for each pixel; determine the R channel color value of the pixel corresponding to the maximum value of the first normalized cross-correlation value of each pixel as a first local offset; and determine the B channel color value of the pixel corresponding to the maximum value of the second normalized cross-correlation value of each pixel as a second local offset.

[0208] Thus, since the image data of the R, B, and G channels are different for each frame of the image, for each pixel corresponding to each RGB image data in the first RGB image data, the first local offset of the R channel relative to the G channel and the second local offset of the B channel relative to the G channel are determined by performing normalized cross-correlation calculations on the color value of the pixel in the R channel and the color value of the pixel in the G channel, as well as normalized cross-correlation calculations on the color value of the pixel in the B channel and the color value of the pixel in the G channel. The first local offset and the second local offset determined in this way are also different. That is, for each frame of the image, when its content changes, the detected first local offset and the second local offset also change dynamically. Therefore, the first local offset and the second local offset can be used to remove purple fringing in a targeted manner for different frames of the image, rather than using a fixed offset to compensate for the pixels in the traditional way, thus improving the accuracy of purple fringing removal.

[0209] Optionally, the processor 510 is further configured to perform edge detection on the second channel image data to obtain an edge intensity set of the G channel for each pixel; the edge intensity set includes the edge intensity value of the G channel for each pixel; and determine the pixel to be corrected based on the edge intensity set of the G channel.

[0210] In this way, the pixels to be corrected are determined by the edge intensity set of the G channel, rather than by using a fixed threshold purple fringing detection method. Since the G channel image data is different for each frame of the image, the determined pixels to be corrected are also different. That is, for each frame of the image, the detected edge intensity of the G channel is different as the content changes, and therefore the detected pixels to be corrected are also dynamically changing. In other words, for each frame of the image, the area for purple fringing repair is also dynamically changing. This allows for targeted determination of the pixels to be corrected for each frame, improving the accuracy of pixel determination.

[0211] Optionally, the processor 510 is further configured to calculate the average and standard deviation of all edge intensity values ​​in the edge intensity set of the G channel to obtain the mean edge intensity and the standard deviation edge intensity; determine the maximum edge intensity of the pixel to be corrected based on the mean edge intensity and the standard deviation edge intensity; and determine the pixel corresponding to the edge intensity value in the edge intensity set of the G channel that is greater than the maximum edge intensity value as the pixel to be corrected.

[0212] Thus, by averaging and calculating the standard deviation of all edge intensity values ​​in the edge intensity set of the G channel, the mean and standard deviation of edge intensity are obtained. Then, based on the mean and standard deviation of edge intensity, the maximum edge intensity of the pixel to be corrected is determined. Pixels with edge intensities greater than the maximum edge intensity in the edge intensity set of the G channel are then identified as pixels to be corrected. Since the edge intensity set corresponding to the G channel changes with the content of each frame, the maximum edge intensity of each frame changes dynamically, and thus the pixels to be corrected also change dynamically. This allows for targeted purple fringing removal for each frame, rather than the traditional method of removing purple fringing from fixed pixels, improving the accuracy of purple fringing removal.

[0213] Optionally, the processor 510 is further configured to correct the color value of the pixel to be corrected in the R channel according to the first local offset to obtain a first corrected color value; correct the color value of the pixel to be corrected in the B channel according to the second local offset to obtain a second corrected color value; and merge the first corrected color value, the second corrected color value and the second channel image data to obtain second RGB image data.

[0214] Thus, since the first local offset, the second local offset, and the pixel to be corrected are different for each frame of the image, the color value of the pixel to be corrected in the R channel can be corrected specifically based on the first local offset of the R channel relative to the G channel, and the color value of the pixel to be corrected in the B channel can be corrected based on the second local offset of the B channel relative to the G channel, instead of using a fixed offset to compensate for a fixed pixel for the entire first RGB image data, thereby improving the accuracy of purple fringing removal.

[0215] Optionally, the processor 510 is further configured to: determine the minimum edge intensity of the pixel to be corrected based on the mean edge intensity; determine a smoothing weight factor for the pixel to be corrected based on the edge intensity set, the minimum edge intensity, and the maximum edge intensity; determine a first color value of the pixel to be corrected in the R channel based on the smoothing weight factor, the color value of the pixel to be corrected in the first channel image data, and the first corrected color value of the pixel to be corrected; determine a second color value of the pixel to be corrected in the B channel based on the smoothing weight factor, the color value of the pixel to be corrected in the third channel image data, and the second corrected color value of the pixel to be corrected; and merge the first color value, the second color value, and the second channel image data of the pixel to be corrected to obtain second RGB image data.

[0216] Thus, by calculating the smoothing weight factor of the pixel to be corrected, the first and second corrected color values ​​are smoothly transitioned according to the smoothing weight factor. This makes the color values ​​of the pixel to be corrected in the R and B channels of the second RGB image data transition naturally with the color values ​​of the surrounding pixels. As a result, the color transition of each pixel in the image after removing purple fringing is natural, thus improving the quality of the image after removing purple fringing.

[0217] It should be understood that, in this embodiment, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The GPU 5041 processes image data of still images or videos obtained by an image capture device (such as a color camera) in video capture mode or image capture mode. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 507 includes at least one of a touch panel 5071 and other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0218] The memory 509 can be used to store software programs and various data. The memory 509 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 509 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 509 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0219] Processor 510 may include one or more processing units; optionally, processor 510 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 510.

[0220] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image purple fringing removal method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0221] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0222] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image purple fringing removal method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0223] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0224] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image purple fringing removal method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0225] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0226] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0227] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for removing purple fringing from an image, characterized in that, The method includes: A normalized cross-correlation calculation is performed on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain a first local offset. A normalized cross-correlation calculation is also performed on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain a second local offset. Based on the first local offset, the color value of the R channel of the pixel to be corrected is corrected, and based on the second local offset, the color value of the B channel of the pixel to be corrected is corrected to obtain the second RGB image data.

2. The method according to claim 1, characterized in that, The process of performing normalized cross-correlation calculations on the first channel image data (R channel) and the second channel image data (G channel) of the first RGB image data to obtain a first local offset, and performing normalized cross-correlation calculations on the third channel image data (B channel) and the third channel image data of the first RGB image data to obtain a second local offset, includes: The color value of the R channel of each pixel is cross-correlated with the color value of the G channel of each pixel in a normalized manner to obtain the first normalized cross-correlation value of each pixel. The color value of the B channel of each pixel is cross-correlated with the color value of the G channel of each pixel in a normalized manner to obtain the second normalized cross-correlation value of each pixel. The R-channel color value of the pixel corresponding to the maximum value among the first normalized cross-correlation values ​​of all pixels is determined as the first local offset. The B-channel color value of the pixel corresponding to the maximum value among the second normalized cross-correlation values ​​of all pixels is determined as the second local offset.

3. The method according to claim 1, characterized in that, Before correcting the color value of the R channel of the pixel to be corrected according to the first local offset, and correcting the color value of the B channel of the pixel to be corrected according to the second local offset to obtain the second RGB image data, the method further includes: Edge detection is performed on the second channel image data to obtain the edge intensity set of the G channel for each pixel; the edge intensity set includes the edge intensity value of the G channel for each pixel. The pixel to be corrected is determined based on the edge intensity set of the G channel.

4. The method according to claim 3, characterized in that, The step of determining the pixel to be corrected based on the edge intensity set of the G channel includes: The mean and standard deviation of all edge intensity values ​​in the edge intensity set of the G channel are calculated to obtain the mean edge intensity and the standard deviation of edge intensity. The maximum edge intensity of the pixel to be corrected is determined based on the mean edge intensity and the standard deviation of edge intensity. The pixels with edge intensity values ​​greater than the maximum edge intensity value in the edge intensity set of the G channel are identified as the pixels to be corrected.

5. The method according to claim 4, characterized in that, The step of correcting the color value of the R channel of the pixel to be corrected according to the first local offset, and correcting the color value of the B channel of the pixel to be corrected according to the second local offset, to obtain the second RGB image data, includes: Based on the first local offset, the color value of the pixel to be corrected in the R channel is corrected to obtain the first corrected color value; Based on the second local offset, the color value of the pixel to be corrected in the B channel is corrected to obtain the second corrected color value; The first corrected color value, the second corrected color value, and the second channel image data are merged to obtain the second RGB image data.

6. The method according to claim 5, characterized in that, After calculating the average and standard deviation of all edge intensity values ​​in the edge intensity set of the G channel to obtain the mean and standard deviation of edge intensity, the method further includes: Based on the average edge intensity, determine the minimum edge intensity of the pixel to be corrected; The step of merging the first corrected color value, the second corrected color value, and the second channel image data to obtain the second RGB image data includes: The smoothing weight factor of the pixel to be corrected is determined based on the set of edge intensities, the minimum edge intensity, and the maximum edge intensity. Based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the first channel image data, and the first corrected color value of the pixel to be corrected, the first color value of the pixel to be corrected in the R channel is determined. Based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the third channel image data, and the second corrected color value of the pixel to be corrected, the second color value of the pixel to be corrected in the B channel is determined. The first color value, the second color value, and the second channel image data of the pixel to be corrected are merged to obtain the second RGB image data.

7. An image purple fringing removal device, characterized in that, The device includes: The processing module is used to perform normalized cross-correlation calculation on the first channel image data of the R channel and the second channel image data of the G channel in the first RGB image data to obtain a first local offset, and to perform normalized cross-correlation calculation on the third channel image data of the B channel and the third channel image data in the first RGB image data to obtain a second local offset. The correction module is used to correct the color value of the R channel of the pixel to be corrected according to the first local offset, and to correct the color value of the B channel of the pixel to be corrected according to the second local offset, so as to obtain the second RGB image data.

8. The apparatus according to claim 7, characterized in that, The processing module is specifically used for: The color value of the R channel of each pixel is cross-correlated with the color value of the G channel of each pixel in a normalized manner to obtain the first normalized cross-correlation value of each pixel. The color value of the B channel of each pixel is cross-correlated with the color value of the G channel of each pixel in a normalized manner to obtain the second normalized cross-correlation value of each pixel. The R-channel color value of the pixel corresponding to the maximum value among the first normalized cross-correlation values ​​of all pixels is determined as the first local offset. The B-channel color value of the pixel corresponding to the maximum value among the second normalized cross-correlation values ​​of all pixels is determined as the second local offset.

9. The apparatus according to claim 7, characterized in that, The device further includes: An edge detection module is used to perform edge detection on the second channel image data before correcting the color value of the R channel of the pixel to be corrected according to the first local offset and correcting the color value of the B channel of the pixel to be corrected according to the second local offset to obtain the second RGB image data, thereby obtaining an edge intensity set of the G channel for each pixel; the edge intensity set includes the edge intensity value of the G channel for each pixel. The processing module is further configured to determine the pixel to be corrected based on the edge intensity set of the G channel.

10. The apparatus according to claim 9, characterized in that, The processing module is specifically used for: The mean and standard deviation of all edge intensity values ​​in the edge intensity set of the G channel are calculated to obtain the mean edge intensity and the standard deviation of edge intensity. The maximum edge intensity of the pixel to be corrected is determined based on the mean edge intensity and the standard deviation of edge intensity. The pixels with edge intensity values ​​greater than the maximum edge intensity value in the edge intensity set of the G channel are identified as the pixels to be corrected.

11. The apparatus according to claim 10, characterized in that, The correction module is specifically used for: Based on the first local offset, the color value of the pixel to be corrected in the R channel is corrected to obtain the first corrected color value; Based on the second local offset, the color value of the pixel to be corrected in the B channel is corrected to obtain the second corrected color value; The first corrected color value, the second corrected color value, and the second channel image data are merged to obtain the second RGB image data.

12. The apparatus according to claim 11, characterized in that, The processing module is further configured to calculate the average and standard deviation of the edge intensity values ​​in the edge intensity set to obtain the mean edge intensity and the standard deviation of the edge intensity, and then determine the minimum edge intensity value of the pixel to be corrected based on the mean edge intensity. The correction module is specifically used for: The smoothing weight factor of the pixel to be corrected is determined based on the set of edge intensities, the minimum edge intensity, and the maximum edge intensity. Based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the first channel image data, and the first corrected color value of the pixel to be corrected, the first color value of the pixel to be corrected in the R channel is determined. Based on the smoothing weight factor of the pixel to be corrected, the color value of the pixel to be corrected in the third channel image data, and the second corrected color value of the pixel to be corrected, the second color value of the pixel to be corrected in the B channel is determined. The first color value, the second color value, and the second channel image data of the pixel to be corrected are merged to obtain the second RGB image data.

13. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the image purple fringing removal method as described in any one of claims 1-6.

Images