Image processing method and device, medium and chip

Abstract

The invention relates to an image processing method and device, a medium and a chip, and the method comprises the steps: obtaining a to-be-processed image and corresponding first position information, and enabling the first position information to be in the process of generating the to-be-processed image through the preset image signal processing of an original image collected by an image sensor; performing highlight detection on the first intermediate image in the original domain to obtain position information corresponding to a highlight area; performing purple point detection on the to-be-processed image to obtain second position information corresponding to a purple point in the to-be-processed image; determining a purple boundary area in the to-be-processed image according to the first position information and the second position information; and performing purple edge elimination processing on the purple edge area in the to-be-processed image to obtain an image after purple edge elimination. By adopting the method, the purple edge elimination effect in the image can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to an image processing method, apparatus, medium, and chip. Background Art

[0002] With the continuous development of modern technology, obtaining high-quality images has become the focus of industry competition. Although camera lenses, photosensitive devices, and hardware structures are constantly improving, due to the current limitations of hardware, optics, etc., purple fringing may appear in the images captured by the imaging system in some scenarios. Therefore, how to better eliminate purple fringing in images has become an important part of image post-processing.

[0003] However, in the related art, the method for eliminating purple fringing in images has the problem of poor purple fringing elimination effect. Summary of the Invention

[0004] To overcome the problems in the related art, the present disclosure provides an image processing method, apparatus, medium, and chip.

[0005] According to the first aspect of the embodiments of the present disclosure, an image processing method is provided, including:

[0006] Obtain a to-be-processed image and corresponding first position information, where the first position information is the position information corresponding to the highlight area obtained by performing highlight detection on a first intermediate image in the original domain during the process of generating the to-be-processed image from the original image collected by an image sensor through preset image signal processing;

[0007] Perform purple dot detection on the to-be-processed image to obtain second position information corresponding to the purple dots in the to-be-processed image;

[0008] Determine the purple fringing area in the to-be-processed image according to the first position information and the second position information;

[0009] Perform purple fringing elimination processing on the purple fringing area in the to-be-processed image to obtain an image with purple fringing eliminated.

[0010] Optionally, the obtaining of the to-be-processed image includes:

[0011] Perform first image signal processing on the first intermediate image to obtain a second intermediate image in the luminance space;

[0012] Perform purple dot detection on the second intermediate image to obtain third position information corresponding to the purple dots in the second intermediate image;

[0013] Determine the purple fringing area and non-purple fringing area in the second intermediate image according to the third position information and the first position information;

[0014] Sharpen the non-purple-edge area in the second intermediate image to obtain a third intermediate image;

[0015] Perform second image signal processing on the third intermediate image to obtain the image to be processed.

[0016] Optionally, the method further includes:

[0017] Obtain the target exposure time corresponding to the shooting of the original image;

[0018] Determine the target brightness threshold corresponding to the target exposure time according to the correlation between the exposure time and the brightness threshold, where the exposure time is positively correlated with the brightness threshold;

[0019] Determine the highlight pixel points in the first intermediate image whose corresponding pixel brightness is greater than the target brightness threshold;

[0020] Determine the first position information based on the highlight pixel points.

[0021] Optionally, the method further includes:

[0022] Unify the color channels of each pixel point in the first intermediate image to a preset channel to obtain the information of the first intermediate image corresponding to the preset channel;

[0023] Determine the pixel brightness of each pixel in the first intermediate image based on the information of the first intermediate image corresponding to the preset channel.

[0024] Optionally, the determining the purple-edge area in the image to be processed according to the first position information and the second position information includes:

[0025] For any position point in the second position information, when there is a position point in the first position information within a preset range around this position point, determine this position point as a purple-edge pixel point;

[0026] Determine the purple-edge area in the image to be processed based on the positions corresponding to each purple-edge pixel point in the image to be processed.

[0027] Optionally, the original image undergoes third image signal processing to obtain the first intermediate image, and the third image signal processing includes lens shading correction processing. The method further includes:

[0028] Determine the image output after lens shading correction processing as the first intermediate image.

[0029] Optionally, the original image undergoes a third image signal processing to obtain the first intermediate image. The third image signal processing includes digital gain processing. The method further includes:

[0030] Determine a candidate image for calculating a high - light region based on a gain coefficient corresponding to the digital gain processing and the first intermediate image;

[0031] Perform high - light detection on the candidate image to obtain position information corresponding to the high - light region in the candidate image;

[0032] Determine the position information corresponding to the high - light region in the candidate image as the first position information.

[0033] According to a second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, including:

[0034] A first acquisition module configured to acquire an image to be processed and corresponding first position information. The first position information is the position information corresponding to a high - light region obtained by performing high - light detection on a first intermediate image in the original domain during the process of generating the image to be processed from an original image collected by an image sensor through a preset image signal processing;

[0035] A purple dot detection module configured to perform purple dot detection on the image to be processed to obtain second position information corresponding to purple dots in the image to be processed;

[0036] A purple edge region detection module configured to determine a purple edge region in the image to be processed according to the first position information and the second position information;

[0037] A purple edge elimination module configured to perform purple edge elimination processing on the purple edge region in the image to be processed to obtain an image with purple edges eliminated.

[0038] According to a third aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, including:

[0039] A processor;

[0040] A memory for storing instructions executable by the processor;

[0041] Wherein, the processor is configured to: execute the steps of the image processing method provided in the first aspect of the present disclosure.

[0042] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer - readable storage medium, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the image processing method provided in the first aspect of the present disclosure are implemented.

[0043] According to a fifth aspect of the embodiments of the present disclosure, there is provided a chip, including a processor and an interface; the processor is configured to read instructions to execute the steps of the image processing method provided in the first aspect of the present disclosure.

[0044] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0045] In the above technical solution, after obtaining the image to be processed and the corresponding first position information, the purple dot detection can be performed on the image to be processed to obtain the second position information corresponding to the purple dots in the image to be processed. Then, the purple edge area in the image to be processed can be determined according to the first position information and the second position information. Next, the purple edge elimination process can be performed on the purple edge area in the image to be processed to obtain the image after purple edge elimination. Since the first position information corresponding to the highlight area is obtained by performing highlight detection on the first intermediate image in the original domain, and the original domain can directly reflect the photosensitive characteristics of the image, avoiding the influence generated after being processed by other image signal processing processes in other domains, so as to distinguish the true highlight and the pseudo highlight, improve the accuracy of highlight detection, and further improve the accuracy of purple edge detection, and finally improve the purple edge elimination effect in the image.

[0046] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0048] Figure 1 is a flowchart of an image processing method shown according to an exemplary embodiment.

[0049] Figure 2 is a schematic diagram of a preset brightness range in a luminance space shown according to an exemplary embodiment.

[0050] Figure 3 is a schematic diagram of the correlation between exposure time and luminance threshold shown according to an exemplary embodiment.

[0051] Figure 4 is a schematic diagram of a preset range around a purple dot shown according to an exemplary embodiment.

[0052] Figure 5 is a schematic diagram of pixel points of an image in the original domain shown according to an exemplary embodiment.

[0053] Figure 6It is a flowchart of image signal processing shown according to an exemplary embodiment.

[0054] Figure 7 It is a block diagram of an image processing apparatus shown according to an exemplary embodiment.

[0055] Figure 8 It is a block diagram of an image processing apparatus shown according to an exemplary embodiment. Detailed implementation manners

[0056] Here, the exemplary embodiments will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0057] It should be noted that all actions of obtaining signals, information, or data in the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where the location is located and obtaining authorization from the corresponding device owner.

[0058] In the related art, a purple fringing elimination module is added during the image signal processing (Image Signal Processor, ISP) process to eliminate purple fringing in the image. Specifically, in the purple fringing elimination module, purple dot detection and highlight detection are performed on the image in the YUV domain. The purple fringing area in the image is judged based on the purple dot detection result and the highlight detection result, and then the purple fringing area is corrected to achieve the function of eliminating purple fringing in the image.

[0059] However, the applicant has found that there are deficiencies in the highlight detection mechanism in the purple fringing elimination module, which cannot distinguish true highlights from false highlights, resulting in inaccurate detection of the purple fringing area and affecting the subsequent purple fringing elimination effect.

[0060] In view of the above discovery, the image processing method, apparatus, medium, and chip of the embodiments of the present disclosure are proposed to improve the purple fringing elimination effect in the image.

[0061] Figure 1 It is a flowchart of an image processing method shown according to an exemplary embodiment. This method can be applied to an electronic device, such as Figure 1 As shown, this method may include:

[0062] In step S110, a to-be-processed image and corresponding first position information are obtained. The first position information is the position information corresponding to the highlight area obtained by performing highlight detection on a first intermediate image in the original domain during the process of generating the to-be-processed image by subjecting the original image collected by an image sensor to preset image signal processing.

[0063] In the embodiments of the present disclosure, the original image collected by an image sensor can generate a to-be-processed image after being subjected to preset image signal processing.

[0064] In some embodiments, after an electronic device collects an original image (RAW image) through its own image sensor, it can continue to perform preset image signal processing on the original image to generate a to-be-processed image, and during the process of performing preset image signal processing on the original image, the position information corresponding to the highlight area obtained by performing highlight detection on a first intermediate image in the original domain (RAW domain), that is, the first position information, can be obtained.

[0065] In some embodiments, the electronic device that executes steps S110 - S140 can be different from the electronic device that executes the preset image signal processing process. In this case, the electronic device can receive the to-be-processed image generated by other electronic devices and the corresponding first position information from the outside.

[0066] In step S120, purple dot detection is performed on the to-be-processed image to obtain second position information corresponding to the purple dots in the to-be-processed image.

[0067] In step S130, according to the first position information and the second position information, a purple edge area in the to-be-processed image is determined.

[0068] In step S140, purple edge elimination processing is performed on the purple edge area in the to-be-processed image to obtain an image with purple edges eliminated.

[0069] By using the above method, after obtaining the to-be-processed image and the corresponding first position information, purple dot detection can be performed on the to-be-processed image to obtain second position information corresponding to the purple dots in the to-be-processed image. Then, the purple edge area in the to-be-processed image can be determined according to the first position information and the second position information. Next, purple edge elimination processing can be performed on the purple edge area in the to-be-processed image to obtain an image with purple edges eliminated. Since the first position information is the position information corresponding to the highlight area obtained by performing highlight detection on the first intermediate image in the original domain, and the original domain can directly reflect the photosensitive characteristics of the image, avoiding the influence generated after being processed by other image signal processing processes in other domains, so as to distinguish the true highlights and pseudo-highlights, improving the accuracy of highlight detection, and further improving the accuracy of purple edge detection, and finally improving the purple edge elimination effect in the image.

[0070] In some embodiments, the preset image signal processing may include dead pixel correction, black level correction, digital gain, lens shading correction, white balance correction, demosaicing, sharpening, color correction, gamma correction, color space conversion, noise reduction, tone mapping, color processing, and sharpening. After the sharpening process following the color processing, the image to be processed can be obtained.

[0071] It should be noted that the above preset image signal processing flow is only an example. According to actual needs, the preset image signal processing flow may include more or fewer processes.

[0072] In some embodiments, after obtaining the image with purple fringing removed, image signal processing processes such as image encoding can be continued to obtain images in different formats. For example, a JPEG format image can be obtained.

[0073] In some embodiments, the original image undergoes third image signal processing to obtain a first intermediate image. The third image signal processing includes lens shading correction processing. In this case, in order to obtain the first intermediate image, the embodiments of the present disclosure further include the following steps:

[0074] Determine the image output after the lens shading correction processing as the first intermediate image.

[0075] In the embodiments of the present disclosure, the original image undergoes third image signal processing to obtain a first intermediate image. If the third image signal processing includes lens shading correction processing, the image obtained after the lens shading correction processing can be determined as the first intermediate image. By determining the image output after the lens shading correction processing as the first intermediate image, the brightness consistency of the center and four corners of the image can be ensured, avoiding the darkening of the four corners of the image and improving the accuracy of highlight detection.

[0076] In some embodiments, the original image undergoes third image signal processing to obtain a first intermediate image. The third image signal processing includes digital gain processing. In this case, in order to improve the accuracy of the determined first position information, the method of the embodiments of the present disclosure further includes the following steps:

[0077] Based on the gain coefficient corresponding to the digital gain processing and the first intermediate image, determine a candidate image for calculating the highlight area;

[0078] Perform highlight detection on the candidate image to obtain the position information corresponding to the highlight area in the candidate image;

[0079] Determine the position information corresponding to the highlight area in the candidate image as the first position information.

[0080] In the embodiments of the present disclosure, the original image undergoes third image signal processing to obtain a first intermediate image. If the third image signal processing includes digital gain processing, when performing highlight detection on the first intermediate image, the influence brought by the digital gain processing can be considered. Thus, first, based on the gain coefficient corresponding to the digital gain processing and the first intermediate image, a candidate image for calculating the highlight area can be determined, and then highlight detection is performed on the candidate image to obtain the position information corresponding to the highlight area in the candidate image, and the position information corresponding to the highlight area in the candidate image is determined as the first position information.

[0081] In some embodiments, determining a candidate image for calculating the highlight area based on the gain coefficient corresponding to the digital gain processing and the first intermediate image may be dividing the first intermediate image by the data gain coefficient to obtain the candidate image.

[0082] In the embodiments of the present disclosure, considering that different digital gain magnitudes will have different effects on the brightness of the image in the original domain, therefore, in order to exclude the brightness interference caused by the digital gain processing when performing highlight detection on the first intermediate image, after the digital gain processing, the corresponding digital gain can be eliminated through the digital gain coefficient, so as to obtain accurate first position information.

[0083] In some embodiments, in addition to digital gain processing and / or lens shading correction processing, the third image signal processing may further include one or more processing flows among processing flows such as dead pixel correction and black level correction.

[0084] In some embodiments, the points in the purple fringing area can be called purple fringing pixel points. Performing purple fringing elimination processing on the purple fringing area in the image to be processed may be correcting the purple fringing pixel points in the purple fringing area using a purple fringing correction algorithm. Optionally, the filter of the correction algorithm may be a bilateral filter, a mean filter, etc.

[0085] In some embodiments, purple fringing correction performs filtering processing on the purple fringing pixel points in the Cb / Cr domain, while the value in the Y domain remains unchanged, ensuring that the brightness of the corrected purple fringing pixel points does not change.

[0086] In some embodiments, let the corrected purple fringing pixel point be I_correct, the original purple fringing pixel point be I_origin, and the finally corrected output image be I_out:

[0087] Then, I_out = weight * I_correct + (1 - weight) * I_origin;

[0088] Among them, weight represents a preset weight, and the range is [0 - 1]. The greater the weight, the stronger the ability to remove purple fringing.

[0089] In addition, the applicant also found that in some image signal processing processes, before the purple fringing removal process, image signal processing processes such as tone mapping and color processing can also be performed. In order to improve the tone mapping and color processing effects, a sharpening process is added before the tone mapping and color processing. However, if the edges of the purple fringing are sharpened and enhanced before the tone mapping and color processing, the purple fringing will become more serious and it will be difficult for the purple fringing removal module to remove. Based on this discovery, in some embodiments, in the method of the present disclosure embodiment, obtaining the image to be processed may include the following steps:

[0090] Perform first image signal processing on the first intermediate image to obtain a second intermediate image in the luminance space;

[0091] Perform purple dot detection on the second intermediate image to obtain the third position information corresponding to the purple dots in the second intermediate image;

[0092] According to the third position information and the first position information, determine the purple fringing region and the non-purple fringing region in the second intermediate image;

[0093] Perform sharpening processing on the non-purple fringing region in the second intermediate image to obtain a third intermediate image;

[0094] Perform second image signal processing on the third intermediate image to obtain the image to be processed.

[0095] In the embodiment of the present disclosure, the second intermediate image can be understood as the image to be sharpened. After obtaining the second intermediate image, purple dot detection can be performed on the second intermediate image to obtain the third position information corresponding to the purple dots in the second intermediate image. Then, according to the third position information and the first position information, the purple fringing region and the non-purple fringing region in the second intermediate image can be determined. Next, sharpening processing can be performed on the non-purple fringing region in the second intermediate image to obtain a third intermediate image. Then, second image signal processing can be further performed on the third intermediate image to obtain the image to be processed.

[0096] By adopting the above method, by determining the purple fringing region and the non-purple fringing region and only performing sharpening increase on the non-purple fringing region, it is possible to avoid sharpening and enhancing the purple fringing, reduce the difficulty of subsequent purple fringing elimination processing, and thus improve the purple fringing elimination effect.

[0097] In some embodiments, the second intermediate image and the image to be processed are in the luminance space. When performing purple dot detection on the image, it can be determined whether the luminance of any pixel point in the image is within a preset luminance range. If it is within the preset luminance range, then it is further determined whether the chrominance of the pixel point is within a preset chrominance range. If it is within the preset chrominance range, then the pixel point is determined to be a purple dot.

[0098] Optionally, the luminance space may be, for example, a YUV space, a YCbCr space, etc.

[0099] Exemplarily, referring to Figure 2 as shown, assuming that the luminance space is a YCbCr space, setting a luminance range threshold Y_min and Y_max, and setting a preset chrominance range as Figure 2 the box range in, then when the luminance value Y of any pixel point in the image satisfies: Y_min <= Y <= Y_max, and when Cb / Cr is within the selected box area range, then this pixel point is determined as a purple point.

[0100] In the embodiments of the present disclosure, the applicant found the characteristics of purple edge pixel points, that is, there is a high - light area near the purple edge pixel points, and the pixel points themselves are purple points. Based on this characteristic, a purple edge detection method is proposed. Therefore, in some embodiments, determining the purple edge area in the image to be processed according to the first position information and the second position information may include the following steps:

[0101] For any position point in the second position information, when there is a position point in the first position information within a preset range around this position point, determine this position point as a purple edge pixel point;

[0102] Based on the positions corresponding to each purple edge pixel point in the image to be processed, determine the purple edge area in the image to be processed.

[0103] In some embodiments, for any position point in the second position information, when there is no position point in the first position information within a preset range around this position point, determine this position point as a non - purple edge pixel point; based on the positions corresponding to each non - purple edge pixel point in the image to be processed, determine the non - purple edge area in the image to be processed.

[0104] In some embodiments, after determining the purple edge area in the image to be processed, the remaining position area in the image to be processed except for the purple edge area can be determined as the non - purple edge area.

[0105] In the embodiments of the present disclosure, the position information corresponding to the high - light points can be determined according to the first position information, and the position information corresponding to the purple points can be determined according to the second position information.

[0106] In some embodiments, the preset range around the position point can be a range size of 3*3, 4*4, N*N around the position point. Among them, the larger N is, the stronger the purple edge detection ability is, and the better the purple edge removal effect is.

[0107] Exemplarily, please refer to Figure 4, taking the preset range around the position point as the 3*3 range around the position point as an example, for any purple point in the second position information, if there is a highlight point within the 3*3 range around the purple point, it can be determined that the purple point is a purple edge pixel point.

[0108] In the embodiments of the present disclosure, the method for determining the purple edge area in the second intermediate image according to the third position information and the first position information can refer to the above embodiments and will not be elaborated here. Among them, after determining the purple edge area, the remaining area in the second intermediate image can be determined as the non-purple edge area.

[0109] In some embodiments, after detecting the edge information of the non-purple edge area in the second intermediate image through methods such as canny edge detection and Laplace detection, the edge information and the non-purple edge area in the second intermediate image can be superimposed to obtain the sharpened third intermediate image.

[0110] In some embodiments, the first image signal processing may include one or more of the processing flows such as white balance correction and demosaicing.

[0111] In some embodiments, the second image signal processing may include one or more of the processing flows such as color correction, gamma correction, color space conversion, noise reduction, tone mapping, color processing, and sharpening.

[0112] In some embodiments, in order to obtain the position information corresponding to the highlight area detected for the first intermediate image, the method of the embodiments of the present disclosure may further include the following steps:

[0113] Determine the pixel points in the first intermediate image whose corresponding pixel brightness is greater than the target brightness threshold as highlight pixel points;

[0114] Based on the highlight pixel points, determine the first position information.

[0115] In the embodiments of the present disclosure, the pixel brightness of each pixel in the first intermediate image can be obtained first, and then, the pixel points in the first intermediate image whose corresponding pixel brightness is greater than the preset brightness threshold can be determined as highlight pixel points, and thus, based on the highlight pixel points, the first position information can be determined.

[0116] Combined with the foregoing content, it can be seen that in some embodiments, the first intermediate image is an image processed by digital gain. In this case, the influence brought by digital gain can be considered. Therefore, when determining the first position information, the pixel points in the candidate image whose corresponding pixel brightness is greater than the preset brightness threshold can be determined as highlight pixel points, and further, based on the highlight pixel points, the first position information can be determined.

[0117] In addition, considering that in the processing of the image automatic exposure module, the exposure time may be shortened due to too bright ambient light or too many high-light areas, so that most areas will not show overexposure. Therefore, in some embodiments, for the accuracy of the first position information detection, the method of the embodiments of the present disclosure may further include the following steps:

[0118] Obtain the target exposure time corresponding to the original image when it is taken;

[0119] According to the correlation between the exposure time and the brightness threshold, determine the target brightness threshold corresponding to the target exposure time, and the exposure time is positively correlated with the brightness threshold;

[0120] Determine the high-light pixel points in the first intermediate image whose corresponding pixel brightness is greater than the target brightness threshold;

[0121] Based on the high-light pixel points, determine the first position information.

[0122] In the embodiments of the present disclosure, at different exposure times, different brightness thresholds can be preset, that is, the correlation between the preset exposure time and the brightness threshold. Thus, the target exposure time corresponding to the original image when it is taken can be obtained first, and then, according to the correlation between the preset exposure time and the brightness threshold, the target brightness threshold corresponding to the target exposure time can be determined. Then, the pixel points in the first intermediate image whose corresponding pixel brightness is greater than the target brightness threshold can be determined as high-light pixel points, and finally, the first position information can be determined based on the high-light pixel points.

[0123] In the embodiments of the present disclosure, the exposure time is positively correlated with the brightness threshold. Exemplarily, the correlation between the exposure time and the brightness threshold can be as Figure 3 shown. It can be seen that as the exposure time increases, the target brightness threshold also increases.

[0124] Exemplarily, in an application scenario, in sunny days when the sun is relatively strong, in order to avoid large-area overexposure, the exposure time will be shortened, and the target brightness threshold can be set relatively low, so that the high-light areas near the object can be detected better. In an indoor environment, the exposure time will be relatively long, and at this time, the target brightness threshold can be appropriately increased to avoid non-high-light areas being recognized.

[0125] Similarly, in some embodiments, when the first intermediate image is an image processed by digital gain, when determining the first position information, the pixel points in the candidate image whose corresponding pixel brightness is greater than the target brightness threshold can be determined as high-light pixel points, and further, the first position information can be determined based on the high-light pixel points.

[0126] In some embodiments, in order to obtain the pixel brightness of each pixel in the first intermediate image, the method of the embodiments of the present disclosure may further include the following steps:

[0127] Unify the color channels of each pixel point in the first intermediate image to a preset channel to obtain the information of the first intermediate image corresponding to the preset channel;

[0128] Based on the information of the first intermediate image corresponding to the preset channel, determine the pixel brightness of each pixel in the first intermediate image.

[0129] In the embodiments of the present disclosure, the first intermediate image is in the original domain. Among them, the image in the original domain may refer to an image in Bayer format, and the original domain image may also be referred to as a Bayer format raw image. An image in Bayer format refers to an image that only includes red (R channel), green (G channel), and blue (B channel).

[0130] Exemplarily, as Figure 5 shown, a schematic diagram of a pixel point of an image in the original domain is shown. Each pixel in this image includes two G channels, one R channel, and one B channel.

[0131] In the embodiments of the present disclosure, in order to obtain the pixel brightness of each pixel in the first intermediate image, the color channels of each pixel point in the first intermediate image can be unified to a preset channel. For example, the four-channel information included in each pixel can be unified to any one of the R channel, G channel, and B channel, so as to obtain the information of the first intermediate image corresponding to the preset channel. Then, based on the information of the first intermediate image corresponding to the preset channel, the pixel brightness of each pixel in the first intermediate image can be determined.

[0132] In some embodiments, the average value of the 4 values corresponding to a pixel in the preset channel can be taken to obtain the brightness of the pixel.

[0133] In some embodiments, considering the pixel characteristics of the image in the original domain, that is, each pixel point includes two G channels, and the human eye is more sensitive to the G channel. Therefore, preferably, the color channels of each pixel point in the first intermediate image can be unified to the G channel, thereby improving the accuracy of brightness calculation.

[0134] In some embodiments, interpolation methods such as neighborhood interpolation method and bilinear interpolation method can be used to unify the color channels of each pixel point in the first intermediate image to a preset channel.

[0135] The following combines Figure 5 , and takes bilinear interpolation as an example for illustration, where:

[0136] After unifying B8 to the G channel, G8 = (G3 + G7 + G9 + G13) / 4

[0137] After unifying R12 to the G channel, G12 = (G7 + G11 + G13 + G17) / 4

[0138] After the above interpolation, the R and B channels can be unified to the G channel.

[0139] Similarly, in some embodiments, when the first intermediate image is an image processed by digital gain, when determining the pixel brightness of each pixel in the first intermediate image, the color channels of each pixel point in the candidate image can be unified to a preset channel to obtain the information of the candidate image corresponding to the preset channel, and then based on the information of the candidate image corresponding to the preset channel, determine the pixel brightness of each pixel in the first intermediate image.

[0140] In some embodiments, please refer to Figure 6 , Figure 6 which is a flowchart of an image signal processing provided by an embodiment of the present disclosure. As Figure 6 shown, the image signal processing flow includes:

[0141] Perform third image signal processing on the original image collected by the image sensor. Among them, the third image signal processing includes performing dead pixel correction, black level correction, digital gain, and lens shading correction in sequence, and determining the image output by the lens shading correction as the first intermediate image;

[0142] On the one hand, perform highlight detection on the first intermediate image to obtain a highlight detection result, that is, the first position information;

[0143] On the other hand, continue to perform first image signal processing on the first intermediate image. Among them, the first image signal processing includes performing white balance correction and demosaicing in sequence, and determining the image output after demosaicing as the second intermediate image;

[0144] Perform purple dot detection on the second intermediate image to obtain the third position information, and determine the purple edge area and non-purple edge area in the second intermediate image according to the third position information and the first position information transmitted by the highlight detection;

[0145] Perform sharpening processing on the non-purple edge area in the second intermediate image to obtain the third intermediate image;

[0146] Perform second image signal processing on the third intermediate image. Among them, the second image signal processing includes performing color correction, gamma correction, color space conversion, noise reduction, tone mapping, color processing, and sharpening in sequence, and determining the image output after sharpening as the image to be processed;

[0147] Perform purple point detection on the image to be processed to obtain second position information, and determine the purple edge area in the image to be processed according to the first position information and the second position information transmitted by the highlight detection;

[0148] Performing purple-edge elimination processing on the purple-edge region in the image to be processed to obtain an image after the purple-edge is eliminated;

[0149] The image after the purple fringing is eliminated is encoded to obtain an image in JPEG format.

[0150] After the above image signal processing flow, highlight detection is performed in the original domain, making the highlight detection result more accurate and avoiding the detection of false highlight areas. In addition, purple fringing area detection is added before sharpening, and only the non-purple fringing area is subsequently sharpened to avoid the problem of purple fringing being aggravated by sharpening, thereby improving the ability to remove purple fringing.

[0151] It should be noted that Figure 6 For steps not described in detail in the image signal processing flow, reference may be made to the aforementioned embodiments and will not be described in detail here.

[0152] It should be noted that Figure 6 The image signal processing flow is only an example. According to actual needs, the image signal processing flow may include: Figure 6 The processing flow shown may be more or less than the flow, and in addition, according to actual needs, Figure 6 One or more of the processing flows shown may be executed in a reversed order.

[0153] The present disclosure also provides an image processing device, such as Figure 7 As shown, the image processing device 700 includes:

[0154] The first acquisition module 710 is configured to acquire the image to be processed and the corresponding first position information, wherein the first position information is position information corresponding to the highlight area obtained by performing highlight detection on the first intermediate image in the original domain during the process of generating the image to be processed by the original image acquired by the image sensor through a preset image signal processing;

[0155] A purple point detection module 720 is configured to perform purple point detection on the image to be processed to obtain second position information corresponding to the purple point in the image to be processed;

[0156] A purple-fringe region detection module 730 is configured to determine a purple-fringe region in the image to be processed according to the first position information and the second position information;

[0157] The purple fringing removal module 740 is configured to perform purple fringing removal processing on the purple fringing area in the image to be processed to obtain an image after the purple fringing is removed.

[0158] Optionally, the first acquisition module 710 includes:

[0159] A first processing sub-module, configured to perform first image signal processing on the first intermediate image to obtain a second intermediate image in the luminance space;

[0160] A purple dot detection sub-module, configured to detect purple dots in the second intermediate image to obtain third position information corresponding to the purple dots in the second intermediate image;

[0161] A purple edge area discrimination sub-module, configured to determine a purple edge area and a non-purple edge area in the second intermediate image according to the third position information and the first position information;

[0162] A sharpening sub-module, configured to perform sharpening processing on the non-purple edge area in the second intermediate image to obtain a third intermediate image;

[0163] A second processing sub-module, configured to perform second image signal processing on the third intermediate image to obtain the image to be processed.

[0164] Optionally, the image processing apparatus 700 further includes:

[0165] A second acquisition module, configured to acquire a target exposure time corresponding to when the original image is captured;

[0166] A target brightness threshold determination module, configured to determine a target brightness threshold corresponding to the target exposure time according to the correlation between the exposure time and the brightness threshold, where the exposure time is positively correlated with the brightness threshold;

[0167] A highlight pixel point determination module, configured to determine pixel points in the first intermediate image whose corresponding pixel brightness is greater than the target brightness threshold as highlight pixel points;

[0168] A first position information determination module, configured to determine the first position information based on the highlight pixel points.

[0169] Optionally, the image processing apparatus 700 further includes:

[0170] A color channel unification module, configured to unify the color channels of each pixel point in the first intermediate image to a preset channel to obtain information corresponding to the first intermediate image under the preset channel;

[0171] A pixel brightness determination module, configured to determine the pixel brightness of each pixel in the first intermediate image based on the information corresponding to the first intermediate image under the preset channel.

[0172] Optionally, the purple fringing area detection module 730 includes:

[0173] A purple fringing pixel point determination sub-module, configured to determine a position point as a purple fringing pixel point when there is a position point in the first position information within a preset range around the position point for any position point in the second position information;

[0174] A purple fringing area determination sub-module, configured to determine the purple fringing area in the to-be-processed image based on the positions corresponding to the respective purple fringing pixel points in the to-be-processed image.

[0175] Optionally, the original image undergoes a third image signal processing to obtain the first intermediate image, and the third image signal processing includes a lens shading correction process. The image processing apparatus 700 further includes:

[0176] A first intermediate image determination module, configured to determine the image output after the lens shading correction process as the first intermediate image.

[0177] Optionally, the original image undergoes a third image signal processing to obtain the first intermediate image, and the third image signal processing includes a digital gain process. The image processing apparatus 700 further includes:

[0178] A candidate image determination module, configured to determine a candidate image for calculating the highlight area based on the gain coefficient corresponding to the digital gain process and the first intermediate image;

[0179] A highlight detection module, configured to perform highlight detection on the candidate image to obtain the position information corresponding to the highlight area in the candidate image;

[0180] A second position information determination module, configured to determine the position information corresponding to the highlight area in the candidate image as the first position information.

[0181] Regarding the apparatus in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0182] The present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the image processing method provided by the present disclosure are implemented.

[0183] Figure 8 is a block diagram of an image processing apparatus 800 shown according to an exemplary embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a camera, a smart wearable device, etc. having an image acquisition function.

[0184] Refer to Figure 8, device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input / output interface 812, sensor component 814, and communication component 816.

[0185] Processing component 802 generally controls the overall operation of device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-described image processing method. In addition, processing component 802 may include one or more modules to facilitate the interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate the interaction between multimedia component 808 and processing component 802.

[0186] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of these data include instructions for any application or method operating on device 800, contact data, phone book data, messages, pictures, videos, etc. Memory 804 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0187] Power component 806 provides power to the various components of device 800. Power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for device 800.

[0188] Multimedia component 808 includes a screen that provides an output interface between device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front camera and / or a rear camera. When device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each of the front camera and the rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0189] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0190] The input / output interface 812 provides an interface between the processing component 802 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0191] The sensor component 814 includes one or more sensors for providing an assessment of the status of the device 800 in various aspects. For example, the sensor component 814 can detect the on / off state of the device 800, the relative positioning of components, such as the display and keypad of the device 800. The sensor component 814 can also detect a change in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and a change in the temperature of the device 800. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0192] The communication component 816 is configured to facilitate communication between the device 800 and other devices in a wired or wireless manner. The device 800 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0193] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above image processing method.

[0194] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 804 including instructions, is also provided. The above instructions may be executed by a processor 820 of the apparatus 800 to complete the above image processing method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0195] In addition to being an independent electronic device, the above apparatus may also be a part of an independent electronic device. For example, in one embodiment, the apparatus may be an integrated circuit (IC) or a chip. The integrated circuit may be a single IC or a collection of multiple ICs. The chip may include, but is not limited to, the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip), etc. The above integrated circuit or chip may be used to execute executable instructions (or code) to implement the above image processing method. The executable instructions may be stored in the integrated circuit or chip, or may be obtained from other devices or equipment. For example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instructions may be stored in the memory, and when the executable instructions are executed by the processor, the above image processing method is implemented; or, the integrated circuit or chip may receive the executable instructions through the interface and transmit them to the processor for execution to implement the above image processing method.

[0196] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, and the computer program has a code portion for performing the above-described image processing method when executed by the programmable device.

[0197] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0198] It should be understood that the present disclosure is not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image processing method, characterized in that, it includes: obtaining an image to be processed and corresponding first position information, where the first position information is the position information corresponding to the highlight area obtained by performing highlight detection on a first intermediate image in the original domain during the process of generating the image to be processed from the original image collected by an image sensor through preset image signal processing; performing purple dot detection on the image to be processed to obtain second position information corresponding to purple dots in the image to be processed; determining a purple edge area in the image to be processed according to the first position information and the second position information; performing purple edge elimination processing on the purple edge area in the image to be processed to obtain an image with purple edges eliminated.

2. The method according to claim 1, characterized in that, the obtaining of the image to be processed includes: performing first image signal processing on the first intermediate image to obtain a second intermediate image in the luminance space; performing purple dot detection on the second intermediate image to obtain third position information corresponding to purple dots in the second intermediate image; determining a purple edge area and a non-purple edge area in the second intermediate image according to the third position information and the first position information; performing sharpening processing on the non-purple edge area in the second intermediate image to obtain a third intermediate image; performing second image signal processing on the third intermediate image to obtain the image to be processed.

3. The method according to claim 1, characterized in that, the method further includes: obtaining a target exposure time corresponding to the shooting of the original image; determining a target luminance threshold corresponding to the target exposure time according to the correlation between the exposure time and the luminance threshold, where the exposure time and the luminance threshold are positively correlated; determining pixels with pixel luminance greater than the target luminance threshold in the first intermediate image as highlight pixels; determining the first position information based on the highlight pixels.

4. The method according to claim 3, characterized in that, the method further includes: unifying the color channels of each pixel point in the first intermediate image to a preset channel to obtain information of the first intermediate image corresponding to the preset channel; determining the pixel luminance of each pixel in the first intermediate image based on the information of the first intermediate image corresponding to the preset channel.

5. The method according to claim 1, characterized in that, the determining of the purple edge area in the image to be processed according to the first position information and the second position information includes: for any position point in the second position information, when there is a position point in the first position information within a preset range around this position point, determining this position point as a purple edge pixel; determining the purple edge area in the image to be processed based on the positions corresponding to each purple edge pixel in the image to be processed.

6. The method according to claim 1, characterized in that, the original image undergoes third image signal processing to obtain the first intermediate image, the third image signal processing includes lens shading correction processing, and the method further includes: Determine the image output after lens shadow correction processing as the first intermediate image.

7. The method according to claim 1, wherein, the original image undergoes a third image signal processing to obtain the first intermediate image, the third image signal processing includes digital gain processing, and the method further includes: determine a candidate image for calculating the highlight area based on the gain coefficient corresponding to the digital gain processing and the first intermediate image; perform highlight detection on the candidate image to obtain the position information corresponding to the highlight area in the candidate image; determine the position information corresponding to the highlight area in the candidate image as the first position information.

8. An image processing apparatus, wherein, comprising: a first acquisition module configured to acquire an image to be processed and corresponding first position information, where the first position information is the position information corresponding to the highlight area obtained by performing highlight detection on the first intermediate image in the original domain during the process of generating the image to be processed by subjecting the original image collected by an image sensor to a preset image signal processing; a purple dot detection module configured to perform purple dot detection on the image to be processed to obtain second position information corresponding to purple dots in the image to be processed; a purple edge area detection module configured to determine the purple edge area in the image to be processed according to the first position information and the second position information; a purple edge elimination module configured to perform purple edge elimination processing on the purple edge area in the image to be processed to obtain an image with purple edges eliminated.

9. An image processing apparatus, wherein, comprising: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to: execute the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, on which computer program instructions are stored, wherein, when the program instructions are executed by a processor, the steps of the method according to any one of claims 1-7 are implemented.

11. A chip, wherein, comprising a processor and an interface; the processor is used to read instructions to execute the method according to any one of claims 1-7.

Images