Image processing method and related device

Abstract

The embodiment of the present application provides an image processing method and a related device, the method comprising: obtaining a first pixel and a plurality of second pixels in an image to be processed, the distance between the first pixel and the second pixel being less than or equal to a first threshold value; performing color correction on the first pixel based on the hue value, brightness value and saturation value of the first pixel and the second pixel to obtain a processed image; the comparison result of the hue value of the first pixel and the second pixel is greater than or equal to a second threshold value, the comparison result of the brightness value of the first pixel and the second pixel is greater than or equal to a third threshold value, and the comparison result of the saturation value of the first pixel and the second pixel is less than or equal to a fourth threshold value. The present application can effectively remove false colors in an image.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and specifically to image processing methods and related devices. Background Art

[0002] In daily situations, due to the dispersion distortion of optical lenses, poor accuracy of interpolation methods, and large color noise, the initial images captured by electronic devices will inevitably contain wrong colors. The wrong colors in the image can be called false colors. For example, the real color of the word "flower" is black, but in the captured image, the edges of the "flower" appear in wrong colors such as purple, green, and yellow.

[0003] False colors not only affect the subjective visual perception of the human eye to the image, but also affect the subsequent research and application of the image and lead to erroneous results. Therefore, how to effectively remove false colors in images is the direction that technicians in this field are constantly working on. Summary of the invention

[0004] The embodiments of the present application provide an image processing method and a related device, which can effectively remove false colors in an image.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0006] Acquire a first pixel and a plurality of second pixels in the image to be processed, wherein a distance between the first pixel and the second pixel is less than or equal to a first threshold;

[0007] The first pixel is color corrected based on the hue value, brightness value and saturation value of the first pixel and the second pixel to obtain a processed image; the comparison result of the hue value of the first pixel and the second pixel is greater than or equal to the second threshold, the comparison result of the brightness value of the first pixel and the second pixel is greater than or equal to the third threshold, and the comparison result of the saturation value of the first pixel and the second pixel is less than or equal to the fourth threshold.

[0008] In combination with the first aspect, in a possible implementation manner, the step of obtaining a first pixel and a plurality of second pixels in the image to be processed includes:

[0009] Determining a boundary area based on segmentation information of the image to be processed, wherein the segmentation information indicates segmentation conditions of different objects included in the image to be processed;

[0010] The first pixel and the plurality of second pixels are acquired from the boundary area.

[0011] In combination with the first aspect, in a possible implementation method, the probability value corresponding to the first pixel is greater than or equal to the fifth threshold, and the probability value corresponding to the first pixel is determined based on the comparison result of the hue value between the first pixel and the second pixel, the comparison result of the brightness value between the first pixel and the second pixel, and the comparison result of the saturation value between the first pixel and the second pixel.

[0012] In combination with the first aspect, in a possible implementation manner, the color correction of the first pixel based on the hue value, brightness value, and saturation value of the first pixel and the second pixel to obtain a processed image includes:

[0013] The above-mentioned first pixel is color corrected based on the colors of multiple third pixels to obtain the above-mentioned processed image; the probability value corresponding to the above-mentioned third pixel is less than or equal to the sixth threshold, and the distance between the above-mentioned third pixel and the above-mentioned first pixel is less than or equal to the seventh threshold.

[0014] In combination with the first aspect, in a possible implementation manner, performing color correction on the first pixel based on colors of the plurality of third pixels to obtain the processed image includes:

[0015] The color of the first pixel is corrected to the color of the third pixel closest to the first pixel among the plurality of third pixels, to obtain the processed image.

[0016] In combination with the first aspect, in a possible implementation manner, performing color correction on the first pixel based on colors of the plurality of third pixels to obtain the processed image includes:

[0017] determining a target color based on a hue value of each third pixel in the plurality of third pixels and a corresponding weighted value, wherein the weighted value corresponding to the third pixel is inversely proportional to a distance between the third pixel and the first pixel;

[0018] The color of the first pixel is corrected to the target color to obtain the processed image.

[0019] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0020] Determine a hue mean value of the first pixel, wherein the hue mean value is an average value of hue values ​​of the first pixel and the plurality of second pixels in the image to be processed;

[0021] When the absolute value of the difference between the corrected hue value of the first pixel and the hue mean is greater than or equal to the eighth threshold, the color of the first pixel is modified based on the hue mean and reference information, and the reference information includes noise information and texture information of the image to be processed.

[0022] In a second aspect, an embodiment of the present application provides a method comprising a unit for executing the method in the first aspect or any possible implementation manner of the first aspect.

[0023] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory; the memory is used to store data and computer execution instructions; the processor is used to execute the computer execution instructions stored in the memory so that the method in the first aspect or any possible implementation of the first aspect is executed.

[0024] In a fourth aspect, an embodiment of the present application provides a chip, comprising a logic circuit and an interface, wherein the logic circuit and the interface are coupled; the interface is used to input and / or output code instructions, and the logic circuit is used to execute the code instructions so that the method in the first aspect or any possible implementation of the first aspect is executed.

[0025] In the fifth aspect, the present application provides a module device, which includes a communication module, a power module, a storage module and a chip module, wherein the power module is used to provide power to the module device; the storage module is used to store data and instructions; the communication module is used for internal communication of the module device, or for the module device to communicate with external devices; the chip module is used to execute the method in the first aspect or any possible implementation of the first aspect.

[0026] In a sixth aspect, an embodiment of the present application discloses a computer program product, which includes program instructions. When the program instructions are executed by a processor, the method in the first aspect or any possible implementation of the first aspect is executed.

[0027] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a processor, the method in the first aspect or any possible implementation of the first aspect is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0029] Figure 2 is a framework diagram of an image processing system provided in an embodiment of the present application;

[0030] Figure 3 is a schematic diagram of a data input module provided in an embodiment of the present application;

[0031] Figure 4is a schematic diagram of a preprocessing module provided in an embodiment of the present application;

[0032] Figure 5 is a schematic diagram of a false color detection module provided in an embodiment of the present application;

[0033] Figure 6 is a schematic diagram of a false color correction module provided in an embodiment of the present application;

[0034] Figure 7 is a schematic diagram of a post-processing module provided in an embodiment of the present application;

[0035] Figure 8 is a schematic diagram of a data output module provided in an embodiment of the present application;

[0036] Fig. 9 is a schematic diagram of an auxiliary module provided in an embodiment of the present application;

[0037] Fig.10 is a structural diagram of an electronic device 100 provided in an embodiment of the present application;

[0038] Fig.11 is a structural diagram of an electronic device 110 provided in an embodiment of the present application;

[0039] Fig.12 It is a structural schematic diagram of a module device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more of the listed items.

[0041] It should be noted that the terms "first", "second", "third" and "fourth" in the specification, claims and drawings of this application are used to distinguish different objects rather than to describe a specific order. It should also be understood that in the embodiments of this application, the numbers before the steps are made to facilitate understanding and description of the scheme, and should not be understood as limiting the order in which the steps are executed.

[0042] In daily life, we can take photos through electronic devices. Take a mobile phone as an example. The optical lens in the mobile phone can reduce the image of the object to project it onto the photosensitive element in the mobile phone. The photosensitive element generates changes in current and voltage due to the irradiation of light. The changes in current and voltage are transmitted to the chip for processing to obtain a photo of the object. The above chip can be an image signal processor (ISP) and related chip modules.

[0043] However, due to the dispersion distortion of optical lenses, inaccurate interpolation methods, and large color noise, it is inevitable that wrong colors will exist in the images collected by electronic devices. The wrong colors in the image can be called false colors. Common false colors include purple fringing and green fringing. It can be understood that once false colors are generated, they will become useless and harmful information. For example, false colors will have a bad impact on the subjective visual perception of the image by the human eye. For example, false colors will affect the subsequent research and application of the image, and may even lead to erroneous research results. Therefore, it is of great significance to remove false colors in images.

[0044] Based on the above problems, the embodiments of the present application provide an image processing method and related devices, by which false colors in images can be effectively removed. The method provided in the embodiments of the present application can be executed by an electronic device, wherein the electronic device can be any electronic device that can execute the technical solution disclosed in the embodiments of the method of the present application, and the electronic device can be a mobile phone, a tablet computer, a laptop computer, a monitor, a vehicle terminal, a mobile Internet device (mobile Internet devices, MID), a wearable device or a scanning device, etc., and can also be a terminal device, a server or a server cluster composed of multiple servers, etc., and the present application does not limit this.

[0045] Optionally, the method embodiment of the present application can also be implemented by a processor executing a computer program code, wherein the processor can be an ISP and a related chip module, etc. It can be understood that when the method embodiment of the present application is executed by a processor, the processor can be placed in an electronic device, for example, placed in an electronic device.

[0046] For example, see Figure 1 , Figure 1 It is a flowchart of an image processing method provided in an embodiment of the present application.

[0047] like Figure 1 As shown, the above method includes:

[0048] 101: Acquire a first pixel and a plurality of second pixels in an image to be processed, wherein a distance between the first pixel and the second pixel is less than or equal to a first threshold.

[0049] In the embodiment of the present application, the image to be processed can be understood as raw image data, such as an image acquired by a sensor and not compressed. Exemplarily, the image format of the image to be processed can be RAW (RAW image format, a color encoding method), RGB (red green blue, a color encoding method) and YUV (a color encoding method, Y represents brightness, U and V represent chrominance), etc.

[0050] The image to be processed includes multiple pixels, the first pixel can be understood as any pixel in the image to be processed, and the distance between the second pixel and the first pixel is less than or equal to the first threshold. The above distance can be a pixel distance or an absolute distance. The above first threshold can be determined according to the category and empirical value of the above distance, and the present application does not limit this. For example, when the above distance is a pixel distance, the above first threshold can be 3, that is, the pixel distance between the first pixel and the second pixel is less than or equal to 3 pixels.

[0051] 102: Perform color correction on the first pixel based on the hue value, brightness value and saturation value of the first pixel and the second pixel to obtain a processed image; the comparison result of the hue value of the first pixel and the second pixel is greater than or equal to the second threshold, the comparison result of the brightness value of the first pixel and the second pixel is greater than or equal to the third threshold, and the comparison result of the saturation value of the first pixel and the second pixel is less than or equal to the fourth threshold.

[0052] In this step, the color of the first pixel is corrected by using the hue value, brightness value and saturation value of the first pixel and the second pixel, wherein the hue value of the pixel can be used to represent the color of the pixel, the brightness value of the pixel can be used to represent the brightness of the pixel color, and the saturation of the pixel can be used to represent the chroma of the pixel color.

[0053] In this step, the comparison result of the hue value between the first pixel and the second pixel, the comparison result of the brightness value between the first pixel and the second pixel, and the comparison result of the saturation value between the first pixel and the second pixel can be obtained by a variety of comparison methods. For example, the comparison result can be the absolute value of the difference in numerical values, or it can be a gradient value, contrast, and mean square error, etc., as long as it can reflect the difference between the first pixel and the second pixel, and this application does not limit this.

[0054] In the embodiment of the present application, the specific values ​​of the second threshold, the third threshold and the fourth threshold are not limited, and can be limited according to empirical values ​​and information in the image to be processed. For example, the hue value in the image to be processed can be statistically analyzed first, and the second threshold can be set according to the range of the hue value. For another example, the image to be processed can be divided into regions, and different second thresholds can be set for different regions. It can be understood that the setting of the third threshold and the fourth threshold is similar to the setting of the second threshold. A single threshold can be set, or different thresholds can be set for different regions. It should be understood that the setting of the threshold can also be set according to the comparison method, that is, different thresholds can be set when different comparison methods are used to calculate the comparison results.

[0055] In an embodiment of the present application, when a comparison result of the hue values ​​of the first pixel and the second pixel is greater than or equal to a second threshold, a comparison result of the brightness values ​​of the first pixel and the second pixel is greater than or equal to a third threshold, and a comparison result of the saturation values ​​of the first pixel and the second pixel is less than or equal to a fourth threshold, the electronic device performs color correction on the first pixel, wherein color correction can be understood as modifying the color of the pixel, for example, it can be understood as modifying the hue value of the color of the pixel.

[0056] For ease of understanding, pixels that meet the above conditions are called false color pixels, and pixels that do not meet the above conditions are called non-false color pixels. It can be understood that since the first pixel is any pixel in the image to be processed, that is, each pixel in each image to be processed will be processed in the same way. Therefore, Figure 1 The method shown can be understood as a loop, and the loop is terminated after each pixel in the image to be processed is determined to be the first pixel.

[0057] Generally, the pixel area where the false color is located in the image has a large difference in hue value and brightness value with the surrounding pixel areas, but a small difference in saturation. Based on the above characteristics (referred to as false color characteristics), in an embodiment of the present application, the electronic device performs color correction on the first pixel based on the hue value, brightness value and saturation value of the first pixel and the second pixel to obtain a processed image; wherein the comparison result of the hue value of the first pixel and the second pixel is greater than or equal to the second threshold, the comparison result of the brightness value of the first pixel and the second pixel is greater than or equal to the third threshold, the comparison result of the saturation value of the first pixel and the second pixel is less than or equal to the fourth threshold, and the distance between the first pixel and the second pixel is less than or equal to the first threshold. In the above manner, the false color pixels in the image to be processed can be effectively detected, and the false colors in the image to be processed can be effectively removed.

[0058] It should also be understood that compared to other solutions in which fixed colors (such as common false colors such as green or purple) are selected to detect false color pixels, the false color pixels detected based on false color features in the embodiments of the present application can include various different colors, thereby increasing the types of false colors that can be detected and removed, making the types of false color removal no longer single, thereby improving the flexibility of false color removal.

[0059] In some embodiments, step 101, obtaining a first pixel and a plurality of second pixels in an image to be processed, includes:

[0060] 1011: Determine a boundary area based on segmentation information of the image to be processed, where the segmentation information indicates segmentation conditions of different objects included in the image to be processed.

[0061] In this embodiment, the segmentation information indicates the segmentation of different objects included in the image to be processed, such as the segmentation of the foreground and background in the image to be processed. The foreground may also include multiple objects, so the segmentation information may also include the segmentation of different objects in the multiple objects. In one possible implementation, the electronic device may obtain the segmentation information from other devices; in another possible implementation, the electronic device may first obtain the image to be processed, and then perform segmentation processing on the image to be processed to obtain the segmentation information.

[0062] Exemplarily, the segmentation information may be a segmentation line, such as a contour line of an object. The electronic device may determine the boundary area based on pixels around the segmentation line, for example, pixels whose distance from the pixel where the segmentation line is located is less than or equal to a threshold value A may be used as the boundary area, wherein the distance may be a pixel distance or an absolute distance. The threshold value A may be determined based on an empirical value, such as 10 pixels, etc., and this application does not limit this.

[0063] 1012: Acquire a first pixel and the plurality of second pixels from the boundary area.

[0064] After determining the boundary area, the electronic device obtains the first pixel and the plurality of second pixels from the boundary area. The first pixel can be understood as any pixel in the above-mentioned boundary area. It can be understood that each pixel in the boundary area will be subjected to the same processing (i.e., the processing of the above-mentioned step 102), and therefore, the method including step 1011 and step 1012 can be understood as a loop, and the loop is terminated after each pixel in the boundary area is determined as the first pixel.

[0065] It is understandable that false color pixels generally exist at the boundary (near the contour line) of the object in the image to be processed, but not inside the object. Therefore, the efficiency of removing false colors from the image to be processed can be improved by first determining the boundary area based on the segmentation information of the image to be processed, and then detecting the false color pixels from the boundary area for color correction.

[0066] In some embodiments, the probability value corresponding to the first pixel is greater than or equal to a fifth threshold, and the probability value corresponding to the first pixel is determined based on the comparison result of the hue value between the first pixel and the second pixel, the comparison result of the brightness value between the first pixel and the second pixel, and the comparison result of the saturation value between the first pixel and the second pixel.

[0067] In this embodiment, the probability value corresponding to a pixel is used to characterize the probability that the pixel is a false color pixel, and the magnitude of the probability value is proportional to the possibility that the pixel is a false color pixel. Taking the first pixel as an example, the probability value corresponding to the first pixel can be specifically determined according to the comparison result of the hue value of the first pixel and the second pixel, the comparison result of the brightness value of the first pixel and the second pixel, and the comparison result of the saturation value of the first pixel and the second pixel.

[0068] For example, according to the characteristics of false color pixels, the greater the difference in hue value between a pixel and surrounding pixels, the greater the difference in brightness value but the smaller the difference in saturation, the greater the probability that the pixel is a false color pixel. Therefore, multiple thresholds can be set for the comparison result, and different thresholds correspond to different probability values, and finally a final probability value is obtained based on each probability value. Optionally, the above probability value can also be represented by a weight value.

[0069] Taking the comparison method as mean square error as an example, when the mean square error result of the hue value of the first pixel and the second pixel (or multiple second pixels) is within interval A, the probability value A is assigned; when the mean square error result is within interval B, the probability value B is assigned. When the mean square error result of the brightness value of the first pixel and the second pixel (or multiple second pixels) is within interval C, the probability value C is assigned; when the mean square error result is within interval D, the probability value D is assigned. When the mean square error result of the saturation value of the first pixel and the second pixel (or multiple second pixels) is within interval E, the probability value E is assigned; when the mean square error result is within interval F, the probability value F is assigned. For example, the mean square error of the hue value of the first pixel and the second pixel (or multiple second pixels) is within interval A, the mean square error of the brightness value is within interval D, and the mean square error of the saturation value is within interval F, then the probability corresponding to the first pixel is obtained based on the probability value A, the probability value D and the probability value F, which can be specifically obtained by averaging, weighted averaging or direct multiplication, etc., which is not limited in this application.

[0070] It can be understood that the above method is only an example. In the specific implementation, more intervals can be divided, and the starting value and the ending value of the interval can also be set according to the specific situation. This application does not limit this.

[0071] In one possible implementation, the electronic device can calculate a corresponding probability value for each pixel in the image to be processed, which is used to characterize the probability that each pixel is a false color pixel. Finally, the pixel whose probability value is greater than or equal to the fifth threshold value can be understood as a false color pixel, that is, a pixel that needs color correction. Among them, the fifth threshold value can be set according to experience, for example, it can be any value greater than or equal to 80%, and this application does not limit this. In another possible implementation, the electronic device can further calculate the corresponding probability value when it is determined that the first pixel meets the conditions, wherein the above conditions include: the comparison result of the hue value of the first pixel and the second pixel is greater than or equal to the second threshold value, the comparison result of the brightness value of the first pixel and the second pixel is greater than or equal to the third threshold value, and the comparison result of the saturation value of the first pixel and the second pixel is less than or equal to the fourth threshold value.

[0072] In this embodiment, the probability value is determined by comparing the hue value, brightness value and saturation value between the first pixel and the second pixel. The probability value can be used to intuitively represent the probability that the first pixel is a false color pixel, which facilitates subsequent processing (such as color correction) procedures.

[0073] In some embodiments, step 102, color correction of the first pixel based on the hue value, brightness value, and saturation value of the first pixel and the second pixel to obtain a processed image, includes:

[0074] 1021: Perform color correction on the first pixel based on colors of multiple third pixels to obtain a processed image; the probability value corresponding to the third pixel is less than or equal to the sixth threshold, and the distance between the third pixel and the first pixel is less than or equal to the seventh threshold.

[0075] In this embodiment, the sixth threshold value can be set according to an empirical value or a specific situation, for example, it can be any value less than 30%, and this application does not limit this. The distance between the first pixel and the third pixel can be a pixel distance or an absolute distance; the seventh threshold value can be set according to an empirical value or a specific situation, and this application does not limit this. The method for determining the probability value corresponding to the third pixel is similar to the method for determining the probability value corresponding to the first pixel. For details, please refer to the previous embodiment, which will not be repeated here.

[0076] In this embodiment, the probability value corresponding to the third pixel is less than or equal to the sixth threshold, and the distance between the third pixel and the first pixel is less than or equal to the seventh threshold. Therefore, the third pixel can be understood as a non-false color pixel around the first pixel, and the color of the third pixel can be understood as a normal (or correct) color. By performing color modification based on multiple third pixels around the first pixel, the color of the first pixel can be effectively corrected.

[0077] In a possible implementation, step 1021, performing color correction on the first pixel based on the colors of the plurality of third pixels to obtain a processed image, includes:

[0078] 10211: Correct the color of the first pixel to the color of the third pixel closest to the first pixel among the plurality of third pixels to obtain a processed image.

[0079] In the embodiment of the present application, the electronic device may modify the color of the first pixel by correcting the hue value of the first pixel, that is, correcting the hue value of the first pixel to the hue value of the third pixel closest to the first pixel among multiple third pixels.

[0080] In another possible implementation, step 1021, performing color correction on the first pixel based on the colors of the plurality of third pixels to obtain a processed image, includes:

[0081] 10212: Determine a target color based on a hue value of each third pixel in a plurality of third pixels and a corresponding weighted value, wherein the weighted value corresponding to the third pixel is inversely proportional to a distance between the third pixel and the first pixel;

[0082] 10213: Correct the color of the first pixel to the target color to obtain a processed image.

[0083] In this embodiment, the electronic device determines the target color by comprehensively analyzing the colors of each third pixel surrounding the first pixel, so that the color transition between the color-corrected first pixel and the surrounding pixels can be better.

[0084] It is understandable that the weight value corresponding to the third pixel can be set according to the distance between the third pixel and the first pixel, and the farther the third pixel is from the first pixel, the smaller the weight value corresponding to the third pixel. It is understandable that in the case where there are multiple third pixels with the same distance from the first pixel, the weight value can be further set according to the number of pixels of the same color. For example, the third pixel A and the third pixel B have the same distance from the first pixel, the first pixel is color A, and the second pixel is color B. Among the multiple third pixels around the first pixel, the number of pixels of color A is greater than the number of pixels of color B. Therefore, the weight value corresponding to the third pixel A is greater than the weight value corresponding to the third pixel B.

[0085] It can be understood that the target color determined by the electronic device can specifically be a hue value, that is, a target hue value, and correcting the color of the first pixel to the target color can be understood as correcting the hue value of the first pixel to the target hue value.

[0086] In some embodiments, Figure 1 The method shown also includes:

[0087] 103: Determine a hue mean value of the first pixel, where the hue mean value is an average value of hue values ​​of the first pixel and a plurality of second pixels in the image to be processed.

[0088] In this step, the hue mean value of the first pixel may be understood as the hue mean value before pseudo-color processing, and may specifically be the average value of the hue values ​​of the first pixel and the plurality of second pixels.

[0089] 104: When the absolute value of the difference between the corrected hue value of the first pixel and the hue mean is greater than or equal to an eighth threshold, modify the color of the first pixel based on the hue mean and reference information, where the reference information includes noise information and texture information of the image to be processed.

[0090] In the embodiment of the present application, the eighth threshold value can be set according to the specific actual situation and experience value, and the present application does not limit this. For example, it can be set according to the hue value interval in the image to be processed, for example, it can be any value that is less than the maximum difference of the hue value in the image to be processed.

[0091] It is understandable that when the absolute value of the difference between the corrected hue value of the first pixel and the hue mean value is greater than or equal to the eighth threshold, it can be considered that the color correction of the first pixel is wrong. Generally, noise information, texture information, etc. in the image to be processed will cause data difference fluctuations, for example, noise will make the hue value difference larger.

[0092] The present application further modifies the color of the first pixel when there is a large difference between the hue values ​​before and after the color correction of the first pixel, which can reduce the probability of false color correction errors. Exemplarily, the hue mean value and the corrected hue value of the first pixel can be weighted to determine a new hue value. For example, when the noise is greater than the intensity threshold and the texture information change is greater than the reference threshold, the hue mean value of the first pixel is assigned a first weight value, and the corrected hue value is assigned a second weight value, wherein the first weight value is greater than the second weight value, and the hue mean value and the corrected hue value are weighted averaged based on the first weight value and the second weight value to obtain a new hue value.

[0093] In order to more clearly understand the method provided in the embodiment of the present application, Figure 2 The image processing system shown introduces the above-mentioned image processing method.

[0094] like Figure 2 As shown, the electronic device may include an image processing system, which includes a data input module, a preprocessing module, a false color processing module, a post-processing module, and a data output module, wherein the false color processing module includes a false color detection module and a false color correction module. Optionally, the false color processing module may also include an auxiliary module and a fusion module. The description of each module is as follows:

[0095] The data input module is used to obtain image data, such as the image to be processed in step 101 above. Optionally, the data input module can also be used to obtain auxiliary information, which is used in the auxiliary module. Figure 3 As shown, the image data format input by the data input module can be RAW data, RGB data, or YUV data. The auxiliary information input by the data input module can be environmental information, such as color temperature information, white balance information, and color information, or noise information, such as signal-to-noise ratio information and color noise size; it can also be sensor information, lens information (such as field of view), etc.

[0096] like Figure 4 As shown, the preprocessing module includes a color gamut conversion module and a color correction module. The color gamut conversion module is used to perform color gamut conversion on the image data, and the color correction module is used to perform color correction on the image data. Exemplarily, the color gamut conversion module can be used to convert RAW into RGB, RGB into YUV, RAW into YUV, etc., which is not limited in this application. Among them, in the process of converting RAW data to three channels, the RAW data can be interpolated into three-channel RGB data. The color correction module can perform color correction on the image after color gamut conversion to reduce the probability of false colors caused by inaccurate colors.

[0097] The false color processing module is used to analyze and integrate the input image data, or the input image data and auxiliary information, so as to detect and correct the false color. The false color processing module mainly includes a false color detection module and a false color correction module, and optionally may also include an auxiliary module and a fusion module. The introduction of each module is as follows:

[0098] like Figure 5 As shown, the false color detection module includes a latent feature simulation module and a real information simulation module. The latent feature simulation module obtains the feature information of the false color by calculating the hue value, brightness value difference and saturation difference; the real information simulation module obtains the real information to be verified in the local pixel area, such as the local hue value, local brightness value and local saturation value, by calculating the local hue information and brightness saturation information.

[0099] Exemplarily, the latent feature simulation module can determine the probability that a certain pixel is a false color pixel based on the hue value, brightness value difference and saturation difference. For details, please refer to the relevant description of the probability value corresponding to the first pixel in the previous text. The real information simulation module is used to screen based on the probability value corresponding to the pixel, for example, taking the pixel with a probability value greater than 85% as a false color pixel for false color correction.

[0100] like Figure 6 As shown, the false color correction module includes a false color information correction module and a transition information protection module. The false color information correction module corrects the color of the false color pixel with reference to the surrounding real color. At the same time, the excessive information protection module is used to protect the local excessive information, such as hue continuity, so that the transition between the corrected pixel and the surrounding normal area is better and the image is more natural. Exemplarily, the false color information correction module can execute the above step 10211, and the transition information protection module can execute the above steps 10212 and 10213.

[0101] The fusion module is used to fuse the corrected data with the original data. For example, by referring to auxiliary information (such as noise information) and the difference between the data before and after correction, the data before and after correction are weighted respectively, thereby further modifying the hue value of the false color pixel. The fusion module can help manually adjust the degree of false color suppression and improve the flexibility of the system's false color suppression.

[0102] like Figure 7 As shown, the post-processing module is used to perform post-processing on the image data after the false color removal processing, such as RAW data restoration, filtering processing, and color adjustment processing. Exemplarily, for the output RAW data format, the image after the false color removal processing can be extracted and restored, and for the output RGB / YUV data format, the pixels in the false color correction area can be filtered, so that the image after the false color removal has a more natural transition.

[0103] like Figure 8 As shown, the data output module is used to output the post-processed image data, and the output information includes the image data after the false color removal processing and auxiliary information. The output auxiliary information may include color type, correction information (such as correction intensity), etc., wherein the output of auxiliary information related to false color is conducive to backtracking and modification.

[0104] Optionally, the probability of whether the pixel at the current position is a false color can be further determined by using the color information, noise information, and image feature information in the auxiliary information. Fig. 9 As shown, the auxiliary modules may include artificial intelligence (AI) auxiliary modules and traditional auxiliary modules.

[0105] The AI-assisted module can be used to obtain scene segmentation information, semantic segmentation information, and edge detection information. The information obtained by the AI-assisted module can be understood as the segmentation of different objects in the image data, such as the separation of foreground and background. Since false colors generally appear at the boundaries of objects, the segmentation information can be used to roughly locate the area where the false color pixels are located, thereby improving the efficiency of false color removal and removing false colors from the image more quickly.

[0106] Traditional auxiliary modules can be used to obtain color information, noise information, and defocus information, etc., to improve the accuracy of removing false colors from images. For example, noise information, noise will cause fluctuations in data differences. When calculating the difference between the current pixel and the surrounding pixels, the noise can be combined to obtain the final difference result. Another example is color temperature information. If the image data is taken under a yellow light (rather than an incandescent light) scene, then the color of the most likely false color can be analyzed based on the yellow light scene, thereby assisting in detecting whether a certain pixel is a false color pixel. For example, defocus information (focused or unfocused) can reflect the clarity of the image. If it is out of focus, the image clarity is low and the data difference will be relatively small, so the threshold (such as the second threshold mentioned above) can be adjusted to a smaller value.

[0107] The method of the embodiment of the present application is described in detail above, and the device provided by the embodiment of the present application is described below.

[0108] For example, see Fig.10 , Fig.10 1 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application. Fig.10 As shown, the electronic device 100 includes an acquisition unit 1001 and a color correction unit 1002. Optionally, it may also include a determination unit 1003 and a color modification unit 1004, which are described in detail as follows:

[0109] An acquisition unit 1001 is used to acquire a first pixel and a plurality of second pixels in an image to be processed, wherein a distance between the first pixel and the second pixel is less than or equal to a first threshold;

[0110] The color correction unit 1002 is used to perform color correction on the first pixel based on the hue value, brightness value and saturation value of the first pixel and the second pixel to obtain a processed image; the comparison result of the hue value of the first pixel and the second pixel is greater than or equal to the second threshold, the comparison result of the brightness value of the first pixel and the second pixel is greater than or equal to the third threshold, and the comparison result of the saturation value of the first pixel and the second pixel is less than or equal to the fourth threshold.

[0111] In a possible implementation, the determining unit 1003 is configured to determine the boundary region based on the segmentation information of the image to be processed, where the segmentation information indicates the segmentation status of different objects included in the image to be processed;

[0112] The acquisition unit 1001 is specifically configured to acquire the first pixel and the plurality of second pixels from the boundary area.

[0113] In one possible implementation, the probability value corresponding to the first pixel is greater than or equal to the fifth threshold, and the probability value corresponding to the first pixel is determined based on the comparison result of the hue value between the first pixel and the second pixel, the comparison result of the brightness value between the first pixel and the second pixel, and the comparison result of the saturation value between the first pixel and the second pixel.

[0114] In a possible implementation, the color correction unit 1002 is specifically used to perform color correction on the above-mentioned first pixel based on the colors of multiple third pixels to obtain the above-mentioned processed image; the probability value corresponding to the above-mentioned third pixel is less than or equal to the sixth threshold, and the distance between the above-mentioned third pixel and the above-mentioned first pixel is less than or equal to the seventh threshold.

[0115] In a possible implementation, the color correction unit 1002 is specifically configured to correct the color of the first pixel to the color of the third pixel closest to the first pixel among the plurality of third pixels, so as to obtain the processed image.

[0116] In a possible implementation, the color correction unit 1002 is specifically configured to determine the target color based on the hue value of each third pixel in the plurality of third pixels and a corresponding weighted value, wherein the weighted value corresponding to the third pixel is inversely proportional to the distance between the third pixel and the first pixel;

[0117] The color correction unit 1002 is specifically configured to correct the color of the first pixel to the target color to obtain the processed image.

[0118] In a possible implementation, the determining unit 1003 is further configured to determine a hue mean value of the first pixel, where the hue mean value is an average value of hue values ​​of the first pixel and the plurality of second pixels in the image to be processed;

[0119] The color modification unit 1004 is used to modify the color of the above-mentioned first pixel based on the above-mentioned hue mean and reference information when the absolute value of the difference between the corrected hue value of the above-mentioned first pixel and the above-mentioned hue mean is greater than or equal to an eighth threshold, and the above-mentioned reference information includes the noise information and texture information of the above-mentioned image to be processed.

[0120] See also Fig.11 , Fig.11 1 is a schematic diagram of the structure of an electronic device 110 provided in an embodiment of the present application. Fig.11 As shown, the electronic device 110 includes a memory 1101 and a processor 1102. Optionally, it may also include a communication interface 1103 and a bus 1104, wherein the memory 1101, the processor 1102 and the communication interface 1103 are connected to each other through the bus 1104.

[0121] The memory 1101 is used to provide a storage space, in which data such as an operating system and a computer program can be stored. The memory 1101 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM).

[0122] The processor 1102 is a module that performs arithmetic and logical operations, and may be one or a combination of multiple processing modules such as a central processing unit (CPU), a graphics processing unit (GPU), or a microprocessor unit (MPU).

[0123] The memory 1101 stores a computer program, and the processor 1102 calls the computer program stored in the memory 1101 to execute the above-mentioned image registration method. Exemplarily, when the above-mentioned electronic device 110 is the above-mentioned electronic device 100, the content acquired by the above-mentioned acquisition unit 1001 can be implemented by the communication interface 1103, and the steps performed by the above-mentioned color correction unit 1002, the determination unit 1003 and the color modification unit 1004 can be implemented by the processor 1102.

[0124] Optionally, the steps performed by the pre-processing module, the false color processing module (such as the auxiliary module, the false color detection module and the false color correction module therein), the fusion module and the post-processing module may be implemented by the processor 1102. The steps performed by the data input module and the data output module may be implemented by the communication interface 1103.

[0125] The embodiment of the present application also provides a chip. The chip includes: a processor and a memory. The number of processors can be one or more, and the number of memories can be one or more. The processor can execute the above method and the steps executed by the related implementation methods by reading the instructions and data stored in the memory. Of course, the chip may also have no memory.

[0126] For example, see Fig.12 , Fig.12 It is a structural schematic diagram of a module device provided in an embodiment of the present application. The module device 120 can execute the relevant steps of the electronic device in the aforementioned method embodiment. The module device 120 includes: a communication module 1201, a power module 1202, a storage module 1203 and a chip module 1204. Among them, the power module 1202 is used to provide power to the module device; the storage module 1203 is used to store data and instructions; the communication module 1201 is used for internal communication of the module device, or for the module device to communicate with external devices, and is used to execute the steps executed by the above-mentioned acquisition unit 1001; the chip module 1204 can execute the steps executed by the above-mentioned color correction unit 1002, the determination unit 1003 and the color modification unit 1004.

[0127] The present application also provides a computer-readable storage medium, in which computer codes are stored. When the computer codes are executed on a computer, the computer executes the method of the above embodiment.

[0128] The present application also provides a computer program product, which includes a computer code or a computer program. When the computer code or the computer program runs on a computer, the method in the above embodiment is executed.

[0129] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the above claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first pixel and a plurality of second pixels in the image to be processed, wherein a distance between the first pixel and the second pixel is less than or equal to a first threshold; Determine a probability value corresponding to the first pixel based on a comparison result of the hue values ​​of the first pixel and the second pixel, a comparison result of the brightness values ​​of the first pixel and the second pixel, and a comparison result of the saturation values ​​of the first pixel and the second pixel, wherein the probability value corresponding to the first pixel is used to characterize the probability that the first pixel is a false color pixel; wherein the comparison result of the hue values ​​of the first pixel and the second pixel is greater than or equal to a second threshold, the comparison result of the brightness values ​​of the first pixel and the second pixel is greater than or equal to a third threshold, and the comparison result of the saturation values ​​of the first pixel and the second pixel is less than or equal to a fourth threshold; Based on the colors of multiple third pixels, color correction is performed on the first pixel whose probability value is greater than the fifth threshold to obtain a processed image; the probability value corresponding to the third pixel is less than or equal to the sixth threshold, and the distance between the third pixel and the first pixel is less than or equal to the seventh threshold; the probability value corresponding to the third pixel is used to characterize the probability that the third pixel is a false color pixel.

2. The method according to claim 1, characterized in that The step of acquiring a first pixel and a plurality of second pixels in an image to be processed includes: Determining a boundary area based on segmentation information of the image to be processed, wherein the segmentation information indicates segmentation conditions of different objects included in the image to be processed; The first pixel and the plurality of second pixels are acquired from the boundary area.

3. The method according to claim 1, characterized in that The step of performing color correction on a first pixel whose probability value is greater than a fifth threshold based on the colors of the plurality of third pixels to obtain a processed image includes: The color of the first pixel is corrected to the color of a third pixel among the plurality of third pixels that is closest to the first pixel, to obtain the processed image.

4. The method according to claim 1, characterized in that: The step of performing color correction on a first pixel whose probability value is greater than a fifth threshold based on the colors of the plurality of third pixels to obtain a processed image includes: determining a target color based on a hue value of each third pixel in the plurality of third pixels and a corresponding weighted value, wherein the weighted value corresponding to the third pixel is inversely proportional to a distance between the third pixel and the first pixel; The color of the first pixel is corrected to the target color to obtain the processed image.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Determine a hue mean value of the first pixel, where the hue mean value is an average value of hue values ​​of the first pixel and the plurality of second pixels in the image to be processed; When the absolute value of the difference between the corrected hue value of the first pixel and the hue mean is greater than or equal to an eighth threshold, the color of the first pixel is modified based on the hue mean and reference information, wherein the reference information includes noise information and texture information of the image to be processed.

6. An electronic device, characterized in that: The method comprises means for executing the method as claimed in any one of claims 1 to 5.

7. An electronic device, characterized in that: include: Processor and memory; The memory is used to store data and computer execution instructions; The processor is used to execute the computer-executable instructions stored in the memory so that the method according to any one of claims 1 to 5 is executed.

8. A chip, characterized in that: comprising a logic circuit and an interface, wherein the logic circuit and the interface are coupled; The interface is used to input and / or output code instructions, and the logic circuit is used to execute the code instructions so that the method described in any one of claims 1 to 5 is executed.

9. A module device, characterized in that: The module device includes a communication module, a power module, a storage module and a chip module, wherein the power module is used to provide power to the module device; the storage module is used to store data and instructions; the communication module is used for internal communication of the module device, or for the module device to communicate with external devices; the chip module is used to execute the method described in any one of claims 1-5.

10. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the method according to any one of claims 1 to 5 is executed.

Images