Image processing method, device, equipment, chip, storage medium and program product

Abstract

The present application discloses an image processing method, apparatus, device, chip, storage medium and program product, the method comprising: determining a first evaluation image and a second evaluation image having the same evaluation object, the first evaluation image and the second evaluation image being determined based on images obtained by shooting the same shooting content; extracting at least one color feature of the first evaluation image to obtain a first color feature quantity, and extracting color features of the second evaluation image to obtain a second color feature quantity; extracting environmental information for shooting the first evaluation image and / or the second evaluation image; and determining the color difference between the first color feature quantity and the second color feature quantity based on a judgment standard corresponding to the environmental information to determine a color evaluation result.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, equipment, chip, storage medium and program product. Background Art

[0002] In recent years, color consistency has shown broad application prospects in the fields of photography or image processing, and has become a research hotspot in the field of computer vision. Among them, the color evaluation between different images taken is mainly achieved through human eyes or color card comparison. Although some image color evaluations already exist in related technologies, there are subjective influences or the evaluation is relatively limited, resulting in inaccurate color consistency evaluation. Summary of the invention

[0003] The present application proposes an image processing method, device, equipment, chip, storage medium and program product, which combine environmental information, adopt judgment criteria corresponding to the environmental information, objectively judge the color consistency of different images, and are relatively stable; and can also consider multiple dimensions in the extraction of color features, thereby expanding the scope of evaluation and improving the accuracy of color evaluation.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0006] Determining a first evaluation image and a second evaluation image having the same evaluation object, wherein the first evaluation image and the second evaluation image are determined based on images obtained by photographing the same photographing content;

[0007] Extracting at least one color feature from the first evaluation image to obtain a first color feature value, and extracting color features from the second evaluation image to obtain a second color feature value;

[0008] Extracting environment information for taking the first evaluation image and / or the second evaluation image;

[0009] Based on a determination criterion corresponding to the environmental information, a color difference between the first color feature quantity and the second color feature quantity is determined to determine a color evaluation result.

[0010] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0011] A determination unit, configured to determine a first evaluation image and a second evaluation image having the same evaluation object, wherein the first evaluation image and the second evaluation image are determined based on images obtained by photographing the same photographing content;

[0012] an extraction unit, configured to extract color features from the first evaluation image to obtain a first color feature quantity, and to extract at least one color feature from the second evaluation image to obtain a second color feature quantity; and to extract environment information in which the first evaluation image and / or the second evaluation image are taken;

[0013] An evaluation unit is used to determine the color difference between the first color feature quantity and the second color feature quantity based on a determination standard corresponding to the environmental information, and determine a color evaluation result.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, wherein:

[0015] A memory for storing computer programs that can be executed on the processor;

[0016] A processor is used to execute the steps of the image processing method as described in the first aspect when running a computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a chip, the chip including a processor, wherein:

[0018] The processor is used to call and run a computer program from a memory, so that a device equipped with the chip executes the steps of the image processing method as described in the first aspect.

[0019] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the image processing method described in the first aspect are implemented.

[0020] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of the image processing method described in the first aspect.

[0021] An image processing method, device, equipment, chip, storage medium and program product provided by the embodiment of the present application, determines a first evaluation image and a second evaluation image with the same evaluation object, the first evaluation image and the second evaluation image are determined based on images obtained by shooting the same shooting content; extracts at least one color feature of the first evaluation image to obtain a first color feature quantity, and extracts color features of the second evaluation image to obtain a second color feature quantity; extracts environmental information for shooting the first evaluation image and / or the second evaluation image; and determines the color difference between the first color feature quantity and the second color feature quantity based on the judgment criteria corresponding to the environmental information, and determines the color evaluation result. In this way, due to the combination of environmental information and the use of the judgment criteria corresponding to the environmental information, the color consistency of different images is objectively determined, so the color evaluation is relatively stable. In addition, since the extraction of color features can also be considered from multiple dimensions, the scope of evaluation and the diversity of evaluation are expanded, so the accuracy will be improved in the process of evaluating color consistency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 1 ;

[0023] Figure 2 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 2 ;

[0024] Figure 3 An exemplary image division diagram provided in the embodiment of the present application Figure 1 ;

[0025] Figure 4 An exemplary image division diagram provided in the embodiment of the present application Figure 2 ;

[0026] Figure 5 An exemplary pixel number-hue angle value statistical curve provided in an embodiment of the present application;

[0027] Figure 6 An exemplary processing framework of an image processing method provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of an exemplary pure color evaluation module provided in an embodiment of the present application;

[0029] Figure 8 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 3 ;

[0030] Fig. 9A schematic diagram of an image processing method provided in an embodiment of the present application Figure 4 ;

[0031] Fig.10 A schematic diagram of an exemplary semantic segmentation model provided in an embodiment of the present application;

[0032] Fig.11 A schematic diagram of a module for evaluating an exemplary non-pure color provided in an embodiment of the present application;

[0033] Fig.12 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0034] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0035] Fig.14 A schematic diagram of the composition structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0038] It should also be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0039] It should also be noted that in the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0040] To facilitate understanding of the technical solutions of the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below. The following related technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0041] The embodiment of the present application provides an image processing method, which determines a first evaluation image and a second evaluation image having the same evaluation object, wherein the first evaluation image and the second evaluation image are determined based on images obtained by shooting the same shooting content; extracts at least one color feature of the first evaluation image to obtain a first color feature quantity, and extracts color features of the second evaluation image to obtain a second color feature quantity; extracts environmental information of shooting the first evaluation image and / or the second evaluation image; and determines the color difference between the first color feature quantity and the second color feature quantity based on the judgment criteria corresponding to the environmental information, and determines the color evaluation result. In this way, since the environmental information is combined and the judgment criteria corresponding to the environmental information are adopted, the color consistency of different images is objectively judged, and therefore, the color evaluation is relatively stable. In addition, since the extraction of color features can also be considered from multiple dimensions, the scope of evaluation and the diversity of evaluation are expanded, and therefore, the accuracy will be improved in the process of evaluating color consistency.

[0042] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0043] In one embodiment of the present application, Figure 1 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 1 .like Figure 1 As shown, the method may include:

[0044] S101 : Determine a first evaluation image and a second evaluation image having the same evaluation object, wherein the first evaluation image and the second evaluation image are determined based on images obtained by photographing the same photographing content.

[0045] It should be noted that in the embodiments of the present application, the method can be applied to an image processing device, or an electronic device integrated with the device. The electronic device can be a device with image processing function or image shooting function, such as a mobile phone, a tablet computer, a laptop computer, a PDA, a smart watch, a personal computer, a camera, etc.

[0046] It should also be noted that, in the embodiment of the present application, the first evaluation image and the second evaluation image can be obtained based on images taken by multiple cameras of the electronic device, or can be obtained from images taken by different electronic devices, or can be obtained from images taken by the same device at different focal lengths. In short, for color images (such as RGB), color consistency evaluation can be performed between different images with the same object or subject in the image screen. The first evaluation image and the second evaluation image are comparison objects for color consistency, and the same evaluation subject or the same evaluation object exists in the image screens of the two.

[0047] Among them, the evaluation object or evaluation subject in the embodiment of the present application can be a specific individual or entity such as a person, an object, the sky, green plants, a building, etc., which is related to the shooting content and is not limited by the embodiment of the present application.

[0048] In the embodiments of the present application, the shooting content can be understood as the scene, object or entity targeted at the time of shooting, which refers to the content corresponding to the actual shooting scene, and is not the picture or result obtained by shooting.

[0049] It should be noted that the pair of images taken by multiple cameras of an electronic device may be the first image and the second image; the pair of images obtained by images taken by different electronic devices may be the first image and the second image; and the pair of images obtained by images taken by the same device at different focal lengths may be the first image and the second image.

[0050] In some embodiments of the present application, one or more electronic devices may capture a first image and a second image based on the same captured content, and the first evaluation image and the second evaluation image may be obtained by processing (e.g., registering) the captured first image and the second image.

[0051] It should also be noted that the first image and the second image may be captured simultaneously by one or more electronic devices in the same scene (same shooting content), or captured at a similar time, and this is not limited in the embodiments of the present application.

[0052] It should be noted that the field of view (FOV) varies based on the focal length. FOV refers to the range of images that a lens (camera) can capture. In the field of photography and videography, FOV is usually used to describe the width of a scene that a camera lens can capture. The shorter the focal length (corresponding to the focal length), the larger the FOV; the longer the focal length, the smaller the FOV.

[0053] In the embodiment of the present application, in the process of shooting based on the same shooting content, if different focal lengths or FOVs are used, the screen contents of the images (first image and second image) shot are different due to the difference in focal lengths or FOVs, wherein the screen contents of the larger FOV include the screen contents of the smaller FOV. In this case, the first image and the second image need to be registered so that they have the same evaluation object or the same evaluation subject, and the redundant screen contents are removed so as to determine the color consistency and ensure the accuracy of the evaluation.

[0054] In the embodiment of the present application, due to factors such as the conditions or field of view or environment or standing position during image acquisition, the contents in the image screens of the first image and the second image may not completely correspond or the distance and size of the evaluation object may be inconsistent, and there may be some inconsistencies in the contents in the image screens. Therefore, the first image and the second image can be registered to determine the first evaluation image and the second evaluation image with the same evaluation object or the same evaluation subject.

[0055] It should be noted that there is at least one same evaluation object or evaluation subject in the first evaluation image and the second evaluation image.

[0056] In the embodiment of the present application, the determination of color consistency is based on the comparison between two images. If there are more than two images that need to be determined, the image processing method provided in the embodiment of the present application can be used to compare the two images in pairs to obtain a color evaluation result.

[0057] It should also be noted that the application scenario of the image processing method provided in the embodiment of the present application can also be for an electronic device such as a mobile phone to collect the first image and the second image, and then execute the image processing method provided in the embodiment of the present application by transmitting the first image and the second image to another electronic device (such as a computer, or other electronic testing equipment, etc.), and the embodiment of the present application is not limited.

[0058] In the above case, the device that captures the first image and the second image may be an electronic device with multiple cameras, or may be different electronic devices with a shooting function, or may be different focal lengths of the same electronic device, etc.

[0059] In some embodiments of the present application, taking focal length as an example, when comparing or color evaluating multiple images of different focal lengths with the same shooting content or evaluation object, image registration is performed first. The electronic device can compare images (including the first image and the second image) taken by switching the focal length of the camera, such as switching to 1X (such as the first image), the second image captured at 0.9X or the second image captured at 1.1X can be registered, or images taken at adjacent focal lengths can be registered, or images taken at a certain focal length (the first image or the second image) can be registered, and the embodiments of the present application are not limited thereto.

[0060] Exemplarily, the focal length can be 0.6X, 1X, 3X, 6X, etc. A triple-camera mobile phone can include a main camera, an ultra-wide-angle camera, and a telephoto / macro camera, corresponding to 1X, 3X, 0.6X, etc.

[0061] In some embodiments of the present application, the first image is used as a reference and image registration is performed with the second image to obtain a first evaluation image and a second evaluation image.

[0062] Among them, the first field of view of the first image is smaller than the second field of view of the second image or the focal length of the first image is larger than the focal length of the second image, and they have the same evaluation object or shooting content.

[0063] In an embodiment of the present application, if the first image and the second image are images with different FOVs or focal lengths, the first field of view angle of the first image is smaller than the second field of view angle of the second image, or the focal length of the first image is larger than the focal length of the second image, then the second image has more picture content. Based on the first image, the second image is registered, that is, a picture with the same content as the first image (that is, the same evaluation object or the same evaluation subject) is selected or intercepted to obtain a second evaluation image. At this time, the first evaluation image may be the first image.

[0064] For example, assuming that a 1X image is used as a comparison object, when 0.6X and 1X jpg images are selected for color evaluation, the FOV corresponding to 1X is used for registration.

[0065] When selecting 2X and 1X jpg for color evaluation, the FOV corresponding to 2X is used for registration.

[0066] When selecting 3X and 1X jpg for color evaluation, the FOV corresponding to 3X is used for registration.

[0067] When selecting 6X and 1X jpg for color evaluation, the FOV corresponding to 6X is used for registration.

[0068] For example, assuming that 0.9X and 1X jpgs are selected for color evaluation, the registration is performed using the FOV corresponding to 1X.

[0069] When selecting 2.9X and 3X jpg for color evaluation, the FOV corresponding to 3X is used for registration.

[0070] When selecting 5.9X and 6X jpg for color evaluation, the FOV corresponding to 6X is used for registration.

[0071] S102 , extract at least one color feature from the first evaluation image to obtain a first color feature quantity, and extract color features from the second evaluation image to obtain a second color feature quantity.

[0072] In the embodiment of the present application, the electronic device performs color consistency evaluation on the first evaluation image and the second evaluation image, firstly extracts color features from the first evaluation image and the second evaluation image, and performs color evaluation based on the extracted color features. The first evaluation image is subjected to color feature extraction to obtain a first color feature; the second evaluation image is subjected to color feature extraction to obtain a second color feature.

[0073] It should be noted that in the embodiment of the present application, the dimensions and methods for weighting the color features of the first evaluation image are consistent with the dimensions and methods for extracting the color features of the second evaluation image to ensure the correspondence and accuracy of subsequent color comparison.

[0074] In the embodiment of the present application, the color feature may include a color value and a color proportion of the color in the entire image.

[0075] In some embodiments of the present application, the color value may be an RGB value, or an average RGB value, which is not limited in the embodiments of the present application.

[0076] In some embodiments of the present application, the electronic device may store the first color feature in a first color buffer and the second color feature in a second color buffer, wherein the first color feature and the second color feature both include a color value and its color proportion in the entire image.

[0077] It should be noted that the color buffer may be a Color RGB buffer, which is not limited in the embodiment of the present application.

[0078] In an embodiment of the present application, the dimensions of color features may include at least one of the following: gray area color, sky color, global white balance, facial skin color, color card color, main color, clothing color (which can also be used as a type of main color and is a dimension with the main color, which is not limited in the embodiment of the present application) and pure color.

[0079] Each evaluation image may include at least one color feature quantity of the above dimensions. The dimensions of the color features of the two evaluation images for each evaluation may be consistent to ensure the objectivity and accuracy of the color evaluation.

[0080] S103: Extracting environment information for shooting the first evaluation image and / or the second evaluation image.

[0081] In the embodiment of the present application, the two images to be evaluated, i.e., the first evaluation image and the second evaluation image, are obtained from images taken based on the same scene and at the same or similar time. Therefore, the environmental information of the two evaluation images is the same or similar. The electronic device can extract environmental information from any evaluation image, or can extract environmental information from both the first evaluation image and the second evaluation image, which is not limited in the embodiment of the present application.

[0082] In an embodiment of the present application, the environmental information may include: at least one of color temperature, scene illumination, and dynamic range.

[0083] It should be noted that the differences in environmental information such as color temperature, scene illumination, and dynamic range when taking images will lead to color differences in the images.

[0084] In the embodiment of the present application, environmental information is used to provide a basis for color evaluation.

[0085] S104 . Determine the color difference between the first color feature quantity and the second color feature quantity based on a determination standard corresponding to the environmental information, and determine a color evaluation result.

[0086] In an embodiment of the present application, different environmental information will affect the color changes of the image. Therefore, when the electronic device performs color evaluation, it can determine the judgment criteria for this evaluation based on the environmental information of the first evaluation image and the second evaluation image currently being evaluated, and perform color evaluation with the judgment criteria corresponding to the environmental information, thereby increasing the accuracy of the evaluation.

[0087] In some embodiments of the present application, a corresponding judgment criterion is determined based on at least one of the color temperature, scene illumination and dynamic range in the environmental information; the judgment criterion represents an allowable error range; the color difference between the first color feature quantity and the second color feature quantity is determined; the color difference between the first color feature quantity and the second color feature quantity is compared with the judgment criterion to determine a color evaluation result.

[0088] It should be noted that the judgment criteria for different environmental information are different. The allowable error value of the actual color under different environmental information is set, that is, the judgment criteria. In other words, different color temperatures, different scene illuminations, and different dynamic range combinations have different corresponding judgment criteria. In addition, for different color feature dimensions, the judgment criteria under different environmental information are also different.

[0089] In the embodiment of the present application, there are objective judgment criteria for different color feature dimensions under different environmental information. In this way, the judgment criteria for each evaluation are objective and unified, which reduces unstable factors, and combined with the influence of environmental information, the judgment criteria are more accurate, thereby improving the accuracy of color evaluation.

[0090] In the embodiment of the present application, after determining the judgment standard corresponding to at least one of the color temperature, scene illumination and dynamic range in the environmental information in the first evaluation image or the second evaluation image, the electronic device performs color difference calculation for the color feature quantity of each color feature dimension to obtain the color difference of each color feature dimension. The color difference of each color feature dimension is then compared with the judgment standard under the environmental information. If the color difference is less than or equal to the judgment standard, the color evaluation result is determined to be color consistent in the color dimension; if the color difference is greater than the judgment standard, the color evaluation result is determined to be inconsistent in the color dimension.

[0091] When the color evaluation results on any color dimension are inconsistent, it can be considered that there is a color difference between the first evaluation image and the second evaluation image, that is, the colors are inconsistent. When the color evaluation results on any color dimension are consistent, it can be considered that there is no color difference between the first evaluation image and the second evaluation image, that is, the colors are consistent.

[0092] In the embodiment of the present application, the electronic device first obtains the first color feature quantity and the second color feature quantity from the first color buffer and the second color buffer, and then performs color analysis and color difference calculation on the first color feature quantity and the second color feature quantity. The color analysis can determine whether color evaluation is required based on the color proportion, and perform color calculation for situations where color evaluation is required.

[0093] In some embodiments of the present application, the first color feature quantity and the second color feature quantity are filtered to remove the non-evaluation part, and then the color difference between the remaining first color feature quantity and the remaining second color feature quantity is judged based on the judgment criteria corresponding to the environmental information to determine the color evaluation result.

[0094] It should be noted that in color evaluation, in order to remove the evaluation of colors with a relatively small proportion or colors that do not correspond, the first color feature quantity and the second color feature quantity can be filtered based on the corresponding color proportion in the color feature quantity, and the color evaluation can be performed after removing the color features with a small or no color proportion or that are incomparable and do not need to be compared. This can reduce the amount of calculation for color evaluation.

[0095] It should also be noted that the color difference calculation and color analysis here are performed on each identical color dimension in the first color feature value and the second color feature value, and not on the analysis and color difference calculation of the first evaluation image and the second evaluation image in different color dimensions.

[0096] It is understandable that, since the environmental information is combined and the judgment standard corresponding to the environmental information is adopted, the color consistency of different images is objectively judged, so the color evaluation is relatively stable. In addition, since the extraction of color features can also be considered from multiple dimensions, the scope and diversity of evaluation are expanded, so the accuracy will be improved in the process of color consistency evaluation.

[0097] It should be noted that the above-mentioned color analysis and color difference calculation can be performed in the color analysis and color difference calculation module, and then the color difference and environmental information are input into the judgment module, and the color difference between the first color feature quantity and the second color feature quantity is judged based on the judgment criteria corresponding to the environmental information in the judgment module to determine the color evaluation result.

[0098] In some embodiments of the present application, the electronic device can select the first image and the second image through the user interface to perform color evaluation on the first evaluation image and the second evaluation image, and obtain the color evaluation result in the form of a report to the user or inspection personnel.

[0099] It should be noted that the above report may include the color evaluation results of the first evaluation image and the second evaluation image, and may also include the color evaluation results on any color feature dimension, so as to facilitate subsequent targeted product adjustments and accurately solve the problem of color inconsistency.

[0100] In another embodiment of the present application, Figure 2 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 2 .like Figure 2 As shown, the method may include:

[0101] S201. Perform pure color detection on the first image or the second image to obtain a detection result; wherein the first evaluation image is obtained based on the first image, the second evaluation image is obtained based on the second image, and the first image and the second image are original images to be evaluated.

[0102] Before performing pure color detection on the first evaluation image and the second evaluation image, the first image or the second image, or the first image and the second image, can be detected to see whether they are pure colors. After the detection result comes out, pure color evaluation or non-pure color evaluation is performed based on whether it is pure color.

[0103] It should be noted that in the embodiment of the present application, the electronic device can determine the next evaluation method by performing pure color detection. The image for pure color detection can be any one of the two images to be evaluated, or both images can be detected. Since the two images are subsequently subjected to color evaluation for the same evaluation object. Therefore, when pure color detection is performed on an image, it can not only ensure the accuracy of the detection, but also reduce the complexity and amount of calculation.

[0104] In the embodiment of the present application, since the first image and the second image may be two images with different focal lengths or different fields of view, the electronic device may detect the image with a smaller field of view or a larger focal length.

[0105] In some embodiments of the present application, since the first field of view of the first image is smaller than the second field of view of the second image or the focal length of the first image is larger than the focal length of the second image, and they have the same evaluation object or shooting content, the electronic device can perform pure color detection on the first image. In this way, since the content of the first image is less than that of the second image, the efficiency of pure color detection on the first image will be higher, and the accuracy is also guaranteed.

[0106] Exemplarily, when a specific focal length corresponds to a first image and other images are evaluated together with the first image, the first image can be used as the object of pure color detection, which is not limited in the embodiments of the present application.

[0107] In some embodiments of the present application, the electronic device may divide the first image or the second image into multiple detection areas; determine the first color value of each detection area and the second color value of the central detection area; determine the statistical number that meets the pure color condition based on the first maximum color channel value in the first color value and the second maximum color channel value of the second color value; when the proportion of the statistical number is greater than a first threshold, determine that the detection result is a pure color; when the proportion of the statistical number is less than or equal to the first threshold, determine that the detection result is a non-pure color.

[0108] It should be noted that the electronic device divides the entire image to be detected (for example, the first image) into multiple ROI areas, namely detection areas. The color value of each detection area, namely the RGB value, is calculated respectively, so that the second color value of the central detection area and the first color value of the remaining detection areas can be obtained. The electronic device calculates the ratio of the first maximum color channel value in the first color value of each detection area to the second maximum color channel value of the second color value, and determines the first statistical value of each detection area. When the first statistical value meets the pure color condition, the statistical number is increased by 1, and after traversing each detection area, the final statistical number is determined. When the proportion of the statistical number in all detection areas is greater than the first threshold, the detection result is determined to be a pure color; when the proportion of the statistical number in all detection areas is less than or equal to the first threshold, the detection result is determined to be a non-pure color.

[0109] In the embodiment of the present application, the pure color condition includes: the first maximum color channel value and the second maximum color channel value belong to the same color channel, and / or the ratio of the first maximum color channel value to the second maximum color channel value satisfies the first range.

[0110] In some embodiments of the present application, it is determined that in each detection area, the first maximum color channel value and the second maximum color channel value belong to the same color channel, and / or the number of detection areas whose ratio of the first maximum color channel value to the second maximum color channel value is within a first range is determined to obtain a statistical number.

[0111] In the embodiment of the present application, the color value may include an RGB value, which is not limited in the embodiment of the present application.

[0112] In the embodiment of the present application, the RGB value includes three color channel values: R value, G value, and B value. In any detection area, the color value of each pixel in the area can be calculated. The electronic device can calculate the R, G, and B mean values ​​of each color channel of each pixel in the detection area, and then determine the first maximum color channel value from the mean values ​​of each color channel, and determine the largest second maximum color channel value from the R, G, and B mean values ​​of each color channel of the central detection area.

[0113] It can be understood that in the detection area, the overall consideration of the pixels can be determined through mean calculation, which is relatively comprehensive, and the overall color trend of the detection area can be reflected through the maximum color channel value in the mean, representing the color of a detection area, which is efficient and accurate.

[0114] In the embodiment of the present application, the first threshold represents the judgment value of color consistency. As a judgment of pure color detection, the more statistical numbers of areas with the same color channel, the greater the probability of representing a pure color. Therefore, the closer the statistical number is to the number of all detection areas, the greater the probability that the detection result is a pure color. The first threshold is the lower limit value for judging a pure color, and its setting principle is that the closer to 1, the more accurate it is. Exemplarily, the first threshold can be 95%, which is not limited in the embodiment of the present application.

[0115] In an embodiment of the present application, in the process of counting the number, the number can be counted by the first maximum color channel value being the same as the second maximum color channel value, the number can also be counted by the ratio of the first maximum color channel value to the second maximum color channel value satisfying the first range, or the number can be counted by satisfying both at the same time, so as to ensure that the pure color difference is within a controllable range, and the embodiment of the present application is not limited. Among them, the first range is used to limit or select the detection area whose color difference with the detection area at the center is not large or within the preset error for statistics, so as to ensure the accuracy of the statistics. The principle of the first range is that the more equal the two are, the better. Therefore, the setting of the first range can be within a range near 1, characterizing the color difference allowed to occur. Exemplarily, the first range can be (0.6, 1.3), and the embodiment of the present application is not limited.

[0116] It should be noted that in the embodiment of the present application, the RGB mean of the entire image can also be calculated, and its maximum color channel value is determined as the second maximum color channel value to calculate the ratio and count the number. In this way, the processing method of the color mean of the entire image represents the color trend of the entire image more accurately.

[0117] It should also be noted that the mean method is a method for calculating the color value of the entire region in the embodiment of the present application. It can also be calculated by other methods such as mean square error, and the embodiment of the present application is not limited.

[0118] For example, Figure 3 As shown, the electronic device can transfer the image (such as Figure 4 The image shown, such as the first image) is divided into 17x13 (or 34x26, which is not limited in the embodiment of the present application) ROI areas; the center ROI ( Figure 3 The average value of R, G, and B (the second color value) R c , G c , B c , and the R, G, B mean (first color value) Rij, Gij, Bij of each ROI in other ROIs; i, j are the number of rows and columns, for example, the mean of the ROI in the first row and the first column is R 11 ,G 11 ,B 11 . Determine the ratio Ratio i =Max(R i ,G i ,B i ) / Max(R c ,G c ,B c ), where Max(R i ,G i ,B i ) is the first maximum color channel value of any detection area, Max(R c ,G c ,B c ) is the second largest color channel value. For 17x13 ROIs, count the number N (statistical number) that meet the following pure color conditions, where N is a positive integer greater than or equal to 1:

[0119] Whether the maximum color channel of ROI_i (if R, i.e. the first maximum color channel value) is the same as the maximum color channel of ROI_c (if G, i.e. the second maximum color channel value), and / or Ratio i Is it between (0.6, 1.3), that is, the first range. If the same, and / or within (0.6, 1.3), then count +1, traverse each ROI, and determine the statistical number N. If N / (17x13)>95% (the first threshold), it is determined to be a pure color scene, that is, the detection result is pure color. Otherwise, it is a non-pure color scene, that is, the detection result is non-pure color. Among them, N / (17x13) is the proportion of the statistical number in all detection areas.

[0120] In another embodiment of the present application, for pure color detection, the first image or the second image can also be downsampled to obtain a sampled image; the hue calculation is performed on the sampled image, and the number of pixels in multiple hue intervals is counted to determine the largest number of first pixels; when the ratio of the first pixel number to the total number of pixels in the sampled image is greater than a second threshold, the detection result is determined to be a pure color; when the ratio of the first pixel number to the total number of pixels in the sampled image is less than or equal to the second threshold, the detection result is determined to be a non-pure color.

[0121] It should be noted that in pure color detection, the hue can be calculated to see whether the number of pixels concentrated in the hue interval is concentrated to determine whether it is a pure color scene. Among them, hue is the primary characteristic of color and the most accurate standard for distinguishing various colors. Hue is a form of color that can convey the basic tendency of color, such as red, blue, green, etc. Hue is determined by the light waves reflected or transmitted by the object, and different light wave lengths correspond to different hues.

[0122] In the embodiment of the present application, the angle value of the hue is in the range of [0,360] degrees.

[0123] In an embodiment of the present application, the electronic device can perform downsampling processing on the image to be detected, and then perform hue calculation on the sampled image after downsampling; or directly perform hue calculation and subsequent processing on the image to be detected, which is not limited in the embodiment of the present application. The image to be detected can be the first image or the second image, or the first image and the second image, which is not limited in the embodiment of the present application, and how to select the image to be detected is consistent with the implementation in the aforementioned implementation, which will not be repeated here.

[0124] In the embodiment of the present application, the hue is divided into multiple hue intervals according to the angle value of the hue, and each hue interval can be of the same range size or of different range sizes, which is not limited in the embodiment of the present application.

[0125] Exemplarily, the hue may be divided into intervals of 30 degrees or 60 degrees. A hue interval may be considered as a hue, and the interval division is within a reasonable range.

[0126] In an embodiment of the present application, the electronic device obtains the RGB value of each pixel of the image, calculates the angle value of the hue based on the RGB value of each pixel, and in each hue interval, counts the number of pixels with the angle value in each hue interval, and finds the first number of pixels with the largest number of pixels, that is, the pixel statistical peak. The electronic device determines whether it is a pure color based on the ratio of the first number of pixels to the total number of pixels in the sampled image. Since the first number of pixels is the largest in a hue interval, it can represent the main tendency of the color. If the first number of pixels does not reach the area ratio of the pure color, that is, the ratio of the first number of pixels to the total number of pixels in the sampled image is less than or equal to the second threshold, then it is characterized that the image to be detected is not a pure color, and the detection result is a non-pure color. On the contrary, if the first number of pixels reaches the area ratio of the pure color, that is, the ratio of the first number of pixels to the total number of pixels in the sampled image is greater than the second threshold, then it is characterized that the image to be detected is a pure color, and the detection result is a pure color.

[0127] Among them, the second threshold represents the judgment value of hue consistency. As a judgment of pure color detection, the more pixels in the interval where the hue is concentrated, the greater the probability of representing a pure color. Therefore, the closer the first pixel number is to the number of all pixels in the image, the greater the probability that the detection result is a pure color. The second threshold is the lower limit value for judging whether it is a pure color, and its setting principle is that the closer it is to 1, the more accurate it is. Exemplarily, the second threshold can be 95%, which is not limited in the embodiments of the present application.

[0128] In some embodiments of the present application, a hue peak value with the largest number of pixels in the hue interval may be found to obtain a first number of pixels.

[0129] It should be noted that in the embodiment of the present application, the electronic device calculates the angle value of the hue based on the RGB value of each pixel in the following process: normalize each color channel value in the RGB of the pixel respectively, and calculate the maximum color channel value V and the minimum color channel value m among the three normalized RGB values; determine the hue or the angle value of the hue based on the ratio of the difference between the other two color channels except the maximum color channel and Vm.

[0130] In other embodiments of the present application, the sampled image may be spatially converted to a color space such as HSV, LCH, and then H (hue) may be calculated, but the embodiments of the present application are not limited thereto.

[0131] For example, 0° or 360° represents red, 60° represents yellow, 120° represents green, 180° represents cyan, 240° represents blue, 300° represents magenta, and so on.

[0132] In some embodiments of the present application, in order to facilitate rapid image processing and improve processing efficiency, the image to be detected may be first blurred, so that the color is not greatly affected and the specific processing content is reduced, effectively improving the processing speed. The blurring of the image may include: Gaussian blurring, mean blurring, median blurring, etc., which are not limited in the embodiments of the present application.

[0133] For example, after blurring the image to be detected, downsampling is performed to obtain a sampled image, wherein the downsampling multiple may be 2-8 times, for example, 4 times, which is not limited in the present embodiment. Figure 5 As shown in the figure, according to the RGB value of the sampled image, the hue is calculated, the number of pixels in each hue interval is counted, and the hue with the largest number of pixels, namely Peak:H, is found. peak , calculate Peak:H peak The hue interval, H pL and H pR The number of pixels between pixel_H (first pixel number); determine the ratio of the first pixel number to the total number of pixels in the sampled image Rate = pixel_H / pixel_All, where the total number of pixels is pixel_All. If Rate>95% (second threshold), it is determined that the scene is a pure color scene. Otherwise, it is a non-pure color scene. Where H peak : The hue with the largest number of pixels (Peak). pL :H pL =H peak -30°; H pR :H pR =H peak +30°.

[0134] S202: When the detection result indicates a pure color, perform a pure color evaluation.

[0135] S203: When the detection result indicates a non-pure color, perform a non-pure color evaluation.

[0136] When an electronic device performs pure color detection on an image, the detection results are divided into two types: one is that the detection result represents a pure color, and the other is that the detection result represents a non-pure color.

[0137] In the embodiment of the present application, when the detection result represents a pure color, a pure color evaluation is performed; when the detection result represents a non-pure color, a non-pure color evaluation is performed.

[0138] It should be noted that in the embodiment of the present application, the implementation process of pure color evaluation and non-pure color evaluation is consistent with the principles of S101 to S104. The difference is that the color feature quantity extracted in the pure color evaluation process is of one color dimension, while in the non-pure color evaluation process, the color feature quantities are multiple and have multiple color dimensions.

[0139] In some embodiments of the present application, if the detection result is not a pure color, the RGB value can be output as empty, that is, return None, None, None. If the detection result is not a pure color, the RGB value can be calculated and output return Rave, Gave, Bave. Rave, Gave, Bave are the RGB average values ​​of the pixel.

[0140] For example, Figure 6 As shown, assuming that pure color detection is performed on the first image, the first image is a narrow field of view image (narrow FOV, if the detection result is pure color, it is fed back to the pure color evaluation module to perform the pure color evaluation process; if the detection result is non-pure color, it is fed back to the non-pure color evaluation module to perform the non-pure color evaluation process.

[0141] It is understandable that in the process of color evaluation, pure color detection is first performed to determine whether it is a pure color, and then multiple color feature dimensions are extracted and processed when evaluating non-pure colors. In the case of pure colors, a color feature quantity is directly extracted from the pure color to perform color analysis, calculation and judgment. Pure color detection can save the evaluation process for pure colors and improve evaluation efficiency.

[0142] It should be noted that, in the process of pure color detection, the average RGB value corresponding to the statistical number, or the average RGB value corresponding to the first number of pixels, can be directly calculated for the image to be detected. When the detection result is a pure color, the aforementioned average RGB value can be used as the color value in the color feature quantity after color feature extraction.

[0143] In some embodiments of the present application, Figure 7 As shown, the pure color evaluation module includes: an image registration module 10, a pure color extraction module 11, an environmental information extraction module (not shown in the figure), a color analysis and color difference calculation module 12, and a determination module 13; wherein,

[0144] The image registration module 10 is used to perform image registration on the second image with reference to the first image to obtain a second evaluation image.

[0145] The pure color extraction module 11 is used to perform color feature extraction (ie, pure color extraction) on the first evaluation image (ie, the first image) and the second evaluation image respectively to obtain a first color feature quantity and a second color feature quantity.

[0146] The environment information extraction module is used to extract the environment information of shooting the first evaluation image and / or the second evaluation image.

[0147] The color analysis and color difference calculation module 12 is used to determine the color difference between the first color feature quantity and the second color feature quantity; or, after filtering the first color feature quantity and the second color feature quantity and removing the non-evaluation part, determine the color difference between the remaining first color feature quantity and the remaining second color feature quantity.

[0148] The judgment module 13 is used to judge the color difference between the first color feature quantity and the second color feature quantity based on the judgment standard corresponding to the environmental information to determine the color evaluation result; or, based on the judgment standard corresponding to the environmental information, to judge the color difference between the remaining first color feature quantity and the remaining second color feature quantity to determine the color evaluation result.

[0149] In the embodiment of the present application, Figure 8 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 3 .like Figure 8 As shown in the figure, the pure color evaluation process includes:

[0150] S301 . Referring to the first image, perform image registration on the second image to obtain a second evaluation image.

[0151] S302 , extracting color features from the first evaluation image (ie, the first image) and the second evaluation image respectively to obtain a first color feature quantity and a second color feature quantity.

[0152] S303, determining the color difference between the first color feature and the second color feature; or, filtering the first color feature and the second color feature to remove the non-evaluation part, and then determining the color difference between the remaining first color feature and the remaining second color feature.

[0153] S304: Extracting environment information for shooting the first evaluation image and / or the second evaluation image.

[0154] S305. Based on the judgment standard corresponding to the environmental information, the color difference between the first color feature quantity and the second color feature quantity is judged to determine the color evaluation result; or, based on the judgment standard corresponding to the environmental information, the color difference between the remaining first color feature quantity and the remaining second color feature quantity is judged to determine the color evaluation result.

[0155] It can be understood that, since the environmental information is combined and the judgment standard corresponding to the environmental information is adopted, the color consistency of different images is objectively judged, and therefore, the color evaluation is relatively stable and accurate.

[0156] In another embodiment of the present application, at least one color feature is extracted from the first evaluation image to obtain a first color feature quantity, and color feature is extracted from the second evaluation image to obtain a second color feature quantity, that is, S102, combined with Figure 1 , Fig. 9 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 4 .like Fig. 9 As shown, the method may include:

[0157] S401 . In case of non-pure color evaluation, semantic segmentation is performed on the image to be evaluated to determine at least one mask image; one mask image corresponds to one scene; the image to be evaluated includes a first evaluation image or a second evaluation image.

[0158] It should be noted that, in the process of non-pure color evaluation, the image to be evaluated is non-pure color, so there are multiple scenes in the image to be evaluated. The electronic device can divide the scene by semantic segmentation, so as to determine at least one mask image. One mask image corresponds to one scene.

[0159] In an embodiment of the present application, at least one mask image includes at least: at least one of a gray area image, a sky image, a portrait image, a color card image, a main color image, a global image, a green plant image, and a clothing image.

[0160] That is to say, in the process of non-pure color evaluation, color evaluation is performed for different scenes, and the scenes may include but are not limited to: sky, green plants, portraits, clothing, main colors, color cards, etc.

[0161] In the embodiment of the present application, semantic segmentation can be implemented by processing using a semantic segmentation model, wherein the semantic segmentation model can be an artificial intelligence model, which is not limited in the embodiment of the present application.

[0162] For example, the semantic segmentation model can be predicted using the mask2Former model. The core of the semantic segmentation model is a Transformer Decoder, that is, the backbone network is connected to the pixel decoder, which can generate segmentation masks and corresponding category labels from a series of queries. Each query can be regarded as a potential segmentation object or scene. Fig.10As shown in the figure, for the mask2Former model, L is 9 layers, each layer can output class (output category) and mask (mask image), and each layer processes features of different scales, so as to capture different levels of detail in the image. Among them, Class: [1,200,150] ([B,C,cls]), Mask: [1,200,304,224] ([B,C,H,W]); the matrix multiplication of the two is [B,cls,H,W], there are 150 classes, and each class will have a mask. Use bilinear interpolation for upsampling to get: [1,150,4096,3072]; perform softmax normalization on the class, find the maximum index value on the class to get mask [1,4096,3072], different index values ​​in the mask represent different categories (for example, 2 represents the sky in segmentation; 9 represents green plants; 12 represents the human body; 0 represents the background), so that at least one category of mask images can be determined.

[0163] It should also be noted that when performing color evaluation of the first evaluation image and the second evaluation image, the principle of the color feature extraction process for each evaluation image is consistent. Therefore, the processing process of the image to be evaluated here is applicable to the extraction of the first color feature quantity and the extraction of the second color feature quantity of the first evaluation image and the second evaluation image.

[0164] S402, extracting color features from at least one mask image to obtain at least one color feature quantity; any color feature quantity includes: a first color feature quantity corresponding to the first evaluation image and a second color feature quantity corresponding to the second evaluation image.

[0165] In an embodiment of the present application, the electronic device may perform color feature extraction on each mask image to obtain a color feature quantity of each mask image. The electronic device performs color feature extraction on at least one mask image of the first evaluation image to obtain a first color feature quantity, and the electronic device performs color feature extraction on at least one mask image of the first evaluation image to obtain a second color feature quantity.

[0166] It should be noted that, in the embodiment of the present application, the color feature extraction for each mask image can be performed in a different extraction module, that is, the color feature extraction for each mask image can be performed in the extraction module corresponding to it.

[0167] In some embodiments of the present application, color feature extraction is performed on each of the at least one mask images to obtain at least one color feature quantity, including at least one of the following:

[0168] S501, extracting color features from the gray area image to obtain gray area color feature quantities.

[0169] In some embodiments of the present application, the gray area image is transformed into a first color space to determine the third color value of each pixel; based on the color temperature corresponding to the gray area image, a first polygonal range is determined from multiple polygonal ranges of chromaticity; the multiple polygonal ranges have a corresponding relationship with the color temperature; from the third color value of each pixel, a target pixel that satisfies the color value within the first polygonal range is determined; based on the color channel value of the target pixel, a gray area decolorization feature is determined.

[0170] It should be noted that the electronic device can perform color feature extraction in a mask image from which other scenes are removed, that is, in a gray area image.

[0171] It should also be noted that, at different color temperatures, the standard or brightness range of the gray area image color feature extraction is different, and its brightness range can be determined according to the color temperature obtained by the gray area image. In the embodiment of the present application, a polygonal range is used to represent its brightness range.

[0172] In an embodiment of the present application, there are corresponding relationships between brightness of multiple polygonal ranges or multiple brightness regions and color temperature. The electronic device can determine the corresponding brightness region, i.e., the first polygonal range, based on the color temperature of the gray area image. The polygonal range includes multiple brightness values. The electronic device obtains the brightness value of each pixel of the gray area image, and determines the pixel corresponding to the brightness value within the first polygonal range as the target pixel. The electronic device determines the gray area color feature based on the RGB value of the target pixel.

[0173] It should be noted that in the embodiment of the present application, the electronic device can first convert the gray area image into a color space from RGB or sRGB to Lab space, and then determine the brightness value of each pixel, that is, the third color value.

[0174] In detail, the electronic device normalizes sRGB and converts it to linear RGB through inverse gamma (inverse gamma correction), then converts the linear RGB to the XYZ color space, and finally converts the XYZ color from the XYZ color space to the Lab color space. Among them, sRGB normalization is to divide each color channel value in the RGB value by 255 and convert it to a decimal in the range of 0-1. Inverse gamma correction is applied to each color channel value to correct the nonlinear response of the display device to obtain the corrected R, G, B values, and then matrix multiplication is used to convert the RGB color from the linear RGB color space to the XYZ color space.

[0175] It should be noted that after the electronic device determines the target pixel, there may be multiple target pixels. The target pixel has a color value in the Lab space, and the target pixel is first converted from the Lab space to the RGB space to obtain the RGB value of the target pixel, and then the RGB of each target pixel is averaged to obtain an average RGB value, which is the gray area color feature.

[0176] In the embodiment of the present application, the gray area color feature quantity further includes: the proportion of the target pixel to the total pixels of the gray area image. The gray area color feature quantity is stored in the color buffer.

[0177] It should be noted that the process of converting LAB to sRGB (JPEG) or RGB involves multiple steps, mainly including the conversion from LAB to XYZ, the conversion from XYZ to linear RGB through gamma, the non-linear correction from linear RGB to sRGB, and the final encoding of sRGB data to JPEG image.

[0178] S502: Extract color features of the sky image to obtain sky color feature quantities.

[0179] It should be noted that the electronic device can extract color features from the mask image that removes other scenes, that is, the sky image.

[0180] In an embodiment of the present application, the electronic device can filter the sky image based on brightness, obtain the sky color feature by calculating the pixel ratio of each element in the sky after filtering, and calculate the average RGB value of the pixel area of ​​each element.

[0181] It should be noted that when extracting color features from sky images, the bluest and most saturated area can be considered as the representative color of the sky.

[0182] S503: Extract color features from the global image to obtain white balance feature quantities.

[0183] It should be noted that the electronic device can perform color feature extraction in a mask image from which other scenes are removed, that is, in a global image.

[0184] In some embodiments of the present application, color conversion is performed on the global image to obtain brightness values ​​of pixels of the global image; pixels whose brightness values ​​are greater than a third threshold are removed to obtain remaining pixels; and white balance feature quantities are obtained based on the color values ​​of the remaining pixels.

[0185] It should be noted that the electronic device calculates the brightness value of the pixel to remove the pixels that affect the color contrast when there is a large black area, thereby improving the difference in color comparison and further improving the accuracy of color comparison.

[0186] In the embodiment of the present application, the electronic device calculates the brightness value of each pixel of the global image, removes pixels whose brightness values ​​are greater than a third threshold value, obtains remaining pixels, and extracts color features from the remaining pixels to obtain white balance feature quantities.

[0187] It should be noted that in the embodiment of the present application, it is necessary to convert the global image in RGB space to XYZ color space, and then convert from XYZ to Lab to obtain the L value of each pixel, that is, the brightness value. The color feature extraction of the remaining pixels is to convert from Lab space to RGB space, calculate the average value of each color channel of the remaining pixels, and the proportion of the remaining pixels in the global image to obtain the white balance feature.

[0188] It should be noted that the third threshold is a gray area extraction range or a critical value that affects color difference. A value greater than the third threshold indicates that factors that affect color difference are reduced.

[0189] In some embodiments of the present application, before color features are extracted from a global image and white balance features are obtained, brightness alignment is performed on the first evaluation image and the second evaluation image based on the first evaluation image to determine a global image of the aligned first evaluation image and a global image of the second evaluation image.

[0190] It should be noted that when the electronic device extracts color features of a global image, it needs to first perform histogram matching and brightness alignment on the two images to be evaluated before performing color feature extraction.

[0191] Exemplarily, 1. The two images to be evaluated (Img_s and Img_m) are compared and histogram matched with Img_m as a reference, and the matched Img_s is Img_s'.

[0192] 2. Brightness alignment of Img_s' and Img_m. Alignment method: Convert RGB to XYZ and calculate the Y value of the whole image.

[0193] 3. Calculate the brightness of Img_s' and Img_m: RGB->XYZ->Lab. Calculate the average values ​​of Rs, Gs, Bs and Rm, Gm, Bm except for pixels with L>8.

[0194] In some embodiments of the present application, the gray area color feature quantity and the white balance feature quantity may also be obtained by directly performing color feature extraction on the image to be evaluated, and the extraction principle is consistent with the above.

[0195] S504: extract color features from the portrait image to obtain facial skin color feature quantities.

[0196] It should be noted that the electronic device can extract color features from the mask image from which other scenes are removed, that is, the portrait image.

[0197] In some embodiments of the present application, facial skin color is extracted from a portrait image to obtain a first sub-image; variance adaptation is performed on the first sub-image to determine a first brightness threshold and a second brightness threshold; based on the first brightness threshold and the second brightness threshold, at least two brightness areas of the first sub-image are determined; based on the color channel values ​​of the at least two brightness areas, a facial skin color feature value is determined.

[0198] In an embodiment of the present application, when a face is recognized in the image to be evaluated, the electronic device first extracts the skin color of the face. Then, two thresholds, namely the first brightness threshold and the second brightness threshold, are found by adaptively using the histogram variance, and the skin area of ​​the face is divided into at least two brightness areas, such as highlight, medium gray and low dark areas, etc., and the brightness of the highlight, medium gray and low dark areas decreases in sequence. The facial skin color feature quantity is determined based on the average value of the color channel value of at least two brightness areas, and the pixel proportion. That is, the RGB average value of all, highlight, medium gray and low dark areas in the facial skin color, as well as the pixel proportion, are calculated respectively, and multiple facial skin color feature quantities are obtained, and they are all stored in the corresponding color buffer.

[0199] It should be noted that in the embodiment of the present application, after the electronic device extracts the skin color of the face to obtain the first sub-image, the first sub-image can also be filtered to remove occlusions or parts such as hair and eyes to improve the progress of the face skin color. For example, the first sub-image can be converted to HSV, and a threshold can be set to filter out hair, glasses, etc., which is not limited in the embodiment of the present application.

[0200] Exemplarily, assuming that the first brightness threshold is th1 and the second brightness threshold is th2, traverse from th1 in [0,254] ​​and th2 in [th1,254], divide the histogram of the first sub-image into three areas, calculate the mean and variance respectively, and use the ratio of between-group variance (difference between different groups) and within-group variance (difference of data within a group) to reflect the degree of distinction. The principle is that the greater the difference between groups, the smaller the difference within the group, so the larger the ratio, the better.

[0201] In some embodiments of the present application, facial skin color is extracted from a portrait image to obtain a first sub-image, including: extracting facial key points from the portrait image to obtain a second sub-image; and removing facial key points from the second sub-image to obtain a first sub-image.

[0202] In an embodiment of the present application, the electronic device extracts the original mask image of the face area in the semantic segmentation, that is, the face image, detects the face image, extracts the face key points, obtains the second sub-image, that is, crops the detected face, and extracts the coordinates of the face key points (eyes, eyebrows, lips, face skin). Construct a convex polygon for the face key points of the second sub-image, fill it with the fourth color value, remove the sky area, and then fill the remaining area with the fifth color value to obtain the first sub-image. The fourth color value is different from the fifth color value.

[0203] Exemplarily, convex polygons are constructed for the key points of the eyes, eyebrows, lips and facial skin respectively, and then the convex polygons constructed by the eyes, eyebrows and lips are filled with (0, 0, 0), the eyes, eyebrows and lips areas in the face are removed, and the convex polygons constructed by the facial skin are filled with (255, 255, 255) to retain the facial skin area.

[0204] In an embodiment of the present application, there may be multiple faces, and the skin color of each face may be extracted, and the final face skin color feature value may be stored in the corresponding color buffer. Alternatively, only the face skin color feature value with the largest feature value may be retained for storage, and this embodiment of the present application does not impose any restrictions.

[0205] It should be noted that in a scene with multiple people, the mask of the person with the largest and most positive face can be taken, that is, the face image Mask person.

[0206] S505: Extract color features from the color card image to obtain color card area feature quantities.

[0207] It should be noted that the electronic device can perform color feature extraction in the mask image (CC24_Mask) from which other scenes are removed, that is, in the color card image.

[0208] In an embodiment of the present application, the electronic device uses a CC24 color card to extract RGB values ​​(24 color block RGB values) and pixel ratios of color features, thereby obtaining a color card area feature value.

[0209] S506: Extract color features from the main color image to obtain main color feature quantities.

[0210] It should be noted that the electronic device can perform color feature extraction in the mask image from which other scenes are removed, that is, in the main color image.

[0211] In the embodiment of the present application, the electronic device extracts the RGB value and pixel ratio of the main color of the part of the image to be evaluated after filtering out other scenes, thereby obtaining the main color feature quantity. In this way, the background color outside the specific scene will not be missed, and the accuracy of color evaluation is improved.

[0212] Among them, the main color can be understood as the color that occupies a relatively large area.

[0213] It should be noted that the RGB values ​​or main color features of the first few main colors with the largest proportion can be stored in the color buffer, for example, the RGB values ​​of the top 3 main colors, etc., which is not limited in the embodiment of the present application.

[0214] S507: Extract color features from the clothing image to obtain clothing color feature quantities.

[0215] It should be noted that the electronic device can extract color features from the mask image that removes other scenes, that is, the clothing image.

[0216] In an embodiment of the present application, the electronic device performs color feature extraction on the clothing image, namely, RGB value and pixel ratio, to obtain clothing color feature quantity.

[0217] It should be noted that, it is also possible to consider taking the average color of each same color area in the first few (for example, 3) main color areas of the clothing as a representative for comparison.

[0218] S508: Extract color features of the green plant image to obtain green plant color feature quantities.

[0219] It should be noted that the electronic device can perform color feature extraction in the mask image that removes other scenes, that is, the green plant image. Among them, in the mask image (Mask_greens) of green plant colors, that is, in the green plant image, it is not necessary to distinguish between green plants of different green colors that appear simultaneously, and this embodiment of the application is not limited thereto.

[0220] In an embodiment of the present application, the electronic device performs color feature extraction on the green plant image to obtain the RGB value and pixel ratio, thereby obtaining the green plant color feature quantity.

[0221] In an embodiment of the present application, color feature extraction of a green plant image can extract green plants of different saturations, and finally filter to obtain an average RGB value and pixel ratio, thereby obtaining a green plant color feature value.

[0222] It should be noted that the brightness processing in the embodiments of the present application can be extracted after color space conversion, and the specific converted color space is not limited in the embodiments of the present application.

[0223] It should be noted that the representative color of green plants can be considered to be the average color for comparison.

[0224] In some embodiments of the present application, for example, Fig.11As shown, the non-pure color evaluation module includes: an image registration module 20, a first semantic segmentation module 21, a first extraction module 22 (including at least one extraction module, for example, a gray area color extraction module, a global extraction module, a color card color extraction module, a face skin color extraction module, a green plant color extraction module, a sky color extraction module, a main color extraction module, a clothing extraction module, etc.), a first color buffer 23, an environmental information extraction module 24, a color analysis and color difference calculation module 25, a determination module 26, and a second extraction module (including at least one extraction module, for example, a gray area color extraction module, a global extraction module, a color card color extraction module, a face skin color extraction module, a green plant color extraction module, a sky color extraction module, a main color extraction module, a clothing extraction module, etc.) (not shown in the figure, and its connection relationship is the same as the first extraction module) a second semantic segmentation module (not shown in the figure, used to perform semantic segmentation on the first evaluation image), and a second color buffer 27; wherein,

[0225] The image registration module 20 is used to perform image registration on the second image with reference to the first image to obtain a second evaluation image, wherein the first image is a narrow field of view image (Img (narrow FOV), the first evaluation image is the first image, and the second image is a wide field of view image (wide FOV). Both the first image and the second image are in JPG format.

[0226] The first semantic segmentation module 21 is used to perform semantic segmentation on the first evaluation image to obtain at least one first mask image (MASK).

[0227] The first extraction module 22 is used to extract color features from at least one first mask image to obtain a first color feature quantity, wherein the first color feature quantity includes the color feature quantity of at least one extraction module.

[0228] The first color buffer 23 is used to store the first color feature value.

[0229] The environment information extraction module 24 is used to extract environment information of shooting the first evaluation image and / or the second evaluation image.

[0230] a second semantic segmentation module, configured to perform semantic segmentation on the second evaluation image to obtain at least one second mask image;

[0231] The second extraction module is used to extract color features from at least one second mask image to obtain a second color feature quantity, wherein the second color feature quantity includes the color feature quantity of at least one extraction module, and has the same module function as the first extraction module.

[0232] The second color buffer 27 is used to store the second color feature value.

[0233] The color analysis and color difference calculation module 25 is used to obtain each first color feature quantity and the second color feature quantity from the first color buffer 23 and the second color buffer 27, and determine the color difference between each first color feature quantity and the second color feature quantity; or, after filtering each first color feature quantity and the second color feature quantity, removing the non-evaluation part, determine the color difference between each remaining first color feature quantity and the remaining second color feature quantity.

[0234] The judgment module 26 is used to judge the color difference between each first color feature quantity and the second color feature quantity based on the judgment standard corresponding to the environmental information to determine the color evaluation result; or, based on the judgment standard corresponding to the environmental information, to judge the color difference between each remaining first color feature quantity and the remaining second color feature quantity to determine the color evaluation result.

[0235] In the embodiment of the present application, the color temperature, scene illumination, and dynamic range in the environmental information are used to classify the judgment criteria to determine whether the colors of the two evaluation images are consistent.

[0236] It is understandable that, since the environmental information is combined and the judgment standard corresponding to the environmental information is adopted, the color consistency of different images is objectively judged, so the color evaluation is relatively stable. In addition, since the extraction of color features can also be considered from multiple dimensions, the scope and diversity of evaluation are expanded, so the accuracy will be improved in the process of color consistency evaluation.

[0237] In yet another embodiment of the present application, based on the same inventive concept as the above-mentioned embodiment, Fig.12 The following is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Fig.12 As shown, the image processing device 30 may include:

[0238] A registration unit 31 is used to determine a first evaluation image and a second evaluation image having the same evaluation object, wherein the first evaluation image and the second evaluation image are determined based on images obtained by shooting the same content;

[0239] The extraction unit 32 is used to extract color features from the first evaluation image to obtain a first color feature value, and to extract at least one color feature from the second evaluation image to obtain a second color feature value; and to extract environment information in which the first evaluation image and / or the second evaluation image are taken;

[0240] The determination unit 33 is configured to determine the color difference between the first color feature quantity and the second color feature quantity based on a determination standard corresponding to the environmental information, and determine a color evaluation result.

[0241] In some embodiments of the present application, the image processing device 30 may further include: a color analysis and color difference determination unit 34;

[0242] The extraction unit 32 is further used to determine a corresponding judgment standard based on at least one of the color temperature, scene illumination and dynamic range in the environmental information; the judgment standard represents an allowable error range;

[0243] A color analysis and color difference determination unit 34, configured to determine the color difference between the first color feature quantity and the second color feature quantity;

[0244] The determination unit 33 is further configured to compare the color difference between the first color feature quantity and the second color feature quantity with the determination criterion to determine the color evaluation result.

[0245] In some embodiments of the present application, the image processing device 30 may further include: a semantic segmentation unit 35;

[0246] The semantic segmentation unit 35 is used to perform semantic segmentation on the image to be evaluated in the case of non-pure color evaluation, and determine at least one mask image; one mask image corresponds to one scene; the image to be evaluated includes the first evaluation image or the second evaluation image;

[0247] The extraction unit 32 is further used to extract color features from each of the at least one mask image to obtain at least one color feature quantity; any color feature quantity includes: a first color feature quantity corresponding to the first evaluation image and a second color feature quantity corresponding to the second evaluation image.

[0248] In some embodiments of the present application, the at least one mask image includes at least: at least one of a gray area image, a sky image, a portrait image, a color card image, a main color image, a global image, a clothing image, and a green plant image;

[0249] The extraction unit 32 is further configured to:

[0250] Extracting color features from the gray area image to obtain gray area color feature quantities;

[0251] Extracting color features from the sky image to obtain sky color feature quantities;

[0252] Extracting color features from the global image to obtain white balance features;

[0253] Extracting color features from the portrait image to obtain facial skin color feature quantities;

[0254] Extracting color features from the color card image to obtain color card area feature quantities;

[0255] Extracting color features from the main color image to obtain main color feature quantities;

[0256] Extracting color features from the clothing image to obtain clothing color feature quantities;

[0257] Color features are extracted from the green plant image to obtain green plant color feature quantities.

[0258] In some embodiments of the present application, the image processing device 30 may further include: a color buffer unit 36, used to store the first color feature quantity in a first color buffer and store the second color feature quantity in a second color buffer, wherein the first color feature quantity and the second color feature quantity both include a color value and its color proportion in the entire image.

[0259] In some embodiments of the present application, the image processing device 30 may further include: a detection unit 37,

[0260] The detection unit 37 is used to perform pure color detection on the first image or the second image to obtain a detection result; wherein the first evaluation image is obtained based on the first image, the second evaluation image is obtained based on the second image, and the first image and the second image are original images to be evaluated; when the detection result represents a pure color, a pure color evaluation is performed; when the detection result represents a non-pure color, a non-pure color evaluation is performed.

[0261] In some embodiments of the present application, the detection unit 37 is also used to divide the first image or the second image into multiple detection areas; determine the first color value of each detection area and the second color value of the central detection area; determine the statistical number that meets the pure color condition based on the first maximum color channel value in the first color value and the second maximum color channel value of the second color value; when the proportion of the statistical number is greater than a first threshold, determine that the detection result is a pure color; when the proportion of the statistical number is less than or equal to the first threshold, determine that the detection result is a non-pure color.

[0262] In some embodiments of the present application, the detection unit 37 is further used to determine that in each detection area, the first maximum color channel value and the second maximum color channel value belong to the same color channel, and / or determine the number of detection areas where the ratio of the first maximum color channel value to the second maximum color channel value is within a first range, to obtain the statistical number;

[0263] The pure color condition includes: the first maximum color channel value and the second maximum color channel value belong to the same color channel, and / or the ratio of the first maximum color channel value to the second maximum color channel value is within a first range.

[0264] In some embodiments of the present application, the detection unit 37 is further configured to downsample the first image or the second image to obtain a sampled image;

[0265] Calculating the hue of the sampled image, counting the number of pixels in a plurality of hue intervals, and thereby determining the largest first number of pixels;

[0266] When the ratio of the first number of pixels to the total number of pixels in the sampled image is greater than a second threshold, determining that the detection result is a pure color;

[0267] When the ratio of the first number of pixels to the total number of pixels in the sampled image is less than or equal to a second threshold, the detection result is determined to be a non-pure color.

[0268] In some embodiments of the present application, the extraction unit 32 is further configured to perform color conversion on the global image to obtain brightness values ​​of pixels of the global image;

[0269] Removing pixels whose brightness values ​​are greater than a third threshold value from among the pixels to obtain remaining pixels;

[0270] The white balance feature is obtained based on the color values ​​of the remaining pixels.

[0271] In some embodiments of the present application, the registration unit 31 is also used to extract color features from the global image, and before obtaining the white balance feature quantity, perform brightness alignment on the first evaluation image and the second evaluation image based on the first evaluation image, and determine the global image of the aligned first evaluation image and the global image of the second evaluation image.

[0272] In some embodiments of the present application, the extraction unit 32 is further used to transform the gray area image into the first color space and determine the third color value of each pixel;

[0273] Based on the color temperature corresponding to the gray area image, determining a first polygon range from a plurality of polygon ranges of chromaticity; the plurality of polygon ranges have a corresponding relationship with the color temperature;

[0274] Determine a target pixel that satisfies a color value within the first polygon from the third color values ​​of each pixel;

[0275] The gray area decolorization feature is determined based on the color channel value of the target pixel.

[0276] In some embodiments of the present application, the extraction unit 32 is further configured to extract the skin color of the face from the portrait image to obtain a first sub-image;

[0277] Performing variance adaptation on the first sub-image to determine a first brightness threshold and a second brightness threshold;

[0278] Determining at least two brightness areas of the first sub-image based on the first brightness threshold and the second brightness threshold;

[0279] The facial skin color feature is determined based on the color channel values ​​of the at least two brightness areas.

[0280] In some embodiments of the present application, the extraction unit 32 is further configured to extract facial key points from the portrait image to obtain a second sub-image;

[0281] The facial key points are removed from the second sub-image to obtain the first sub-image.

[0282] In some embodiments of the present application, the registration unit 32 is further used to perform image registration with the second image based on the first image to obtain the first evaluation image and the second evaluation image;

[0283] Among them, the first field of view of the first image is smaller than the second field of view of the second image or the focal length of the first image is larger than the focal length of the second image, and they have the same evaluation object or shooting content.

[0284] In some embodiments of the present application, the color analysis and color difference determination unit 34 is also used to perform color feature extraction on the first evaluation image to obtain a first color feature quantity, and to perform color feature extraction on the second evaluation image to obtain a second color feature quantity, and then filter the first color feature quantity and the second color feature quantity to remove the non-evaluation part, and then determine the color difference between the remaining first color feature quantity and the remaining second color feature quantity based on the determination criteria corresponding to the environmental information by the determination unit 33 to determine the color evaluation result.

[0285] It is understandable that, since the environmental information is combined and the judgment standard corresponding to the environmental information is adopted, the color consistency of different images is objectively judged, so the color evaluation is relatively stable. In addition, since the extraction of color features can also be considered from multiple dimensions, the scope and diversity of evaluation are expanded, so the accuracy will be improved in the process of color consistency evaluation.

[0286] In the embodiments of the present application, a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc. It may also be a module or a non-modular one. Moreover, the components in the present embodiment may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional module.

[0287] In addition, those skilled in the art should understand that the relevant description of the above-mentioned image processing device in the embodiment of the present application can be understood by referring to the relevant description of the image processing method in the embodiment of the present application.

[0288] In yet another embodiment of the present application, Fig.13 The following is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.13 As shown, the electronic device 120 includes a processor 1201 and a memory 1202. The memory 1202 can store a computer program, and the processor 1201 can call and run the computer program from the memory 1202 to implement the method in the embodiment of the present application.

[0289] The memory 1202 may be a separate device independent of the processor 1201 , or may be integrated into the processor 1201 .

[0290] In some embodiments, Fig.13 As shown, the electronic device 120 may further include a transceiver 1203, and the processor 1201 may control the transceiver 1203 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices.

[0291] The transceiver 1203 may include a transmitter and a receiver. The transceiver 1203 may further include an antenna, and the number of antennas may be one or more.

[0292] In some embodiments, the present application also provides another electronic device composition, wherein the electronic device may include the image processing device 30 described in any one of the aforementioned embodiments.

[0293] Fig.14 The schematic diagram of the structure of a chip provided in the embodiment of the present application is shown in FIG. Fig.14 As shown, the chip 130 includes a processor 1301, and the processor 1301 can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0294] In some embodiments, Fig.14As shown, the chip 130 may further include a memory 1302. The processor 1301 may call and run a computer program from the memory 1302 to implement the method in the embodiment of the present application.

[0295] The memory 1302 may be a separate device independent of the processor 1301 , or may be integrated into the processor 1301 .

[0296] In some embodiments, the chip 130 may further include an input interface 1303. The processor 1301 may control the input interface 1303 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0297] In some embodiments, the chip 130 may further include an output interface 1304. The processor 1301 may control the output interface 1304 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0298] In some embodiments, the chip can be applied to the electronic devices in the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0299] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0300] It can be understood that the processor of the embodiment of the present application may be an integrated circuit chip with information processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0301] It can also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0302] It can also be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application can also be static random access memory (Static RAM, SRAM), dynamic random access memory (DynamicRAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Dynch Link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0303] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0304] In some embodiments, the computer-readable storage medium can be applied to the electronic device in the embodiments of the present application, and when the computer program is executed by at least one processor, it implements the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0305] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0306] In some embodiments, the computer program product can be applied to the electronic device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0307] The embodiment of the present application also provides a computer program.

[0308] In some embodiments, the computer program can be applied to the electronic devices in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the electronic devices in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0309] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0310] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0311] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0312] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0313] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0314] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0315] It should be noted that, in this application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0316] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0317] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0318] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0319] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0320] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Determining a first evaluation image and a second evaluation image having the same evaluation object, wherein the first evaluation image and the second evaluation image are determined based on images obtained by photographing the same photographing content; Extracting at least one color feature from the first evaluation image to obtain a first color feature value, and extracting color features from the second evaluation image to obtain a second color feature value; Extracting environment information for taking the first evaluation image and / or the second evaluation image; Based on a determination criterion corresponding to the environmental information, a color difference between the first color feature quantity and the second color feature quantity is determined to determine a color evaluation result.

2. The method according to claim 1, characterized in that The step of determining the color difference between the first color feature quantity and the second color feature quantity based on a determination standard corresponding to the environmental information to determine a color evaluation result includes: Determine a corresponding judgment standard based on at least one of the color temperature, scene illumination and dynamic range in the environmental information; the judgment standard represents an allowable error range; determining a color difference between the first color feature quantity and the second color feature quantity; The color difference between the first color feature quantity and the second color feature quantity is compared with the determination standard to determine the color evaluation result.

3. The method according to claim 1 or 2, characterized in that: The extracting at least one color feature from the first evaluation image to obtain a first color feature quantity, and extracting color features from the second evaluation image to obtain a second color feature quantity, include: In the case of non-pure color evaluation, semantic segmentation is performed on the image to be evaluated to determine at least one mask image; one mask image corresponds to one scene; the image to be evaluated includes the first evaluation image or the second evaluation image; Color features are extracted from each of the at least one mask image to obtain at least one color feature quantity; any color feature quantity includes: a first color feature quantity corresponding to the first evaluation image and a second color feature quantity corresponding to the second evaluation image.

4. The method according to claim 3, characterized in that The at least one mask image includes at least one of a gray area image, a sky image, a portrait image, a color card image, a main color image, a global image, a clothing image, and a green plant image; The color feature extraction is performed on each of the at least one mask image to obtain at least one color feature quantity, including at least one of the following: Extracting color features from the gray area image to obtain gray area color feature quantities; Extracting color features from the sky image to obtain sky color feature quantities; Extracting color features from the global image to obtain white balance features; Extracting color features from the portrait image to obtain facial skin color feature quantities; Extracting color features from the color card image to obtain color card area feature quantities; Extracting color features from the main color image to obtain main color feature quantities; Extracting color features of the clothing image to obtain clothing color feature quantities; Color features are extracted from the green plant image to obtain green plant color feature quantities.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The first color feature quantity is stored in a first color buffer, and the second color feature quantity is stored in a second color buffer, wherein the first color feature quantity and the second color feature quantity both include a color value and its color proportion in the entire image.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Performing pure color detection on the first image or the second image to obtain a detection result; wherein the first evaluation image is obtained based on the first image, the second evaluation image is obtained based on the second image, and the first image and the second image are original images to be evaluated; In the case where the detection result represents a pure color, performing a pure color evaluation; In the case where the detection result indicates a non-pure color, a non-pure color evaluation is performed.

7. The method according to claim 6, characterized in that The performing pure color detection on the first image or the second image to obtain a detection result includes: Dividing the first image or the second image into a plurality of detection areas; Determine a first color value of each detection area and a second color value of a central detection area; Determining a statistical number that satisfies a pure color condition based on a first maximum color channel value in the first color value and a second maximum color channel value in the second color value; When the proportion of the statistical quantity is greater than a first threshold, determining that the detection result is a pure color; When the proportion of the statistical quantity is less than or equal to the first threshold, it is determined that the detection result is a non-pure color.

8. The method according to claim 7, characterized in that The determining, based on a first maximum color channel value in the first color value and a second maximum color channel value in the second color value, a statistical quantity satisfying a pure color condition comprises: Determine in each detection area that the first maximum color channel value and the second maximum color channel value belong to the same color channel, and / or determine the number of detection areas where the ratio of the first maximum color channel value to the second maximum color channel value is within a first range, to obtain the statistical number; The pure color condition includes: the first maximum color channel value and the second maximum color channel value belong to the same color channel, and / or the ratio of the first maximum color channel value to the second maximum color channel value is within a first range.

9. The method according to claim 6, characterized in that The performing pure color detection on the first image or the second image to obtain a detection result includes: Downsampling the first image or the second image to obtain a sampled image; Calculating the hue of the sampled image, counting the number of pixels in a plurality of hue intervals, and thereby determining the largest first number of pixels; When the ratio of the first number of pixels to the total number of pixels in the sampled image is greater than a second threshold, determining that the detection result is a pure color; When the ratio of the first number of pixels to the total number of pixels in the sampled image is less than or equal to a second threshold, the detection result is determined to be a non-pure color.

10. The method according to claim 4, characterized in that Color feature extraction is performed on the global image to obtain white balance feature quantities, including: Performing color conversion on the global image to obtain brightness values ​​of pixels of the global image; Remove pixels whose brightness values ​​are greater than a third threshold value from among the pixels to obtain remaining pixels; The white balance feature is obtained based on the color values ​​of the remaining pixels.

11. The method according to claim 10, characterized in that Before extracting color features from the global image to obtain white balance feature quantities, the method further includes: The first evaluation image is used as a reference, brightness alignment is performed on the first evaluation image and the second evaluation image, and a global image of the aligned first evaluation image and a global image of the aligned second evaluation image are determined.

12. The method according to claim 4, characterized in that The step of extracting color features from the gray area image to obtain gray area color feature quantities includes: Transforming the gray area image into a first color space, and determining a third color value of each pixel; Based on the color temperature corresponding to the gray area image, determining a first polygon range from a plurality of polygon ranges of chromaticity; the plurality of polygon ranges have a corresponding relationship with the color temperature; Determine a target pixel that satisfies a color value within the first polygon from the third color values ​​of each pixel; The gray area decolorization feature is determined based on the color channel value of the target pixel.

13. The method according to claim 4, characterized in that The step of extracting color features from the portrait image to obtain facial skin color feature values ​​includes: Extracting the skin color of the face from the portrait image to obtain a first sub-image; Performing variance adaptation on the first sub-image to determine a first brightness threshold and a second brightness threshold; Determining at least two brightness areas of the first sub-image based on the first brightness threshold and the second brightness threshold; The facial skin color feature is determined based on the color channel values ​​of the at least two brightness areas.

14. The method according to claim 13, characterized in that The step of extracting the skin color of the face from the portrait image to obtain a first sub-image includes: Extracting facial key points from the portrait image to obtain a second sub-image; The facial key points are removed from the second sub-image to obtain the first sub-image.

15. The method according to any one of claims 1 to 14, characterized in that The determining of the first evaluation image and the second evaluation image having the same evaluation object includes: Taking the first image as a reference, performing image registration with the second image to obtain the first evaluation image and the second evaluation image; Among them, the first field of view of the first image is smaller than the second field of view of the second image or the focal length of the first image is larger than the focal length of the second image, and they have the same evaluation object or shooting content.

16. The method according to any one of claims 1 to 15, characterized in that After extracting color features from the first evaluation image to obtain a first color feature value, and extracting color features from the second evaluation image to obtain a second color feature value, the method further includes: The first color feature quantity and the second color feature quantity are filtered to remove the non-evaluation part, and then the color difference between the remaining first color feature quantity and the remaining second color feature quantity is judged based on the judgment standard corresponding to the environmental information to determine the color evaluation result.

17. An image processing device, characterized in that: include: A registration unit, used to determine a first evaluation image and a second evaluation image having the same evaluation object, wherein the first evaluation image and the second evaluation image are determined based on images obtained by photographing the same photographing content; an extraction unit, configured to extract color features from the first evaluation image to obtain a first color feature quantity, and to extract at least one color feature from the second evaluation image to obtain a second color feature quantity; and to extract environment information in which the first evaluation image and the second evaluation image are taken; The determination unit is configured to determine the color difference between the first color feature quantity and the second color feature quantity based on a determination standard corresponding to the environmental information, and determine a color evaluation result.

18. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store a computer program that can be run on the processor; The processor is configured to execute the steps of the image processing method according to any one of claims 1 to 16 when running the computer program.

19. A chip, characterized in that: comprising a processor, wherein: The processor is used to call and run a computer program from a memory, so that a device equipped with the chip executes the steps of the image processing method according to any one of claims 1 to 16.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 16 are implemented.

21. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 16 are implemented.

Images