Image color cast grading method and apparatus, readable storage medium, and terminal device

Abstract

The application belongs to the technical field of image processing, and particularly relates to an image color cast grading method and device, a computer readable storage medium and a terminal device. The method comprises the following steps: obtaining a target image to be processed; extracting each local color cast reference image in the target image; wherein each local color cast reference image is an image of a specified reference object; extracting a global color cast reference image in the target image; wherein the global color cast reference image is an image of a global pixel block; extracting color cast features of each local color cast reference image and the global color cast reference image respectively; and determining a color cast level of the target image according to the color cast features. According to the application, the color cast features of the local color cast reference image and the global color cast reference image containing the specified reference object in the image can be used to determine the color cast rating of the image, so that the local color cast information and the global color cast information of the image can be fully utilized, and the severity of the color cast of the image can be more comprehensively and accurately evaluated.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to an image color cast grading method and device, a computer readable storage medium and a terminal device. BACKGROUND

[0002] In the process of image shooting, due to light or angle reasons, the image shot may easily have the problem of image color cast, and how to accurately evaluate the severity of image color cast has gradually evolved into a basic problem in the field of image processing.

[0003] However, the prior art usually evaluates the severity of image color cast based on global image information, resulting in low accuracy of the evaluation result. SUMMARY

[0004] Therefore, the embodiments of the present application provide an image color cast grading method, device, computer readable storage medium and terminal device to solve the problem of low accuracy of the image color cast grading method in the prior art.

[0005] The first aspect of the embodiments of the present application provides an image color cast grading method, which can include:

[0006] obtaining a target image to be processed;

[0007] extracting each local color cast reference image in the target image; wherein each local color cast reference image is an image of a specified reference object;

[0008] extracting a global color cast reference image in the target image; wherein the global color cast reference image is an image of a global pixel block;

[0009] extracting color cast features of each local color cast reference image and the global color cast reference image, respectively;

[0010] determining a color cast level of the target image according to the color cast features.

[0011] In a specific implementation manner of the first aspect, the extracting each local color cast reference image in the target image can include:

[0012] extracting reference objects from the target image by using a preset reference object extraction model to obtain each local reference object image of the target image; wherein the reference object extraction model is a pre-trained artificial intelligence model for extracting reference objects, and each local reference object image contains a specified reference object;

[0013] performing edge correction on each local reference object image to obtain each local color cast reference image of the target image.

[0014] In a specific implementation manner of the first aspect, the extracting the global color cast reference image in the target image can include:

[0015] performing superpixel segmentation on the target image by using a preset superpixel segmentation algorithm to obtain each pixel block of the target image;

[0016] selecting the global pixel block according to the area of each pixel block;

[0017] obtaining the global color cast reference image of the target image according to the global pixel block.

[0018] In a specific implementation manner of the first aspect, the extracting the color cast features of each local color cast reference image and the global color cast reference image can include:

[0019] extracting color cast features of a specified color cast reference image by using a preset specified color cast feature extraction model to obtain first color cast features of the specified color cast reference image; wherein the specified color cast feature extraction model is a color cast feature extraction model corresponding to the specified color cast reference image, and the specified color cast reference image is any one of the local color cast reference images and the global color cast reference image;

[0020] calculating second color cast features of the specified color cast reference image;

[0021] determining the color cast features of the specified color cast reference image according to the first color cast features and the second color cast features of the specified color cast reference image.

[0022] In a specific implementation manner of the first aspect, before the extracting the color cast features of the specified color cast reference image by using the preset specified color cast feature extraction model, the color cast feature extraction model is updated;

[0023] The updating process of the color cast feature extraction model can include:

[0024] obtaining a pre-stored candidate image;

[0025] extracting each local color cast reference image of the candidate image;

[0026] extracting a global color cast reference image of the candidate image;

[0027] determining whether a specified color cast reference image is a model updating image by using the color cast feature extraction model; wherein the color cast feature extraction model corresponds to the specified color cast reference image, and the specified color cast reference image is any one of the local color cast reference images and the global color cast reference image;

[0028] If the specified color cast reference image is a model update image, the specified color cast reference image is taken as a new training sample of the color cast feature extraction model;

[0029] The color cast feature extraction model is updated by using the new training sample.

[0030] In an implementation form of the first aspect, the determination process of the candidate image can include:

[0031] A preset number of historical images are acquired;

[0032] Structural similarities between the historical images are calculated to obtain similarity values of the historical images;

[0033] If a specified historical image has a similarity value less than a preset similarity threshold, the specified historical image is determined as a candidate image; wherein the specified historical image is any one of the historical images.

[0034] In an implementation form of the first aspect, the determination of the color cast level of the target image according to the color cast feature can include:

[0035] The color cast level of the target image is determined according to the color cast feature and a preset color cast level mapping relationship; wherein the color cast level mapping relationship is a mapping relationship between the color cast feature and the color cast level.

[0036] The second aspect of the embodiments of the present application provides an image color cast grading device, which can include:

[0037] A target image acquisition module is configured to acquire a target image to be processed;

[0038] A local color cast reference image extraction module is configured to extract each local color cast reference image in the target image; wherein each local color cast reference image is an image of a specified reference object;

[0039] A global color cast reference image extraction module is configured to extract a global color cast reference image in the target image; wherein the global color cast reference image is an image of a global pixel block;

[0040] A color cast feature extraction module is configured to extract color cast features of each local color cast reference image and the global color cast reference image respectively;

[0041] A color cast level determination module is configured to determine a color cast level of the target image according to the color cast features.

[0042] In an implementation form of the second aspect, the local color cast reference image extraction module can include:

[0043] The reference extraction unit is configured to extract references from the target image by using a preset reference extraction model to obtain each local reference image of the target image; the reference extraction model is a pre-trained artificial intelligence model for reference extraction, and each local reference image contains a specified reference;

[0044] The edge correction unit is configured to correct edges of each local reference image to obtain each local color cast reference image of the target image.

[0045] In an implementation form of the second aspect, the global color cast reference image extraction module can include:

[0046] The superpixel segmentation unit is configured to segment superpixels of the target image by using a preset superpixel segmentation algorithm to obtain each pixel block of the target image;

[0047] The global pixel block selection unit is configured to select the global pixel block according to areas of each pixel block;

[0048] The global color cast reference image determination unit is configured to obtain the global color cast reference image of the target image according to the global pixel block.

[0049] In an implementation form of the second aspect, the color cast feature extraction module can include:

[0050] The first color cast feature extraction unit is configured to extract color cast features of a specified color cast reference image by using a preset specified color cast feature extraction model to obtain first color cast features of the specified color cast reference image; the specified color cast feature extraction model is a color cast feature extraction model corresponding to the specified color cast reference image, and the specified color cast reference image is any one of the local color cast reference images and the global color cast reference image;

[0051] The second color cast feature calculation unit is configured to calculate second color cast features of the specified color cast reference image;

[0052] The color cast feature determination unit is configured to determine color cast features of the specified color cast reference image according to the first color cast features and the second color cast features of the specified color cast reference image.

[0053] In an implementation form of the second aspect, the image color cast grading device can further include:

[0054] The candidate image acquisition module is configured to acquire a pre-stored candidate image;

[0055] The first reference image extraction module is configured to extract each local color cast reference image of the candidate image;

[0056] a second reference image extraction module configured to extract a global color cast reference image of the candidate image;

[0057] a model update image determination module configured to determine whether a specified color cast reference image is a model update image by using the color cast feature extraction model, wherein the color cast feature extraction model corresponds to the specified color cast reference image, and the specified color cast reference image is any one of the local color cast reference images and the global color cast reference image;

[0058] a training sample determination module configured to, if the specified color cast reference image is the model update image, take the specified color cast reference image as a new training sample of the color cast feature extraction model;

[0059] a model update module configured to update the color cast feature extraction model by using the new training sample.

[0060] In an implementation form of the second aspect, the candidate image acquisition module can further include:

[0061] a historical image acquisition unit configured to acquire a preset number of historical images;

[0062] a structural similarity calculation unit configured to calculate structural similarities between the historical images to obtain similarity degree values of the historical images;

[0063] a candidate image determination unit configured to, if a similarity degree value of a specified historical image is less than a preset similarity degree threshold, determine the specified historical image as a candidate image, wherein the specified historical image is any one of the historical images.

[0064] In an implementation form of the second aspect, the color cast level determination module can include:

[0065] a color cast level determination unit configured to determine a color cast level of the target image according to the color cast feature and a preset color cast level mapping relationship, wherein the color cast level mapping relationship is a mapping relationship between color cast features and color cast levels.

[0066] A third aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of any of the image color cast grading methods.

[0067] A fourth aspect of the embodiments of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of any of the image color cast grading methods when executing the computer program.

[0068] A fifth aspect of the embodiments of the present application provides a computer program product, which, when running on a terminal device, causes the terminal device to perform the steps of any of the image color cast grading methods.

[0069] Compared with the prior art, the embodiments of the present application have the beneficial effects that: the embodiments of the present application can determine the color cast rating of the target image according to the color cast characteristics of the local color cast reference image and the global color cast reference image containing the specified reference object in the target image, thereby fully utilizing the local color cast information and the global color cast information of the image, achieving more comprehensive and accurate evaluation of the severity of the image color cast, and having strong ease of use and practicality. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0071] Figure 1 A schematic flow chart for the updating process of the color cast characteristic extraction model in the embodiments of the present application;

[0072] Figure 2 A schematic flow chart for determining the candidate image;

[0073] Figure 3 An embodiment flow chart of the image color cast grading method in the embodiments of the present application;

[0074] Figure 4 A schematic flow chart for extracting the local color cast reference image;

[0075] Figure 5 A schematic flow chart for extracting the color cast characteristics of the specified color cast reference image;

[0076] Figure 6 An embodiment structure diagram of the image color cast grading device in the embodiments of the present application;

[0077] Figure 7 A schematic block diagram of a terminal device in the embodiments of the present application. DETAILED DESCRIPTION

[0078] In order to make the purposes, characteristics and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the embodiments described below are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0079] It should be understood that the term "comprising" as used in the specification and the appended claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0080] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0081] It should be further understood that the term "and / or" as used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0082] As used in the present application specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [a described condition or event]" or "in response to detecting [a described condition or event]" depending on the context.

[0083] In addition, in the description of the present application, the terms "first", "second", "third" and the like are only used to distinguish description and cannot be understood as indicating or implying relative importance.

[0084] In the process of shooting an image, due to light or angle, it is easy to cause the problem of image color cast in the shot image, and how to accurately evaluate the severity of image color cast has gradually evolved into a basic problem in the field of image processing.

[0085] However, the prior art usually evaluates the severity of image color cast based on global image information, resulting in low accuracy of the evaluation result.

[0086] Therefore, the embodiment of the present application provides an image color cast grading method, device, computer readable storage medium and terminal equipment. Through the embodiment of the present application, the color cast characteristics of the local color cast reference image and the global color cast reference image containing the specified reference object in the target image are used to determine the color cast rating of the target image, so that the local color cast information and the global color cast information of the image are fully utilized, the severity of the image color cast is more comprehensively and accurately evaluated, and the method has strong ease of use and practicality.

[0087] It should be noted that the execution subject of the method of the present application is a terminal equipment, specifically, it can be a common computing device such as a desktop computer, a notebook computer, a palm computer, etc., or other computing devices.

[0088] In the embodiment of the present application, a plurality of color cast feature extraction models can be used to extract local color cast reference images and global color cast reference images from the target image, and the color cast rating of the target image is determined according to the color cast characteristics of the two types of color cast reference images obtained.

[0089] The color cast feature extraction model can be a pre-trained artificial intelligence model used for color cast feature extraction, which can include but is not limited to existing artificial intelligence models such as Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), etc. Preferably, SVM is used.

[0090] In actual use, the color cast feature extraction model can be continuously updated as the image data is continuously accumulated. Please refer to Figure 1 The updating process of the color cast feature extraction model in the embodiment of the present application can include the following steps:

[0091] Step S101, obtaining a pre-stored candidate image.

[0092] It can be understood that the pre-stored candidate image sequence I = [I1, I2,..., In] can be obtained from a pre-set memory module or a server. nwherein n is the number of candidate images, each element in the sequence represents a candidate image, and the sequence of candidate images is composed of candidate images selected from the historical images, and the subscript of each candidate image represents the order of the candidate image in the sequence of candidate images. If a candidate image is determined as a model update image, it indicates that the candidate image can be used for the update training of the color cast feature extraction model, so that the color cast feature extraction model can adapt to different application scenarios.

[0093] Specifically, referring to Figure 2 Step S101 can include the following process:

[0094] Step S1011, obtaining a preset number of historical images.

[0095] In the embodiments of the present application, the historical images can be obtained according to a preset acquisition time period and a preset acquisition time interval. The acquisition time period is positively correlated with the resource size of the camera acquisition device, and the acquisition time interval is negatively correlated with the resource size of the camera acquisition device. The specific values of the two can be set according to the actual resource situation of the camera acquisition device, which is not limited in the present application.

[0096] It should be understood that the historical images can be obtained according to the preset acquisition time period. For example, if the acquisition time period is 1 hour, the obtained object is the historical images taken in the past 1 hour. In actual application, the resources of different camera acquisition devices can be different. If the resource of the camera acquisition device is large, it indicates that the camera acquisition device can process more historical images. If the resource of the camera acquisition device is small, it indicates that the camera acquisition device can only process fewer historical images. Therefore, the specific value of the acquisition time period can be positively correlated with the resource size of the camera acquisition device. If the resource of the camera acquisition device is large, the acquisition time period can be set to a larger value. If the resource of the camera acquisition device is small, the acquisition time period can be set to a smaller value.

[0097] In addition, the historical images can also be obtained according to the preset acquisition time interval. For example, if the acquisition time interval is 5 seconds, the historical images obtained in the foregoing example can be sampled every 5 seconds. The specific value of the acquisition time interval can be negatively correlated with the resource of the camera acquisition device. If the resource of the camera acquisition device is large, it indicates that the camera acquisition device can process more historical images. Therefore, the acquisition time interval can be set to a smaller value to obtain historical images more frequently. If the resource of the camera acquisition device is small, it indicates that the camera acquisition device can only process fewer historical images. Therefore, the acquisition time interval can be set to a larger value to obtain historical images less frequently.

[0098] Step S1012: Calculate the structural similarity between each pair of historical images to obtain the similarity value between each pair of historical images.

[0099] It can be understood that there can be highly similar images in the obtained historical images, for example, several historical images taken at the same angle in high-speed continuous shooting can be highly similar to each other. In order to save computing resources, in the embodiment of the present application, highly similar historical images can be excluded.

[0100] Specifically, the similarity value between each pair of historical images can be obtained by calculating the structural similarity (Structure Similarity Index Measure, SSIM) between each pair of historical images.

[0101] It can be understood that in addition to calculating the SSIM between the historical images, the similarity between the historical images can also be calculated by any similarity calculation method in the prior art, which is not limited in the present application and can be set according to the actual situation.

[0102] In the embodiment of the present application, whether each historical image can be used as a candidate image can be determined by judging the similarity value between the historical image and other historical images. For the sake of description, the process of determining the candidate image in the embodiment of the present application will be described below by taking any one of the historical images (denoted as a specified historical image) as an example.

[0103] Step S1013: Judge whether the similarity value between the specified historical image and other historical images is less than a preset similarity threshold value.

[0104] In the embodiment of the present application, whether the specified historical image is a candidate image can be determined according to the similarity value between the specified historical image and other historical images. The other historical images are historical images recorded before the specified historical image.

[0105] Specifically, if the similarity value between the specified historical image and other historical images is greater than or equal to the preset similarity threshold value THD ssim , it can be determined that the specified historical image is a non-candidate image, at this time step S1014 can be executed, and if the similarity value between the specified historical image and other historical images is less than the preset similarity threshold value THD ssim , it can be determined that the specified historical image is a candidate image, at this time step S1015 and the subsequent steps can be executed.

[0106] Step S1014: Determine that the specified historical image is a non-candidate image.

[0107] In the embodiment of the present application, if the similarity value between the specified historical image and other historical images is greater than or equal to the preset similarity threshold THD ssim , it indicates that the specified historical image is highly similar to other historical images, and the specified historical image can be determined as a non-candidate image. At this time, the specified historical image can be excluded to save computing resources.

[0108] Step S1015, determining the specified historical image as a candidate image.

[0109] In the embodiment of the present application, if the similarity value between the specified historical image and other historical images is less than the preset similarity threshold THD ssim , the specified historical image can be determined as a candidate image.

[0110] It can be understood that each historical image can be traversed and judged by the above process. When the traversal is completed, the candidate image sequence I = [I1, I2, …, In] can be obtained. n

[0111] Step S102, extracting each local color cast reference image of the candidate image.

[0112] Each local color cast reference image is an image of a specified reference object.

[0113] It should be understood that the specified reference object is an object having a specific color in a specific scene, which can include but is not limited to sky, cloud, rock, road, tree, leaf, etc. The type and number of the specified reference object can be set according to the actual application scene, and the present application does not make a specific limitation thereon. Here, the sky, road and tree can be preferably selected as the three specified reference objects.

[0114] It can be understood that the specified reference object has reference significance for determining the color cast degree of the candidate image. Therefore, in the embodiment of the present application, each local color cast reference image can be determined according to the specified reference object.

[0115] Specifically, step S102 can include the following process:

[0116] Step S1021, performing reference object extraction on the candidate image by using a preset reference object extraction model to obtain each local reference object image of the candidate image.

[0117] In the embodiment of the present application, the reference object extraction model is a pre-trained artificial intelligence model for reference object extraction, which can be any common artificial intelligence model in the prior art. Here, the YOLO target detection model can be preferably used.​

[0118] It can be understood that the artificial intelligence model selected by the embodiments of the present application can be trained before the reference object extraction of the target image is performed for the first time to obtain a reference object extraction model, and then the reference object extraction model can be directly used when the reference object extraction of the target image is performed thereafter. Specifically, the images containing the corresponding reference objects can be collected according to the specified reference objects selected in the previous step, and a training sample set is formed, and then the selected artificial intelligence model is trained according to the training sample set to obtain the reference object extraction model.

[0119] In the embodiments of the present application, the reference object extraction model is used to perform reference object extraction on the candidate image, and each local reference object image of the candidate image can be detected and segmented, wherein each local reference object image contains a specified reference object, that is, a local reference object image containing the sky, a local reference object image containing the road and a local reference object image containing the tree can be obtained.

[0120] Step S1022, edge correction is performed on each local reference object image to obtain each local color cast reference image of the candidate image.

[0121] It can be understood that the images detected and segmented using the reference object extraction model usually have redundant pixels, in order to make each local color cast reference image obtained as much as possible to contain clean and clear edges, in the embodiments of the present application, edge correction can be performed on each local reference object image, so as to suppress the generation of false edges and the influence of noise, and each local color cast reference image of the candidate image is obtained.

[0122] Specifically, in the embodiments of the present application, the simple linear iterative clustering (SLIC) can be used to perform edge correction on each local reference object image to obtain each local color cast reference image of the candidate image.

[0123] It can be understood that in addition to using SLIC to perform edge correction on the local reference object image, other edge detection and correction algorithms in the prior art can also be used to correct the local reference object image to obtain the local color cast reference image.

[0124] It should be noted that the candidate image sequence can be traversed and the above processing can be performed, and when the traversal is completed, the local color cast reference image of each candidate image can be obtained.

[0125] In addition, in order to distinguish the local color cast reference images of each candidate image, the sky color cast reference image sequence I S =[S1, S2,..., S n ], the road color cast reference image sequence I R=[R1,R2,...,R n ] and tree color cast reference image sequence I T =[T1,T2,...,T n Store the image sequence I = [I1, I2, ..., I...] n Each of the local color-skewed reference images in the candidate image sequence represents a local color-skewed reference image, and the index of each local color-skewed reference image in the candidate image sequence corresponds one-to-one with the order of the candidate image in the candidate image sequence.

[0126] It should be noted that, typically, each candidate image can have three corresponding local color-cast reference images, but this does not mean that each candidate image necessarily contains three different reference objects. For example, a candidate image may only contain trees, in which case the above process can only extract the local color-cast reference image containing the trees. In this case, the local color-cast reference image containing the sky and road can be set to 0 in its corresponding color-cast reference image sequence.

[0127] Step S103: Extract the global color cast reference image of the candidate image.

[0128] The global color cast reference image is an image of global pixel blocks.

[0129] Specifically, in this embodiment, a preset superpixel segmentation algorithm can be used to segment the candidate image into superpixel blocks, obtaining individual pixel blocks of the candidate image. Then, a global pixel block is selected based on the area of ​​each pixel block, and a global color cast reference image is obtained based on this global pixel block. The superpixel segmentation algorithm can be SLIC or other common superpixel segmentation algorithms in the prior art. SLIC can group similar pixels into pixel blocks, achieving the purpose of extracting the main elements of the image. Then, the j largest pixel blocks are selected as global pixel blocks, and a global color cast reference image of the candidate image is obtained based on these global pixel blocks. Here, j is positively correlated with device memory and computing resources, and its specific value can be set according to the actual situation of device memory and computing resources; this application does not limit this value.

[0130] Understandably, if the actual training device has large memory and computing resources, it means it can process more pixel blocks, and in this case, the value of j can be set to a larger value; if the actual training device has small memory and computing resources, it means it can only process fewer pixel blocks, and in this case, the value of j can be set to a smaller value.

[0131] It is understandable that by traversing the candidate image sequence, a global color cast reference image for each candidate image can be obtained.

[0132] It should be noted that there is no prior relationship in the execution order between step S102 and step S103, and both can be executed in parallel.

[0133] Step S104, using the color cast feature extraction model, determining the specified color cast reference image as the model update image.

[0134] In the embodiments of the present application, the color cast feature extraction model corresponding to the specified color cast reference image can be used to determine whether the specified color cast reference image is a model update image. The specified color cast reference image is any one of the local color cast reference images and the global color cast reference image.

[0135] In the embodiments of the present application, the candidate image corresponds to three local color cast reference images and one global color cast reference image, that is, the candidate image corresponds to four color cast reference images in total, and the specified color cast reference image is any one of the four color cast reference images. Each color cast reference image can be input to a corresponding color cast feature extraction model.

[0136] Specifically, the local color cast reference image containing the sky can be input to the corresponding sky color cast feature extraction model SVM-S, the local color cast reference image containing the road can be input to the corresponding road color cast feature extraction model SVM-R, and the local color cast reference image containing the tree can be input to the corresponding tree color cast feature extraction model SVM-T.

[0137] The method of the embodiments of the present application can be applied in various scenarios. Taking security monitoring as an example, since the background in the monitoring picture is relatively fixed, in the embodiments of the present application, the specified color cast reference image can be input to the corresponding color cast feature extraction model. If the image elements of the image do not satisfy the Karush–Kuhn–Tucker conditions (KKT), it can be considered that the specified color cast reference image contains support vectors that are not contained in the color cast feature extraction model, and then it can be determined that the specified color cast reference image is a model update image. At this time, step S105 and the subsequent steps can be executed. If the image elements of the image satisfy the KKT conditions, it can be determined that the specified color cast reference image is a non-model update image, and at this time, there is no need to update the color cast feature extraction model.

[0138] Step S105, using the color cast feature extraction model, determining the specified color cast reference image as the model update image.

[0139] In the embodiments of the present application, if the specified color cast reference image is determined to be a model update image, it indicates that the image can be used for the update training of the corresponding color cast feature extraction model. Therefore, the specified color cast reference image can be used as a new training sample of the corresponding color cast feature extraction model.

[0140] Step S106, updating the color cast feature extraction model by using the new training sample.

[0141] In the embodiments of the present application, the color cast feature extraction model can be trained by using the new training sample, so as to update the corresponding color cast feature extraction model according to the actual scene of the user.

[0142] It can be understood that each color cast feature extraction model can be updated by traversing each local color cast reference image and global color cast reference image and according to the obtained new training sample.

[0143] The color cast feature extraction model in the embodiments of the present application can be used to extract the color cast feature of the image after being pre-trained, and the image color cast classification can be determined according to the pre-established mapping relationship between the color cast feature and the color cast level.

[0144] Referring to Figure 3 , one embodiment of the image color cast classification method in the embodiments of the present application can include:

[0145] Step S301, obtaining a target image to be processed.

[0146] In the embodiments of the present application, the target image to be processed can be an image collected in advance by a preset camera acquisition device, or can be a real-time collected image.

[0147] Specifically, the target image can be obtained from a preset memory module or a server.

[0148] Step S302, extracting each local color cast reference image in the target image.

[0149] Referring to Figure 4 , step S302 can include the following specific process:

[0150] Step S3021, extracting a reference object from the target image by using a preset reference object extraction model to obtain each local reference object image of the target image.

[0151] Step S3022, performing edge correction on each local reference object image to obtain each local color cast reference image of the target image.

[0152] It can be understood that the content of step S302 can be referred to the detailed description of step S102, which will not be repeated here.

[0153] Step S303, extracting a global color cast reference image in the target image.

[0154] In the embodiment of the present application, the preset superpixel segmentation algorithm can be used to segment the target image into superpixels, and each pixel block of the target image is obtained, and then the global pixel block is selected according to the area of each pixel block, and then the global color cast reference image of the target image can be obtained according to the global pixel block.

[0155] It can be understood that the specific content of step S303 can refer to the detailed description of step S103, which will not be repeated here.

[0156] It should be noted that there is no order relationship between steps S302 and S303 in terms of execution, and the two steps can be executed in parallel.

[0157] Step S304, respectively extracting the color cast features of each local color cast reference image and global color cast reference image.

[0158] Please refer to Figure 5 Step S304 can include the following specific process:

[0159] Step S3041, using a preset specified color cast feature extraction model to extract the color cast features of the specified color cast reference image to obtain the first color cast features of the specified color cast reference image.

[0160] In the embodiment of the present application, the preset specified color cast feature extraction model can be used to extract the color cast features of the specified color cast reference image to obtain the first color cast features of the specified color cast reference image.

[0161] Wherein, the specified color cast feature extraction model is a color cast feature extraction model corresponding to the specified color cast reference image, and the specified color cast reference image is any one of the local color cast reference image and the global color cast reference image. For example, the specified color cast reference image can be a local color cast reference image containing the sky, and its corresponding specified color cast feature extraction model is SVM-S.

[0162] In the embodiment of the present application, after the specified color cast reference image is processed by the color cast feature extraction model (SVM in the embodiment of the present application), the specified color cast reference image is usually mapped to a multi-dimensional space as a support vector on a hyperplane, and the Euclidean distance of the support vector to the best color cast decision plane is the feature distance K. If the feature distance K is smaller, it means that the distance of the support vector to the best color cast decision plane is closer, which means that the color cast degree of the specified color cast reference image is smaller and it is more difficult to make a decision; otherwise, it means that the color cast degree of the specified color cast reference image is larger and it is easier to make a decision. Based on the above principle, the feature distance K can be used as the first color cast feature.

[0163] Step S3042, calculating the second color cast feature of the specified color cast reference image.

[0164] In the embodiments of the present application, the second color cast feature of the specified color cast reference image can also be calculated.

[0165] Specifically, the color cast intensity (i.e., average brightness L), color cast area (i.e., the ratio R of the pixels of the specified color cast reference image to the pixels of the target image), and color cast position (i.e., the distance D between the center of the specified color cast reference image and the center of the target image) of the specified color cast reference image can be calculated, and the three results obtained are taken as the second color cast feature of the specified color cast reference image.

[0166] It can be understood that the first color cast feature and the second color cast feature of the specified color cast reference image can be set according to the selected color cast feature extraction model and the actual situation of the application scenario, and the embodiments of the present application only serve as an example and do not constitute a specific limitation.

[0167] In a possible embodiment, the Lab value and the color cast factor can also be taken as the color cast feature.

[0168] It should be noted that there is no prior relationship in the order of execution between step S3041 and step S3042, and the two steps can be executed in parallel.

[0169] Step S3043, determining the color cast feature of the specified color cast reference image according to the first color cast feature and the second color cast feature of the specified color cast reference image.

[0170] In the embodiments of the present application, the color cast feature of the specified color cast reference image can be determined according to the first color cast feature and the second color cast feature of the specified color cast reference image.

[0171] Specifically, the first color cast feature and the second color cast feature of the specified color cast reference image can be combined to form a color cast base. For example, if the specified color cast reference image is a local color cast reference image containing sky, a sky color cast base Csky can be formed S = [K S , L S , R S , D S ], where K S , L S , R S , and D S are the feature distance, color cast intensity, color cast area, and color cast position of the local color cast reference image containing sky, respectively. Correspondingly, a road color cast base Croad can be formed for a local color cast reference image containing road R = [K R , L R , R R , D R ], where K R , L R , R R , and DR respectively are feature distance, color cast intensity, color cast area and color cast position of the local color cast reference image containing road; the local color cast reference image containing tree can constitute a tree color cast base C T = [K T , L T , R T , D T ], wherein K T , L T , R T and D T respectively are feature distance, color cast intensity, color cast area and color cast position of the local color cast reference image containing tree; the global color cast reference image can constitute a global color cast base C A = [K A , L A , R A , D A ], wherein K A , L A , R A and D A respectively are feature distance, color cast intensity, color cast area and color cast position of the global color cast reference image.

[0172] It can be understood that the color cast base of each local color cast reference image and global color cast reference image can be obtained by traversing each local color cast reference image and global color cast reference image according to the above method. If the target image does not exist a color cast reference image, the color cast base corresponding to the color cast reference image should be taken as 0. For example, when the reference object of the target image is extracted, it is detected that the target image does not contain a tree region, that is, the local color cast reference image containing tree does not exist, at this time, the tree color cast base of the target image should be taken as 0.

[0173] In step S305, the color cast level of the target image is determined according to the color cast feature.

[0174] In the embodiment of the present application, the color cast level of the target image can be determined according to the color cast feature. Specifically, the mapping relationship between the color cast feature and the color cast level can be established in advance, and the color cast level of the target image is determined according to the mapping relationship.

[0175] In a possible embodiment, one image and the subjective opinion score (color cast degree) of the human eye corresponding to the image can be taken as a training sample, and a preset number of training samples can be taken to form a training sample set, and then the image and the color cast degree of the image can be data fitted through training full connection neural network and the like, to obtain the weight matrix W = [W S , W R , W T , W AAfterwards, the weight matrix W can be directly used to weight the color cast base matrix C when subsequently determining the color cast level of the target image, to obtain the color cast level Q. The value range and mapping level of the color cast level can be adjusted according to actual scenarios. Here, the value range of Q can be preferably set to [1, 5], and the color cast degree of each value is respectively [1-difficult to picky, 2-careful to find slight color cast, 3-easy to observe slight color cast, 4-easy to observe obvious color cast, 5-easy to observe serious color cast].

[0176] In another possible embodiment, a manually designed element correlation method or other method of judging the correlation between two elements can also be used to obtain the mapping relationship between the color cast feature and the color cast level. For example, any one of Pearson Liner Correlation Coefficient (PLCC), Spearman Rank-Order Correlation Coefficient (SROCC), or mutual information, etc. can be used to obtain the mapping relationship between the color cast feature and the color cast level, and the color cast level of the target image is determined accordingly.

[0177] It can be understood that after the color cast level of the target image is determined, whether the target image needs to be subjected to image color cast correction can be determined according to the color cast level. Specifically, if the color cast level of the target image is greater than a preset level threshold, it indicates that the target image has a more serious color cast, and the target image can be subjected to image color cast correction at this time. Conversely, if the color cast level of the target image is less than or equal to the preset level threshold, it indicates that the target image has a lighter color cast, and the target image does not need to be subjected to image color cast correction at this time. The value of the level threshold can be set according to actual application scenarios, which is not limited in the embodiments of the present application.

[0178] Further, the color cast intensity, color cast area, and color cast position of the target image in the color cast base can also provide information and basis for subsequent image color cast correction work, so that a better color cast correction effect can be achieved.

[0179] In summary, the embodiments of the present application can determine the color cast level of the target image according to the color cast features of the local color cast reference image and the global color cast reference image containing the specified reference object in the target image, so that the local color cast information and the global color cast information of the image are fully utilized, the severity of the image color cast is more comprehensively and accurately evaluated, and the embodiments have strong ease of use and practicality.

[0180] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0181] A method for grading image color cast corresponding to the above embodiment, Figure 6 An embodiment structure diagram of a device for grading image color cast provided by the embodiments of the present application is shown.

[0182] In the embodiments of the present application, a device for grading image color cast can include:

[0183] A target image acquisition module 601 is configured to acquire a target image to be processed;

[0184] A local color cast reference image extraction module 602 is configured to extract each local color cast reference image in the target image; wherein each local color cast reference image is an image of a specified reference object;

[0185] A global color cast reference image extraction module 603 is configured to extract a global color cast reference image in the target image; wherein the global color cast reference image is an image of a global pixel block;

[0186] A color cast feature extraction module 604 is configured to extract color cast features of each local color cast reference image and the global color cast reference image, respectively;

[0187] A color cast level determination module 605 is configured to determine a color cast level of the target image according to the color cast features.

[0188] In a specific implementation manner of the embodiments of the present application, the local color cast reference image extraction module can include:

[0189] A reference object extraction unit is configured to extract reference objects from the target image by using a preset reference object extraction model to obtain each local reference object image of the target image; wherein the reference object extraction model is a pre-trained artificial intelligence model used for reference object extraction, and each local reference object image contains a specified reference object;

[0190] An edge correction unit is configured to perform edge correction on each local reference object image to obtain each local color cast reference image of the target image.

[0191] In a specific implementation manner of the embodiments of the present application, the global color cast reference image extraction module can include:

[0192] A superpixel block unit is configured to perform superpixel block division on the target image by using a preset superpixel block algorithm to obtain each pixel block of the target image;

[0193] a global pixel block selection unit, configured to select the global pixel block according to areas of the pixel blocks;

[0194] a global color cast reference image determination unit, configured to obtain a global color cast reference image of the target image according to the global pixel block.

[0195] In a specific implementation manner of the embodiment of the present application, the color cast feature extraction module can include:

[0196] a first color cast feature extraction unit, configured to perform color cast feature extraction on a specified color cast reference image by using a preset specified color cast feature extraction model to obtain first color cast features of the specified color cast reference image; wherein the specified color cast feature extraction model is a color cast feature extraction model corresponding to the specified color cast reference image, and the specified color cast reference image is any one of the local color cast reference images and the global color cast reference image;

[0197] a second color cast feature calculation unit, configured to calculate second color cast features of the specified color cast reference image;

[0198] a color cast feature determination unit, configured to determine color cast features of the specified color cast reference image according to the first color cast features and the second color cast features of the specified color cast reference image.

[0199] In a specific implementation manner of the embodiment of the present application, the image color cast grading device can further include:

[0200] a candidate image acquisition module, configured to acquire a pre-stored candidate image;

[0201] a first reference image extraction module, configured to extract each local color cast reference image of the candidate image;

[0202] a second reference image extraction module, configured to extract a global color cast reference image of the candidate image;

[0203] a model update image determination module, configured to determine whether a specified color cast reference image is a model update image by using the color cast feature extraction model corresponding to the specified color cast reference image, the specified color cast reference image being any one of the local color cast reference images and the global color cast reference image;

[0204] a training sample determination module, configured to, if the specified color cast reference image is the model update image, take the specified color cast reference image as a new training sample of the color cast feature extraction model;

[0205] a model update module, configured to update the color cast feature extraction model by using the new training sample.

[0206] In a specific implementation process of the embodiment of the present application, the candidate image acquisition module can further include:

[0207] a historical image acquisition unit, configured to acquire a preset number of historical images;

[0208] a structural similarity calculation unit, configured to calculate structural similarities between the historical images to obtain similarity degree values of the historical images;

[0209] a candidate image determination unit, configured to determine a specified historical image as a candidate image if the similarity degree value of the specified historical image is less than a preset similarity degree threshold; wherein the specified historical image is any one of the historical images.

[0210] In a specific implementation process of the embodiment of the present application, the color cast level determination module can include:

[0211] a color cast level determination unit, configured to determine the color cast level of the target image according to the color cast feature and a preset color cast level mapping relationship; wherein the color cast level mapping relationship is a mapping relationship between the color cast feature and the color cast level.

[0212] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described apparatuses, modules and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0213] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0214] Figure 7 A schematic block diagram of a terminal device provided by an embodiment of the present application is shown, and only the parts related to the embodiments of the present application are shown for the convenience of description.

[0215] As shown in Figure 7 The terminal device 7 of this embodiment includes a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. The processor 70 implements the steps in each of the image color cast grading method embodiments when executing the computer program 72, such as steps S301 to S305 as shown in Figure 3 Alternatively, the processor 70 implements the functions of each module / unit in the above-described apparatus embodiments when executing the computer program 72, such as the functions of the modules 601 to 605 as shown in Figure 6

[0216] ​For example, the computer program 72 can be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 72 in the terminal device 7.

[0217] Those skilled in the art can understand that, Figure 7 The terminal device 7 is only an example and does not constitute a limitation on the terminal device 7, and can include more or fewer components than those shown, or combine certain components, or different components, for example, the terminal device 7 can also include an input / output device, a network access device, a bus, etc.

[0218] The processor 70 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0219] The memory 71 can be an internal storage unit of the terminal device 7, such as a hard disk or a memory of the terminal device 7. The memory 71 can also be an external storage device of the terminal device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 71 can include both the internal storage unit and the external storage device of the terminal device 7. The memory 71 is used to store the computer program and other programs and data required by the terminal device 7. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0220] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0221] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0222] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0223] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are only schematic, and the division of the modules or units is only a logical function division, and there can be another division in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0224] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0225] In addition, each of the function units in each of the embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0226] The integrated module / unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be implemented by a computer program instructing related hardware to complete, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer readable storage medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable storage medium does not include electric carrier signals and telecommunication signals.

[0227] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of image color cast grading, the method comprising: The method comprises the following steps: acquiring a target image to be processed; extracting each local color cast reference image in the target image; each local color cast reference image is an image of a specified reference object; extracting a global color cast reference image in the target image; the global color cast reference image is an image of a global pixel block; extracting color cast features of each local color cast reference image and the global color cast reference image respectively; determining a color cast level of the target image according to the color cast features.

2. The image decentering grading method of claim 1, wherein, The step of extracting each local color cast reference image in the target image comprises the following steps: extracting reference objects in the target image by using a preset reference object extraction model to obtain each local reference object image of the target image; the reference object extraction model is a pre-trained artificial intelligence model for reference object extraction; each local reference object image contains a specified reference object; performing edge correction on each local reference object image to obtain each local color cast reference image of the target image.

3. The image decentering grading method of claim 1, wherein, The step of extracting a global color cast reference image in the target image comprises the following steps: performing superpixel segmentation on the target image by using a preset superpixel segmentation algorithm to obtain each pixel block of the target image; selecting the global pixel block according to the area of each pixel block; obtaining the global color cast reference image of the target image according to the global pixel block.

4. The image decentering grading method of claim 1, wherein, The step of extracting color cast features of each local color cast reference image and the global color cast reference image respectively comprises the following steps: extracting color cast features of a specified color cast reference image by using a preset specified color cast feature extraction model to obtain first color cast features of the specified color cast reference image; the specified color cast feature extraction model is a color cast feature extraction model corresponding to the specified color cast reference image; the specified color cast reference image is any one of each local color cast reference image and the global color cast reference image; calculating second color cast features of the specified color cast reference image; determining color cast features of the specified color cast reference image according to the first color cast features and the second color cast features of the specified color cast reference image.

5. The image decentering grading method of claim 4, wherein, Before the step of extracting color cast features of a specified color cast reference image by using a preset specified color cast feature extraction model, the color cast feature extraction model is updated. The updating process of the color cast feature extraction model comprises the following steps: acquiring a pre-stored candidate image; extracting each local color cast reference image of the candidate image; extracting a global color cast reference image of the candidate image; determining whether a specified color cast reference image is a model updating image by using the color cast feature extraction model; the color cast feature extraction model corresponds to the specified color cast reference image; the specified color cast reference image is any one of each local color cast reference image and the global color cast reference image; if the specified color cast reference image is a model updating image, the specified color cast reference image is used as a new training sample of the color cast feature extraction model; updating the color cast feature extraction model by using the new training sample.

6. The image decentering grading method of claim 5, wherein, The determination process of the candidate image comprises the following steps: acquiring a preset number of historical images; Calculate structural similarity between each historical image to obtain a similarity value between each historical image; If the similarity value of a specified historical image is less than a preset similarity threshold, the specified historical image is determined as a candidate image; wherein the specified historical image is any one of the historical images.

7. The image skew grading method of any one of claims 1 to 6, wherein, The color cast level of the target image is determined according to the color cast feature, including: The color cast level of the target image is determined according to the color cast feature and a preset color cast level mapping relationship; wherein the color cast level mapping relationship is a mapping relationship between the color cast feature and the color cast level.

8. An image color cast grading device, characterized by, Including: A target image acquisition module is configured to acquire a target image to be processed; A local color cast reference image extraction module is configured to extract each local color cast reference image in the target image; wherein each local color cast reference image is an image of a specified reference object; A global color cast reference image extraction module is configured to extract a global color cast reference image in the target image; wherein the global color cast reference image is an image of a global pixel block; A color cast feature extraction module is configured to extract a color cast feature of each local color cast reference image and the global color cast reference image, respectively; A color cast level determination module is configured to determine a color cast level of the target image according to the color cast feature.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the steps of the image color cast grading method according to any one of claims 1 to 7.

10. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the image color cast grading method according to any one of claims 1 to 7.

Images