Color cast identification method and device based on region weight, equipment and storage medium

Abstract

The invention provides a region weight-based color cast identification method, device and equipment and a readable storage medium. The method comprises the following steps of: obtaining a certificate image and a plurality of historical certificate images; processing the certificate image through the first network layer to obtain a segmented region image in the certificate image; processing the segmented region image through a second network layer to obtain a first image feature; and determining a color cast identification result of the certificate image according to the first image feature and a second image feature corresponding to the historical certificate image. The segmented region images corresponding to different main body information in the certificate image are obtained through processing, the image features of the corresponding regions are obtained, and compared with color cast identification from the whole image, color cast feature analysis is carried out based on different significance of different segmented regions, so that the identification accuracy is improved. And especially in a color cast recognition scene in which a portrait base map is lacked as a comparison reference, a better color cast recognition effect can be ensured.

Description

Technical Field

[0001] The present application relates to the technical field of image recognition, and in particular to a method, device, equipment and storage medium for color cast recognition based on regional weights. Background Art

[0002] Image color is an important feature of images and is crucial in the fields of smart security, medical imaging, and printing. However, due to factors such as light sources, acquisition, and imaging components, there will be a certain degree of difference in color between the image and the real image. This color cast difference not only affects the appearance, but may also have a serious negative impact on its use and value.

[0003] Especially in the passport printing process, due to the influence of ink nozzle blockage and other factors, the passport will have various abnormal printing color casts. The ID portrait in the passport is one of the most important personalized information in the passport. The characteristics of the cardholder such as skin color do not match the passport printed portrait, which may cause recognition errors of the automatic identity recognition system. At present, most of the portrait color cast recognition is performed from the entire image. Especially in the scene without sufficient portrait background, the color cast recognition only from the entire image will lead to unsatisfactory color cast results due to the lack of sufficient feature comparison. Summary of the invention

[0004] Based on this, it is necessary to provide a color cast identification method, device, equipment and storage medium based on regional weight to address the above technical problems, so as to solve the technical problem that the color cast identification effect in the related technology is not ideal.

[0005] In a first aspect, the present application provides a color cast identification method based on region weights, comprising:

[0006] Obtaining a document image to be identified and multiple historical document images;

[0007] Processing the document image through the first network layer in the trained feature extraction model to obtain multiple segmented area images in the document image;

[0008] Processing the plurality of segmented region images through the second network layer in the feature extraction model to obtain a first image feature corresponding to the document image;

[0009] A color cast recognition result of the document image is determined according to the first image feature and a second image feature corresponding to the historical document image.

[0010] As a feasible embodiment of the present application, the processing of the multiple segmented region images by the second network layer in the feature extraction model to obtain the first image feature corresponding to the document image includes:

[0011] Processing the portrait region image in the segmented region image through the second network layer in the feature extraction model to obtain a first portrait feature;

[0012] Processing the background area image in the segmented area image through the second network layer to obtain a first background feature;

[0013] Processing a reference area image in the segmented area image through the second network layer to obtain a first reference feature; the reference area image is used to indicate a clothing area and / or a hairstyle area in the document image;

[0014] The first portrait feature, the first background feature and the first reference feature are fused to obtain a first image feature corresponding to the document image.

[0015] As a feasible embodiment of the present application, the feature extraction model is trained by the following steps:

[0016] Acquire a plurality of sample ID images; the sample ID images include positive sample ID images and negative sample ID images;

[0017] Inputting the sample ID image into the first initial network layer in the initial feature extraction model to obtain a sample segmentation image of the sample ID image;

[0018] Inputting the sample segmented image into the second initial network layer in the initial feature extraction model to obtain the first sample image feature of the positive sample ID image and the second sample image feature of the negative sample ID image;

[0019] Determining a model loss value of the initial feature extraction model according to the first sample image feature and the second sample image feature;

[0020] The initial feature extraction model is updated according to the model loss value until the model loss value obtained by inputting the sample document image into the updated feature extraction model meets a preset condition, and the current updated feature extraction model is determined as the trained feature extraction model.

[0021] As a feasible embodiment of the present application, determining the model loss value of the initial feature extraction model according to the first sample image feature and the second sample image feature includes:

[0022] Determine a first model loss value of the initial feature extraction model according to a first sample portrait feature in the first sample image feature and a second sample portrait feature in the second sample image feature;

[0023] Determine a second model loss value of the initial feature extraction model according to a first sample background feature in the first sample image feature and a second sample background feature in the second sample image feature;

[0024] Determine a third model loss value of the initial feature extraction model according to a first sample reference feature in the first sample image feature and a second sample reference feature in the second sample image feature;

[0025] A model loss value of the initial feature extraction model is determined according to the first model loss value, the second model loss value and the third model loss value.

[0026] As a feasible embodiment of the present application, determining the model loss value corresponding to the initial feature extraction model according to the first model loss value, the second model loss value, and the third model loss value includes:

[0027] Determining a training round corresponding to the initial feature extraction model;

[0028] The first model loss value, the second model loss value, and the third model loss value are weighted according to the weight coefficient corresponding to the training round to obtain a model loss value corresponding to the initial feature extraction model.

[0029] As a feasible embodiment of the present application, the step of obtaining a plurality of sample document images includes:

[0030] Determine a positive sample document image in a first color space corresponding to the sample portrait base image;

[0031] Converting the positive sample document image into a reference document image in a second color space;

[0032] The values ​​of each color channel in the reference document image are adjusted to obtain a negative sample document image.

[0033] As a feasible embodiment of the present application, the second image feature corresponding to the historical document image includes the third image feature of the historical document image with color cast and the fourth image feature of the historical document image without color cast;

[0034] The determining, according to the first image feature and the second image feature corresponding to the historical document image, a color cast recognition result of the document image includes:

[0035] determining a first feature similarity between the first image feature and the third image feature;

[0036] determining a second feature similarity between the first image feature and the fourth image feature;

[0037] The color cast recognition result of the document image is determined according to the magnitude relationship between the first feature similarity and the second feature similarity.

[0038] In a second aspect, the present application provides a color cast identification device based on region weight, comprising:

[0039] An acquisition module, used to acquire an image of a certificate to be identified and multiple historical certificate images;

[0040] A segmentation module, used for processing the document image through the first network layer in the trained feature extraction model to obtain a plurality of segmented area images in the document image;

[0041] A processing module, used for processing the plurality of segmented region images through the second network layer in the feature extraction model to obtain a first image feature corresponding to the document image;

[0042] The recognition module is used to determine the color cast recognition result of the document image according to the first image feature and a second image feature corresponding to the historical document image.

[0043] In a third aspect, the present application further provides a computer device, the computer device comprising:

[0044] one or more processors; memory; and

[0045] One or more application programs, wherein the one or more application programs are stored in the memory and are configured to be executed by the processor to implement the color cast identification method based on area weight provided above.

[0046] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the color cast identification method based on regional weights provided above.

[0047] In a fifth aspect, an embodiment of the present application provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the color cast identification method based on regional weights provided above.

[0048] The color cast recognition method based on regional weights provided in the embodiment of the present application first processes the certificate image to obtain a certificate image to be recognized and multiple historical certificate images to obtain multiple segmented region images corresponding to different subject information in the certificate image, and then processes the multiple different segmented region images to obtain image features focusing on different regions, thereby determining the color cast recognition result of the certificate image through the similarity between the image features corresponding to the multiple historical certificate images. Compared with color cast recognition from the entire image, color cast feature analysis based on the different significance of different segmented regions can ensure better color cast recognition effect, especially in color cast recognition scenarios where there is a lack of a portrait base map as a comparison reference. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1a A schematic diagram of the steps of a color cast identification method based on regional weights provided in an embodiment of the present application;

[0051] Figure 1b A schematic diagram of the effect of multiple segmented area images in a document image (virtual synthetic portrait) provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of a step flow of training a feature extraction model provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a process flow for obtaining a sample document image provided in an embodiment of the present application;

[0054] Figure 4a A schematic diagram of a process flow for constructing a model loss value by using regional weights provided in an embodiment of the present application;

[0055] Figure 4b A schematic diagram of the effect of a training feature extraction model provided in an embodiment of the present application;

[0056] Figure 4c A schematic diagram showing the effect of multiple different color feature extractors provided in an embodiment of the present application processing images of different regions respectively to obtain image features;

[0057] Figure 5 A schematic diagram of the steps for determining a color cast recognition result provided in an embodiment of the present application;

[0058] Figure 6a A schematic diagram of the effect of a network structure provided in an embodiment of the present application;

[0059] Figure 6b A schematic diagram of the effect of a subsequent color cast recognition application scenario provided by an embodiment of the present application;

[0060] Figure 7 A schematic diagram of the structure of a portrait color cast recognition device provided in an embodiment of the present application;

[0061] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0063] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0064] In the description of the present application, the word "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0065] To facilitate understanding of the color cast recognition method based on regional weights provided in the embodiment of the present application, the relevant application background of the portrait color cast recognition method is first explained. Specifically, image color is an important characteristic of an image, which is crucial in the fields of intelligent security, medical imaging, and printing. However, in specific application scenarios, due to the influence of external factors such as light sources, acquisition, and imaging components, there is usually a certain degree of difference in color between the image and the real object. Especially in the field of portrait documents such as passports, during the passport printing process, due to the influence of ink nozzle blockage, etc., various abnormal printing color casts will usually appear in the portrait of the passport. The portrait of the passport is one of the most important personalized information in the passport. The characteristics of the holder such as skin color do not match the portrait printed in the passport, which often leads to recognition errors of the automatic identity recognition system. Therefore, compared with other images, the tolerance for color cast in the portrait of the passport is very low, so the requirements for color cast recognition are very high.

[0066] Most of the current methods for identifying color cast in portraits rely on identifying the color cast of the entire image. For example, the two-dimensional histogram of the color channels in the image is processed, and the color cast is determined based on the maximum connected domain to identify the color cast of the image, or the color space histogram information entropy is compared to determine whether the image has color cast. Although the above method can identify the color cast of the image, the overall color cast identification of the image is prone to losing important subject information in the image, and in some scenes where there is no portrait base map as a comparison reference, the final color cast identification effect is not ideal due to the lack of sufficient feature comparison.

[0067] In order to solve the above problems, the present application provides a method, device, equipment and readable storage medium for color cast identification based on regional weights. By using the idea of ​​machine learning, it is trained that the image can be segmented to obtain multiple segmented regions first, and then the differences in the significance of color cast in different regions are considered. By analyzing the different segmented regions, image features focusing on the local area of ​​the image are obtained. Especially in the scene where there is a lack of sufficient portrait background, by comparing the features of the different regions, a more sufficient comparison effect can be provided, thereby effectively improving the color cast identification effect of the document image in the scene where there is a lack of portrait background. Specifically, the color cast identification method based on regional weights is usually set in the color cast identification device based on regional weights in the form of a computer program, and the color cast identification device based on regional weights is usually set in the computer device in the form of a processor. The color cast identification device in the computer device executes the computer program corresponding to the color cast identification method to execute the color cast identification method based on regional weights provided in the embodiment of the present application.

[0068] For details, please refer to Figure 1a , Figure 1aA schematic flow chart of a method for identifying color cast in a portrait provided in an embodiment of the present application, specifically comprising steps S110 to S140:

[0069] S110, obtaining a document image to be identified and a plurality of historical document images.

[0070] In the embodiments of the present application, combined with the aforementioned description of the relevant application scenarios, it can be known that the document image to be identified usually refers to the document image obtained by photographing the document containing a portrait, for example, the more common document image is the document image obtained by photographing the portrait on the passport. Unlike the conventional method of further obtaining a portrait base map of the document image to be identified for feature comparison, in the technical solution provided in the embodiments of the present application, multiple historical document images are utilized, and these historical document images can generally be understood as document images of other portraits. Specifically, these historical document images generally include historical document images with color cast and historical document images without color cast.

[0071] S120, processing the document image through the first network layer in the trained feature extraction model to obtain a plurality of segmented area images in the document image.

[0072] Combined with the above-mentioned related description, due to the difference in the significance of color cast in different areas of the document image, for example, the portrait area, especially the skin color reflected in the face area has a higher degree of influence on the significance of color cast, and the color cast of the skin color in the document often leads to recognition errors of the identity recognition system, while the background area, such as the color of the background wall, has a relatively low degree of influence on the significance of color cast, and other reference areas such as the hairstyle area and the clothing area have a weaker degree of influence on the significance of color cast. Therefore, in the embodiment of the present application, it is considered to first segment the document image through the first network layer in the trained feature extraction model, so as to obtain multiple segmented area images in the document image. Among them, these segmented area images are the portrait area, background area and other reference areas mentioned above. Among them, it should be noted that since the first network layer in the feature extraction model usually realizes the segmentation processing of the image, the first network layer can usually adopt a network structure similar to the image segmentation model, or in other words, the first network layer can also be called a segmentation model.

[0073] For more information, please refer to Figure 1b , Figure 1bThat is, a schematic diagram of the effect of multiple segmented area images in a document image provided by an embodiment of the present application, wherein it should be noted that the portrait in the document image is a virtual synthetic portrait, not a real portrait, and therefore, does not involve privacy leakage issues. It can be seen that after the document image is input into the segmentation model, that is, after being processed by the first network layer, it will be roughly subdivided into portrait area, background area and other areas (reference area), or in other words, each pixel in the document image can be correspondingly identified as a portrait, background or reference mark, thereby forming corresponding mask information for subsequent further processing.

[0074] Of course, the segmented area image provided above is only a feasible effect schematic diagram, and the document image can also be divided into more detailed segmented area images. For example, the portrait area can be further subdivided into areas where the facial features are located, and the reference area can be further subdivided into hairstyles, clothing, etc. It can be understood that the more detailed the image segmentation is, the better the color cast recognition effect can be achieved, but the corresponding image processing calculation requirement is higher. The specific segmented area image obtained can be set based on the actual needs of the user, and the embodiment of the present application is not limited here. However, combined with specific experimental measurements, dividing the document image into a portrait area, a background area, and a reference area can effectively meet the color cast recognition accuracy requirements for portraits. Therefore, in order to simplify the description, the portrait area, background area, and reference area are used as examples for explanation.

[0075] Furthermore, the first network layer can usually use a variety of network structures to process the document image. However, as an optional embodiment of the present application, a solution is provided for better implementing the segmentation of the image through a network layer of an encoder-decoder structure. Specifically, the encoder part consists of multiple convolution blocks and downsampling layers. Each convolution block contains two 3x3 convolution layers, followed by a ReLU (Linear rectification function) activation function, and a 2x2 maximum pooling layer is used for downsampling. Through this structure, the spatial size of the feature map is reduced layer by layer, while the abstraction level of the feature is increased. For example, the initial size of the input document image is 416*416*3 (h*w*3). After four downsamplings, the size of the feature map becomes 208*208*64, 104*104*128, 52*52*256, and finally reaches 26*26*512 at the end of the encoder. The decoder part gradually restores the spatial size of the image through upsampling and convolution operations, and combines the feature map of the corresponding layer of the encoder to enhance the detail information of the feature map through jump connections. Finally, the decoder outputs a feature map of size (h*w*3), in which the three channels represent the probability values ​​of the portrait, background, and reference areas respectively. The three channel values ​​of each pixel are processed by the softmax (normalized exponential function) function, and the channel with the maximum probability value is selected as the classification result of the pixel. A segmentation map of size h*w can be generated, and each pixel is marked as one of the portrait, background, or reference areas. In the above way, the portrait, background, and reference areas can be accurately segmented, thus providing a basis for subsequent portrait color cast recognition.

[0076] S130, processing the plurality of segmented region images through the second network layer in the feature extraction model to obtain a first image feature corresponding to the document image.

[0077] In the embodiment of the present application, after completing the segmentation process of the document image through the aforementioned network layer to obtain multiple segmented region images, in the technical solution provided in the embodiment of the present application, each segmented region image will be further processed through the second network layer in the feature extraction model, so as to finally obtain the first image feature corresponding to the document image. It can be understood that since the second network layer is mainly used to extract color feature information in the image, the second network layer can adopt a common network model structure used to extract color features in the image. In other words, the second network layer can also be understood as a color feature extractor.

[0078] Specifically, for ease of understanding, taking the aforementioned segmented region images including a portrait region image, a background region image, and a reference region image as an example, at this time, the second network layer in the feature extraction model processes the multiple segmented region images to obtain the first image feature corresponding to the document image, specifically including:

[0079] Processing the portrait region image in the segmented region image through the second network layer in the feature extraction model to obtain a first portrait feature;

[0080] Processing the background area image in the segmented area image through the second network layer to obtain a first background feature;

[0081] Processing a reference area image in the segmented area image through the second network layer to obtain a first reference feature; the reference area image is used to indicate a clothing area and / or a hairstyle area in the document image;

[0082] The first portrait feature, the first background feature and the first reference feature are fused to obtain a first image feature corresponding to the document image.

[0083] It can be seen that in the technical solution provided in the embodiment of the present application, the portrait area image, the background area image and the reference area image in the segmented area image are processed separately through the second network layer, and the feature information obtained by the processing is fused to obtain the first image feature corresponding to the document image.

[0084] Of course, it should be noted that, in combination with the above-mentioned related description, considering the differences in the significance of color cast in different regions (such as portraits, backgrounds, and other reference regions), therefore, in order to enable the model to pay more attention to the regions with significant color cast effects, thereby improving the color cast recognition accuracy, it is possible to select to fuse the first portrait feature, the first background feature, and the first reference feature according to a specific weight to obtain the first image feature corresponding to the document image, for example, the weight of the first portrait feature is higher than that of the first background feature, and the weight of the first background feature is higher than that of the first reference feature. However, in order to further ensure the color cast recognition effect of the portrait, as an implementation scheme of the present application, a loss function is constructed by relying on the contrast loss of the regional weights during the training process. By assigning different weights to different regions, the focus of the model is refined during the model training process. Compared with simply setting the weights of each feature based on human experience, by making the model itself pay attention to the differences between different regions during the model training process, the model's recognition ability for a specific region with a high degree of significant influence on the printing color cast can be enhanced, which can improve the overall recognition effect and the robustness of the model, thereby effectively improving the performance in the color cast recognition task. Specifically, to facilitate understanding of the above content, the following will further provide a specific implementation scheme for training feature extraction models. Figure 2 , Figure 2 A schematic flow chart of a training feature extraction model provided in an embodiment of the present application, specifically, including steps S210 to S250:

[0085] S210, obtaining a plurality of sample ID images; the sample ID images include positive sample ID images and negative sample ID images.

[0086] In the embodiments of the present application, the sample ID image generally refers to a photograph of the ID portrait obtained by printing the sample portrait base map. For example, it can be obtained by network technology means, such as crawlers or extracted from a portrait database. The embodiments of the present application do not limit the method of obtaining the sample portrait base map.

[0087] In order to better achieve the training effect of the subsequent model, in the embodiment of the present application, a method based on contrastive learning is provided to effectively ensure the color cast recognition effect of the trained feature extraction model. That is to say, the sample ID images provided in the embodiment of the present application may include positive sample ID images of ID portraits without color cast and negative sample ID images of ID portraits with color cast. The training effect of the model is improved by shortening the distance between the ID images without color cast and increasing the distance between the ID images with color cast. The specific implementation scheme of improving the model training effect by contrastive learning will be further supplemented in each subsequent training process.

[0088] However, in actual application scenarios, it is considered that the printed color-biased image and the printed non-color-biased image are generally mutually exclusive, that is, in the process of making certificates, if the printed certificate has no color-bias or other printing quality problems, it will not be printed repeatedly, which will result in the number of printed color-biased images, that is, the number of negative sample certificate images, being much lower than the number of printed non-color-biased images, that is, the number of positive sample certificate images. Therefore, in order to enrich the training sample pairs of the obtained sample portrait base map and the corresponding sample certificate images to improve the training effect of the model, as another feasible embodiment of the present application, the present application also provides an implementation scheme for expanding the sample certificate images. For details, please refer to Figure 3 , Figure 3 A schematic diagram of a process flow for obtaining a sample document image provided in an embodiment of the present application includes steps S310 to S330:

[0089] S310, determining a positive sample document image in a first color space corresponding to the sample portrait base image.

[0090] In the embodiment of the present application, combined with the above-mentioned related description, since there are few negative sample ID images with color cast corresponding to the sample portrait base map, the sample portrait base map and the positive sample ID image corresponding to the sample portrait base map in the first color space can be read from the database first. The first color space here is usually an RGB (red, green, blue, three primary colors of red, green and blue light) color space, that is, a color space based on the principle of color additive mixing.

[0091] S320: Convert the positive sample document image into a reference document image in a second color space.

[0092] Considering that the printing process is usually carried out in a color space based on the color subtraction principle, that is, in the CMYK (cyan, magenta, yellow, black, i.e., printing is performed using four colors of ink, cyan, magenta, yellow, and black) color space, the color cast of the real ID portrait is usually also changed due to the ratio of the above four colors. Therefore, in order to fit a more realistic negative sample ID image, in the embodiment of the present application, the positive sample ID image in the corresponding first color space, that is, the RGB color space, will be converted into a reference ID image in the CMYK color space. Among them, the conversion formula between the RGB color space and the CMYK color space is not specifically described in detail in the embodiment of the present application.

[0093] S330, adjusting the values ​​of each color channel in the reference document image to obtain a negative sample document image corresponding to the sample portrait base image.

[0094] In an embodiment of the present application, by appropriately adjusting the values ​​of each color channel, that is, C, M, Y, and K, in a reference document image in a second color space, that is, the CMYK color space used for printing, a negative sample document image having a color cast error relative to the sample portrait base image can be formed.

[0095] For example, as a common feasible implementation scheme, in CMYK space, the value range of each color channel is 0 to 100. Therefore, 20 can be selected as a unit to ensure that each color channel has 6 different values ​​(0, 20, 40, ..., 100). In this way, each color channel can have 6 changes, and for each sample ID portrait, a total of 1296 (6×6×6×6) different color cast images can be generated as negative sample ID images. Of course, in order to further enrich the negative sample ID images, a floating coefficient can be further set. For example, a floating coefficient between -2 and 2 can be randomly generated based on each value, thereby improving the richness of a single class sample and further improving the accuracy and robustness of the feature extraction model. Of course, in actual application, the number of negative sample ID images can be set based on actual needs. Therefore, the present application can usually obtain an appropriate number of negative sample ID images for subsequent model training by appropriately adjusting the values ​​of each color channel in the reference ID image.

[0096] The embodiment of the present application provides a method for generating negative sample ID images by converting the positive sample ID images in the RGB color space into the CMYK space used for printing, and by adjusting the values ​​of each color channel in the CMYK color space. This method can better fit the color cast problem of the ID images caused by changes in the proportion of ink used during the printing process. Compared with the ID images obtained by directly adjusting the color channel values ​​in the RGB color space, the negative sample ID images with color differences in the CMYK color space obtained in batches can further effectively improve the effect of the feature extraction model on color cast recognition in the CMYK color space when used for subsequent model training, thereby improving the color cast recognition effect on printed ID portraits.

[0097] S220, inputting the sample ID image into a first initial network layer in an initial feature extraction model to obtain a sample segmentation image of the sample ID image.

[0098] In order to better realize the information of image feature extraction, in the embodiment of the present application, the first network layer used to implement image segmentation in the feature extraction model also needs to rely on model training. Therefore, similar to the process of implementing image segmentation by the first network layer in the trained feature extraction model, the sample ID image needs to be input into the first initial network layer with randomly initialized parameters in the initial feature extraction model in the embodiment of the present application, so as to obtain the corresponding sample segmentation image. Specifically, the sample segmentation image here usually includes the first sample segmentation image of the positive sample ID image without color cast and the second sample segmentation image of the negative sample ID image with color cast. Among them, similar to the above, the first sample segmentation image of the positive sample ID image usually includes the first sample portrait image, the first sample background image and the first sample reference image. In addition, the second sample segmentation image of the negative sample ID image usually includes the second sample portrait image, the second sample background image and the second sample reference image. It should be noted that the reference meaning of each word mentioned above is understood and confirmed by those skilled in the art after reading the description of this solution, and there is no ambiguity in reference.

[0099] S230, inputting the sample segmented image into the second initial network layer in the initial feature extraction model to obtain the first sample image feature of the positive sample ID image and the second sample image feature of the negative sample ID image.

[0100] In an embodiment of the present application, after the first initial network layer in the initial feature extraction model completes the segmentation processing of the sample document image to obtain the first sample segmentation image and the second sample segmentation image, by further inputting these sample segmentation images into the second initial network layer in the initial feature extraction model with randomly initialized parameters, the corresponding first sample image features and second sample image features can be obtained.

[0101] Specifically, taking the first sample segmentation image including the first sample portrait image, the first sample background image and the first sample reference image, and the second sample segmentation image including the second sample portrait image, the second sample background image and the second sample reference image as an example, at this time, the obtained first sample image features correspondingly include the first sample portrait features, the first sample background features and the first sample reference features, and the second sample image features usually include the second sample portrait features, the second sample background features and the second sample reference features, and the complete implementation steps are as follows:

[0102] Inputting the first sample portrait image in the first sample segmented image into the second initial network layer in the initial feature extraction model to obtain the first sample portrait feature in the first sample image feature;

[0103] Inputting the first sample background image in the first sample segmented image into the second initial network layer in the initial feature extraction model to obtain the first sample background features in the first sample image features;

[0104] Inputting the first sample reference image in the first sample segmented image into the second initial network layer in the initial feature extraction model to obtain the first sample reference feature in the first sample image feature; and

[0105] Inputting the second sample portrait image in the second sample segmented image into the second initial network layer in the initial feature extraction model to obtain the second sample portrait feature in the second sample image feature;

[0106] Inputting the second sample background image in the second sample segmented image into the second initial network layer in the initial feature extraction model to obtain the second sample background features in the second sample image features;

[0107] The second sample reference image in the second sample segmented image is input into the second initial network layer in the initial feature extraction model to obtain the second sample reference feature in the second sample image feature.

[0108] S240: Determine a model loss value of the initial feature extraction model according to the first sample image feature and the second sample image feature.

[0109] On the basis of the foregoing, it can be understood that the first sample image features and the second sample image features can reflect to a certain extent the effectiveness of the initial feature extraction model for image segmentation and feature extraction. It can be understood that for positive sample ID images without color cast, after the feature extraction model with better performance completes the processing of different positive sample ID images, the multiple first sample image features obtained should be relatively similar. Conversely, for positive sample ID images without color cast and negative sample ID images with color cast, after the feature extraction model with better performance completes the processing of the positive sample ID images and the negative sample ID images, the first sample image features and the second sample image features obtained should have low similarity. Therefore, the model loss value of the initial feature extraction model can be determined according to the similarity between the first sample image features and the second sample image features.

[0110] Of course, in combination with the above-mentioned related descriptions, it can be known that in order to enable the model to pay attention to the differences between different regions during the training process, thereby enhancing the model's ability to recognize specific regions that have a high degree of significant impact on printing color cast, the embodiment of the present application also relies on the contrast loss of the sub-regional weights to construct a loss function, that is, in the process of determining the model loss value of the initial feature extraction model based on the first sample image features and the second sample image features, the model loss value can be determined based on the image features of different regions of the first sample image features and the image features of different regions of the second sample image features. For details, please refer to Figure 4a , Figure 4a A schematic flow chart of the steps of constructing a model loss value by regional weights provided in an embodiment of the present application, specifically, including steps S410 to S440:

[0111] S410: Determine a first model loss value of the initial feature extraction model according to a first sample portrait feature in the first sample image features and a second sample portrait feature in the second sample image features.

[0112] S420: Determine a second model loss value of the initial feature extraction model according to a first sample background feature in the first sample image feature and a second sample background feature in the second sample image feature.

[0113] S430: Determine a third model loss value of the initial feature extraction model according to a first sample reference feature in the first sample image feature and a second sample reference feature in the second sample image feature.

[0114] S440: Determine a model loss value of the initial feature extraction model according to the first model loss value, the second model loss value and the third model loss value.

[0115] In the embodiment of the present application, in combination with the above-mentioned related description, it can be known that the first sample portrait feature refers to the image feature of the portrait area in the first sample segmented image, the first sample background feature refers to the image feature of the background area in the first sample segmented image, and the first sample reference feature refers to the image feature of the reference area in the first sample segmented image. Similarly, the second sample portrait feature refers to the image feature of the portrait area in the second sample segmented image, the second sample background feature refers to the image feature of the background area in the second sample segmented image, and the second sample reference feature refers to the image feature of the reference area in the second sample segmented image. Therefore, the model loss values ​​of different areas can be determined based on the feature similarity between the image features of the corresponding areas in the first sample segmented image and the second sample segmented image. Among them, for multiple different positive sample document images, the model loss value is usually positively correlated with the feature similarity between the image features of multiple corresponding areas in the multiple first sample image features, and negatively correlated with the feature similarity between the image features of the corresponding areas in the first sample image features and the second sample image features.

[0116] Specifically, to facilitate understanding of the above content, the following will provide specific implementation steps for calculating the model loss value in combination with a specific calculation formula, where two samples a are randomly selected from the sample space. i and b i , forming the i-th group of sample pairs. At this time, the sample pairs may consist of the following situations: two negative samples, two positive samples, or one positive sample and one negative sample. i Constitute a set A, and multiple samples b i Constitute the set B, that is, {a1,a2,...a N}∈A,{b1,b2,...b N}∈B.

[0117] Furthermore, f S1 (a i ), f S2 (a i ) and f S3 (a i ) represent the feature extraction model for the S1 region, S2 region, and S3 region, that is, the sample document image a i The image features are obtained by processing the portrait area, background area and reference area. Similarly, f S1 (b i ), f S2 (b i ) and f S3 (b i ) represent the feature extraction model for the sample document image b i The image features are obtained by processing the portrait area, background area and reference area. At this time, the first model loss value L S1, the second model loss value L S2 And the third model loss value L S3 The calculation formula is as follows:

[0118]

[0119] Among them, θ, γ is a preset hyperparameter used to control the minimum distance between pairs of samples of different classes to prevent them from being too close, N is the total number of input image groups, [·] + represents a non-negative value, y is whether the two ID images in the i-th group of images match the binary label, that is, the sample ID image a i and b i For the same type of ID images, for example, when they are all positive ID images without color cast, y = 1, and the sample ID image a i and b i For different types of document images, for example, when one is a positive sample document image without color cast and the other is a negative sample document image with color cast, y=0.

[0120] On the basis of obtaining the model loss values ​​describing different regions, in order to make the model pay attention to the differences between different regions during the training process, the model loss values ​​of different regions can be weighted and fused to obtain the model loss value of the initial feature extraction model, that is, the model loss value L of the feature extraction model. tol The specific calculation formula is:

[0121]

[0122] st(α+β+λ)=1

[0123] Among them, α, β and λ are the weight parameters corresponding to the losses of three different regions. In specific application scenarios, α, β and λ can be set based on actual needs, but in order to ensure the training effect of the model, as a common feasible implementation scheme, α is set to 0.7, β is set to 0.2, and λ is set to 0.1, which can effectively ensure that the model can recognize the color cast of the image according to the differences in different regions, thereby maximizing the training effect of the model.

[0124] Of course, as a further implementation scheme of the present application, the weight coefficient provided above can also be dynamically updated and adjusted based on the training rounds of the model, that is, the model loss value corresponding to the initial feature extraction model is determined according to the first model loss value, the second model loss value and the third model loss value, specifically including:

[0125] Determining a training round corresponding to the initial feature extraction model;

[0126] The first model loss value, the second model loss value, and the third model loss value are weighted according to the weight coefficient corresponding to the training round to obtain a model loss value corresponding to the initial feature extraction model.

[0127] For example, as a common feasible implementation scheme, when the number of training rounds is small, that is, in the first few training rounds, the weights of the model loss values ​​of different regions can be set more evenly, for example, α=β=λ=0.333 is set, so as to ensure that the model can ignore the influence of the region in the initial training rounds, so as to quickly learn the difference in feature information of the image and improve the model training efficiency. At the same time, in order to further enable the model to pay attention to the differences between different regions during the training process, as the subsequent training rounds gradually increase, the weight of the first model loss value can be gradually increased, for example, from 0.33 to 0.7, and the weights of the second model loss value and the third model loss value can be gradually reduced, for example, from 0.333 to 0.2, 0.1, etc., respectively, so as to further improve the model's attention to the color information of key areas in the image and ensure the model's recognition effect on color cast.

[0128] Specifically, in order to understand the above training process, please refer to Figure 4b , Figure 4b A schematic diagram of the effect of a training feature extraction model provided in an embodiment of the present application is described in detail as follows.

[0129] It can be seen that during the training process, the sample pairs used come from different sample sets A and B, where these sample pairs may be two negative sample ID images, two positive sample ID images, or one positive sample ID image and one negative sample ID image, and the figure takes one positive sample ID image and one negative sample ID image as an example for explanation. At this time, the sample pairs will pass through the first network layer of the feature extraction model, that is, the segmentation model, to implement the image segmentation processing to obtain the segmented region image, and since the segmented region image can be understood as the feature mask image of each region, the color feature extraction of the segmented region image can also be understood as first passing through the second network layer of the feature extraction model, that is, the color feature extractor to extract the color features of the entire image, and then through the mask image corresponding to the segmented region image to perform mask downsampling processing on the processed overall image color features f(A) and f(B), thereby obtaining the image features of each region, that is, f S1 (A), f S2 (A) and f S3 (A), and f S1 (B) f S2 (B) and f S3(B) Then, based on the loss function provided above, the segmentation model (first network layer) and the color feature extractor (second network layer) are trained until the final feature extraction model is obtained.

[0130] By constructing the model loss value through the aforementioned designed regional weights for model training, the model can focus on the color deviations of different regions in the image, thereby further improving the model's recognition effect on image color cast.

[0131] Of course, in addition to the aforementioned use of a single color feature extractor to respectively process the segmented area images of different areas, as a further optional implementation scheme of the present application, the color feature extractor may also include multiple color feature extractors at the same time, for example, a skin color feature extractor (portrait area color feature extractor), a background color feature extractor, and other area color feature extractors (background area color feature extractor) may be included to process different image areas respectively. For details, please refer to Figure 4c , Figure 4c A schematic diagram of the effect of multiple different color feature extractors provided in the embodiment of the present application processing images of different regions respectively to obtain image features. It can be seen that by processing the portrait region, background region and reference region respectively by the different color feature extractors provided above, feature vectors of each region can be obtained, and then these feature vectors are spliced ​​and fused according to certain rules to obtain the final image feature vector, that is, the color feature representation vector.

[0132] S250, updating the initial feature extraction model according to the model loss value until the model loss value obtained by inputting the sample document image into the updated feature extraction model meets a preset condition, and determining the current updated feature extraction model as the trained feature extraction model.

[0133] In the embodiment of the present application, after determining the loss value of the feature extraction model as mentioned above, based on the idea of ​​back propagation, the parameters of the initial feature extraction model are optimized using the model loss value, that is, the initial feature extraction model is updated, so that the updated feature extraction model can shorten the feature distance between the positive sample ID images without color cast, and increase the feature distance between the positive sample ID images without color cast and the negative sample ID images with color cast and the sample portrait base map, that is, the model loss value obtained by inputting the sample ID image into the updated feature extraction model will gradually decrease, and the color cast recognition effect of the model will be more ideal. Through multiple repeated training processes, that is, the feature extraction model is continuously updated and optimized based on the latest model loss value, until the model loss value obtained by inputting the sample ID image into the updated feature extraction model meets the preset conditions, for example, if it is less than the preset loss value threshold for several consecutive rounds, it can be considered that the feature extraction model has been optimized. At this time, the current feature extraction model can be determined as a trained feature extraction model to be used for feature extraction of the ID image to be identified in the color cast recognition process to complete the color cast recognition.

[0134] It can also be seen that through the aforementioned training method, there is no need to use a sample portrait background map during the entire training process, and a feature extraction model with a sufficiently ideal color cast recognition effect can be trained, which is particularly suitable for color cast recognition scenarios where there is no portrait background map.

[0135] S140: Determine a color cast recognition result of the document image according to the first image feature and a second image feature corresponding to the historical document image.

[0136] In the embodiment of the present application, similar to the first image feature obtained by processing the document image, the second image feature corresponding to the historical document image can be obtained by performing the same processing on the historical document image. That is, the second image feature corresponding to the historical document image can usually be obtained by the following steps:

[0137] Processing the historical document image through the first network layer in the trained feature extraction model to obtain a plurality of second segmented region images in the historical document image;

[0138] The plurality of second segmented region images are processed by the second network layer in the feature extraction model to obtain second image features corresponding to the historical document image.

[0139] The step of processing the plurality of second segmented region images through the second network layer in the feature extraction model to obtain the second image features corresponding to the historical document image may include:

[0140] Processing the second portrait region image in the second segmented region image through the second network layer in the feature extraction model to obtain a second portrait feature;

[0141] Processing the second background region image in the second segmented region image through the second network layer to obtain a second background feature;

[0142] Processing a second reference region image in the second segmented region image through the second network layer to obtain a second reference feature; the second reference region image is used to indicate a clothing region and / or a hairstyle region in the portrait base image;

[0143] The second portrait feature, the second background feature and the second reference feature are fused to obtain a second image feature corresponding to the document image.

[0144] It can be understood that since the first image and the second image features are obtained by the model by focusing on different areas in the image and focusing on the difference in the degree of influence of different areas on the significant color cast of printing on the document image and the historical document image, the color cast recognition result of the document image can be determined according to the feature similarity or feature distance between the first image feature and the second image feature. For example, as a most common feasible implementation scheme, when the historical document image includes a plurality of negative historical document images with color cast, by comparing the feature similarity between the first image feature and the second image feature with a preset similarity threshold, when the feature similarity between the first image feature and the second image feature is higher than the preset similarity threshold, it can be considered that the color features in the specific area of ​​the document image and the historical document image with color cast are relatively similar, and at this time, it can be considered that the color cast recognition result of the document image is that there is color cast, and conversely, when the feature similarity between the first image feature and the second image feature is lower than the preset similarity threshold, it can be considered that the color cast recognition result of the document image is that there is no color cast. It should be noted that the similarity threshold here can be dynamically set based on the actual color cast tolerance requirement. When the color cast tolerance is low, the similarity threshold can be set higher, and vice versa, the similarity threshold can be set lower, thereby satisfying the dynamic adjustment of different color cast tolerances for different document portraits.

[0145] Of course, in the actual process, in order to further improve the accuracy of the identification result of the color cast of the document image, considering that the historical document images may include historical document images with or without color cast and historical document images with color cast, the second image feature of the historical document image may further include the third image feature of the historical document image with color cast and the fourth image feature of the historical document image without color cast. Figure 5 , Figure 5Another step flow diagram of determining a color cast recognition result provided in an embodiment of the present application, specifically, includes steps S510 to S530:

[0146] S510: Determine a first feature similarity between the first image feature and the third image feature.

[0147] S520: Determine a second feature similarity between the first image feature and the fourth image feature.

[0148] In the embodiment of the present application, the feature similarity between the first image feature and the third image feature can be understood as the similarity of color features between the certificate image and the historical certificate image with color cast. Specifically, the higher the feature similarity, the more likely the certificate image is to have color cast. Correspondingly, the feature similarity between the first image feature and the fourth image feature can be understood as the similarity of color features between the certificate image and the historical certificate image without color cast. Specifically, the higher the feature similarity, the more likely the certificate image is to have no color cast.

[0149] S530: Determine a color cast recognition result of the document image according to the first feature similarity and the second feature similarity.

[0150] In the embodiment of the present application, in combination with the above-mentioned related description, it can be known that by comparing the size relationship between the first feature similarity and the second feature similarity, it can be further determined whether the document image tends to have color cast or tends to not have color cast. For example, as a common feasible implementation scheme, when the first feature similarity is greater than the second feature similarity, it can be considered that the document image is more inclined to have color cast, and vice versa, it can be considered that the document image is more inclined to have no color cast. Alternatively, the difference between the first feature similarity and the second feature similarity (considering positive and negative) can also be compared with the size relationship of the preset threshold. For example, when the difference is greater than the preset threshold, it can be considered that the document image tends to have color cast, and when the difference is less than the preset threshold, it can be considered that the document image tends to have no color cast.

[0151] Of course, considering that there may be multiple historical document images with and without color cast, therefore, in order to further improve the color cast recognition result of the document image, the first feature similarity may also be the average similarity (i.e., average distance) of the feature similarities between the first image feature and multiple third image features, and the second feature similarity may be the average similarity of the feature similarities between the first image feature and multiple fourth image features. Of course, it is also possible to further select K image features with higher similarity to the first image feature from the multiple third image features, and then calculate the average similarity between these K image features and the first image feature as the first feature similarity, and similarly for the second feature similarity, it can be determined by first selecting K image features with higher similarity to the first image feature from the multiple fourth image features.

[0152] In addition, similar to the aforementioned dynamic setting of the similarity threshold based on color cast tolerance, in the embodiment of the present application, since the acquired historical document images are usually images that are manually determined to have color cast, the flexibility of color cast recognition can also be improved to a certain extent based on the different existing color cast images selected by the user.

[0153] The color cast recognition method based on regional weights provided in the embodiment of the present application first processes the certificate image to obtain a certificate image to be recognized and multiple historical certificate images to obtain multiple segmented region images corresponding to different subject information in the certificate image, and then processes the multiple different segmented region images to obtain image features focusing on different regions, thereby determining the color cast recognition result of the certificate image through the similarity between the image features corresponding to the multiple historical certificate images. Compared with color cast recognition from the entire image, color cast feature analysis based on the different significance of different segmented regions can ensure better color cast recognition effect, especially in color cast recognition scenarios where there is a lack of a portrait base map as a comparison reference.

[0154] In order to facilitate understanding of the color cast identification method provided in the embodiment of the present application, the following will be combined with the aforementioned Figure 1a to Figure 5 The content provided provides a complete implementation scenario of a color cast identification method, which includes model training and model application stages. Specifically, it includes the following steps:

[0155] (1) A triplet of data consisting of a sample portrait base image, a positive ID image, and a negative ID image is constructed from the portrait dataset. Specifically, a photo of a person used for printing is used as the sample portrait base image, and a printed ID image without color cast is used as the positive ID image. The negative ID image is a printed ID image with color cast.

[0156] (2) Considering that in the actual printing process, the printed color-biased image and the printed color-unbiased image are generally mutually exclusive, that is, in the process of printing, if the printed document has no color-bias or other printing quality problems, it will not be printed repeatedly. Compared with the printed color-unbiased image, the printed color-biased image is also less likely to be generated. Therefore, in order to construct a rich triplet data combination, the negative sample document images of different color-biased categories can be expanded;

[0157] (3) Considering that printing uses CMYK color space, the positive sample ID image in RGB color space can be converted into a reference ID image in CMYK color space, and then the value range of each color channel is adjusted in units of 20 to have 6 different values ​​(for example: 0, 20, 40, ... 100). In this way, a certain number of color cast images can be generated as negative sample ID images;

[0158] (4) After the samples are ready, a color feature extractor based on a convolutional neural network is constructed. The color feature extractor mainly includes a two-layer structure. The first layer is used to complete the segmentation of the portrait. Specifically, an encoder-decoder structure can be used to process the image step by step. The encoder part consists of multiple convolution blocks and downsampling layers. Each convolution block contains two 3x3 convolution layers, followed by a ReLU activation function, and a 2x2 maximum pooling layer is used for downsampling. Through this structure, the spatial size of the feature map is reduced layer by layer, while the abstraction level of the feature is increased. For example, the initial size of the input image is 416*416*3 (h*w*3). After four downsamplings, the size of the feature map becomes 208*208*64, 104*104*128, 52*52*256, and finally reaches 26*26*512 at the end of the encoder. The decoder part gradually restores the spatial size of the image through upsampling and convolution operations, and combines the feature map of the corresponding layer of the encoder to enhance the detail information of the feature map through jump connections. Finally, the decoder outputs a feature map of size (h*w*3), in which the three channels represent the probability values ​​of the portrait, background, and other reference areas respectively. The three channel values ​​of each pixel are processed by the softmax function, and the channel with the maximum probability value is selected as the classification result of the pixel, generating a segmentation map of size h*w, in which each pixel is marked as a portrait, background, and other reference areas, as a mask image for subsequent processing.

[0159] The second layer is mainly used to extract color feature information, mainly including a twin color feature extractor with a segmentation mask, which inputs each data pair to extract the color features and other information in the ID portrait in a region-by-region manner. The twin color feature extractor consists of two parallel sub-networks with shared weights and a mask network. For details, please refer to Figure 6a , Figure 6aA schematic diagram of the effect of a network structure provided for an embodiment of the present application, wherein the subnetwork is a model of feature extraction and fusion based on a convolutional neural network, including 13 convolutional layers and 5 maximum pooling layers. The convolutional layer uses a 3×3 convolution kernel, and a ReLU activation function is used after each convolutional layer. The convolutional layer is stacked to form 5 convolutional blocks, and a 2×2 maximum pooling layer is used at the end of each block to reduce the size of the feature map by half. Then the features of the intermediate layer convolution block 2 and the convolution block 5 are extracted, and a suitable interpolation method is selected to upsample the convolution block 5 so that it can be added to the convolution block 2 for feature fusion, thereby constructing a feature representation that can effectively extract the color information of the portrait, which is used for processing each area image, and the mask network does not participate in network training, and adjusts the feature map size as the subnetwork downsamples or upsamples to match the different feature map sizes output by the subnetwork;

[0160] (5) After inputting each group of input images into the parameter-initialized feature extraction model, feature vectors of different regions can be obtained, such as the first sample portrait feature, the first sample background feature and the first sample reference feature of the sample portrait base map, and the second sample portrait feature, the second sample background feature and the second sample reference feature of the sample ID image. Then, the model loss value is constructed using the regional weights, and the training weights of the color feature extractor are continuously back-propagated and updated, so that the feature vectors of samples of the same category are closer and the feature vectors of samples of different categories are farther, so that it can accurately identify whether the printed ID portrait has a color cast problem, thereby improving the model's color cast recognition accuracy. The feature vector distance here can be Euclidean distance, cosine distance or Mahalanobis distance;

[0161] (6) After the feature extraction model is trained, the obtained feature extraction model can be used for subsequent color cast identification. Figure 6b , Figure 6b The schematic diagram of the effect of the subsequent color cast recognition application scenario provided for the embodiment of the present application shows that by inputting the document image to be identified, including the historical document images with and without color cast, into the feature extraction model, the color feature vector of the image to be identified and the set of historical image feature vectors can be extracted respectively. Then, the color feature vector of the image to be identified and the color feature vector in the set of historical image feature vectors are compared, and the Euclidean distance or cosine similarity between the feature vectors and the average distance between the topK, that is, the K document images with the highest similarity or the closest distance in the historical document images are calculated respectively. On this basis, by comparing the weighted sizes of the two, it is further determined whether the document image to be identified has color cast.

[0162] In order to better implement the color cast identification method based on regional weights provided in the embodiment of the present application, on the basis of the color cast identification method based on regional weights provided in the embodiment of the present application, a color cast identification device based on regional weights is also provided in the embodiment of the present application, such as Figure 7 As shown, the color cast identification device comprises:

[0163] An acquisition module 710 is used to acquire a document image to be identified and a plurality of historical document images;

[0164] A segmentation module 720, configured to process the document image through the first network layer in the trained feature extraction model to obtain a plurality of segmented region images in the document image;

[0165] A processing module 730 is used to process the document image through the first network layer in the trained feature extraction model to obtain multiple segmented area images in the document image;

[0166] The recognition module 740 is used to determine the color cast recognition result of the document image according to the first image feature and the second image feature corresponding to the historical document image.

[0167] In some embodiments of the present application, the processing module 730 is used to process the portrait region image in the segmented region image through the second network layer in the feature extraction model to obtain a first portrait feature;

[0168] Processing the background area image in the segmented area image through the second network layer to obtain a first background feature;

[0169] Processing a reference area image in the segmented area image through the second network layer to obtain a first reference feature; the reference area image is used to indicate a clothing area and / or a hairstyle area in the document image;

[0170] The first portrait feature, the first background feature and the first reference feature are fused to obtain a first image feature corresponding to the document image.

[0171] In some embodiments of the present application, the feature extraction model is obtained by training the processing module 730 through the following steps:

[0172] Acquire a plurality of sample ID images; the sample ID images include positive sample ID images and negative sample ID images;

[0173] Inputting the sample ID image into the first initial network layer in the initial feature extraction model to obtain a sample segmentation image of the sample ID image;

[0174] Inputting the sample segmented image into the second initial network layer in the initial feature extraction model to obtain the first sample image feature of the positive sample ID image and the second sample image feature of the negative sample ID image;

[0175] Determining a model loss value of the initial feature extraction model according to the first sample image feature and the second sample image feature;

[0176] The initial feature extraction model is updated according to the model loss value until the model loss value obtained by inputting the sample document image into the updated feature extraction model meets a preset condition, and the current updated feature extraction model is determined as the trained feature extraction model.

[0177] In some embodiments of the present application, the processing module 730 is further used to determine a first model loss value of the initial feature extraction model according to a first sample portrait feature in the first sample image feature and a second sample portrait feature in the second sample image feature;

[0178] Determine a second model loss value of the initial feature extraction model according to a first sample background feature in the first sample image feature and a second sample background feature in the second sample image feature;

[0179] Determine a third model loss value of the initial feature extraction model according to a first sample reference feature in the first sample image feature and a second sample reference feature in the second sample image feature;

[0180] A model loss value of the initial feature extraction model is determined according to the first model loss value, the second model loss value and the third model loss value.

[0181] In some embodiments of the present application, the processing module 730 is further used to determine the training round corresponding to the initial feature extraction model;

[0182] The first model loss value, the second model loss value, and the third model loss value are weighted according to the weight coefficient corresponding to the training round to obtain a model loss value corresponding to the initial feature extraction model.

[0183] In some embodiments of the present application, the acquisition module 710 is further used to determine a positive sample document image in the first color space corresponding to the sample portrait base image;

[0184] Converting the positive sample document image into a reference document image in a second color space;

[0185] The values ​​of each color channel in the reference document image are adjusted to obtain a negative sample document image.

[0186] In some embodiments of the present application, the recognition module 740 is further used to determine a first feature similarity between the first image feature and the third image feature;

[0187] determining a second feature similarity between the first image feature and the fourth image feature;

[0188] The color cast recognition result of the document image is determined according to the first feature similarity and the second feature similarity.

[0189] The specific definition of the color cast identification device can be found in the definition of the color cast identification method above, which will not be repeated here. Each module in the above-mentioned color cast identification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0190] In some embodiments of the present application, the portrait color cast recognition device can be implemented in the form of a computer program. The computer program can be used in Figure 8 The computer device is run on the computer device shown. The memory of the computer device can store various program modules that constitute the portrait color cast recognition device, such as: Figure 7 The acquisition module 710, segmentation module 720, processing module 730 and identification module 740 are shown. The computer program composed of various program modules enables the processor to execute the steps of the portrait color cast identification method of each embodiment of the present application described in this specification.

[0191] For example, Figure 8 The computer device shown can be Figure 7 The acquisition module 710 in the color cast identification device shown executes step S110. The computer device can execute step S120 through the segmentation module 720. The computer device can execute step S130 through the processing module 730. The computer device can execute step S140 through the identification module 740. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external computer device through a network connection. When the computer program is executed by the processor, a method for identifying color cast of a portrait is implemented.

[0192] Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0193] In some embodiments of the present application, a computer device is provided, comprising one or more processors; a memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the following steps:

[0194] Obtaining a document image to be identified and multiple historical document images;

[0195] Processing the document image through the first network layer in the trained feature extraction model to obtain multiple segmented area images in the document image;

[0196] Processing the plurality of segmented region images through the second network layer in the feature extraction model to obtain a first image feature corresponding to the document image;

[0197] A color cast recognition result of the document image is determined according to the first image feature and a second image feature corresponding to the historical document image.

[0198] In some embodiments of the present application, a computer-readable storage medium is provided, which stores a computer program. The computer program is loaded by a processor, so that the processor performs the following steps:

[0199] Obtaining a document image to be identified and multiple historical document images;

[0200] Processing the document image through the first network layer in the trained feature extraction model to obtain multiple segmented area images in the document image;

[0201] Processing the plurality of segmented region images through the second network layer in the feature extraction model to obtain a first image feature corresponding to the document image;

[0202] A color cast recognition result of the document image is determined according to the first image feature and a second image feature corresponding to the historical document image.

[0203] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Any reference to memory, storage, information library or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0204] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0205] The above is a detailed introduction to a color cast identification method, device, computer equipment and storage medium based on regional weights provided in an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A color cast identification method based on regional weight, characterized in that: include: Obtaining a document image to be identified and multiple historical document images; Processing the document image through the first network layer in the trained feature extraction model to obtain multiple segmented area images in the document image; Processing the plurality of segmented region images through the second network layer in the feature extraction model to obtain a first image feature corresponding to the document image; A color cast recognition result of the document image is determined according to the first image feature and a second image feature corresponding to the historical document image.

2. The method according to claim 1, characterized in that The step of processing the plurality of segmented region images through the second network layer in the feature extraction model to obtain a first image feature corresponding to the document image includes: Processing the portrait region image in the segmented region image through the second network layer in the feature extraction model to obtain a first portrait feature; Processing the background area image in the segmented area image through the second network layer to obtain a first background feature; Processing a reference area image in the segmented area image through the second network layer to obtain a first reference feature; the reference area image is used to indicate a clothing area and / or a hairstyle area in the document image; The first portrait feature, the first background feature and the first reference feature are fused to obtain a first image feature corresponding to the document image.

3. The method according to claim 2, characterized in that The feature extraction model is trained by the following steps: Acquire a plurality of sample ID images; the sample ID images include positive sample ID images and negative sample ID images; Inputting the sample ID image into the first initial network layer in the initial feature extraction model to obtain a sample segmentation image of the sample ID image; Inputting the sample segmented image into the second initial network layer in the initial feature extraction model to obtain the first sample image feature of the positive sample ID image and the second sample image feature of the negative sample ID image; Determining a model loss value of the initial feature extraction model according to the first sample image feature and the second sample image feature; The initial feature extraction model is updated according to the model loss value until the model loss value obtained by inputting the sample document image into the updated feature extraction model meets a preset condition, and the current updated feature extraction model is determined as the trained feature extraction model.

4. The method according to claim 3, characterized in that The determining the model loss value of the initial feature extraction model according to the first sample image feature and the second sample image feature includes: Determine a first model loss value of the initial feature extraction model according to a first sample portrait feature in the first sample image feature and a second sample portrait feature in the second sample image feature; Determine a second model loss value of the initial feature extraction model according to a first sample background feature in the first sample image feature and a second sample background feature in the second sample image feature; Determine a third model loss value of the initial feature extraction model according to a first sample reference feature in the first sample image feature and a second sample reference feature in the second sample image feature; A model loss value of the initial feature extraction model is determined according to the first model loss value, the second model loss value and the third model loss value.

5. The method according to claim 4, characterized in that The determining the model loss value corresponding to the initial feature extraction model according to the first model loss value, the second model loss value, and the third model loss value includes: Determining a training round corresponding to the initial feature extraction model; The first model loss value, the second model loss value, and the third model loss value are weighted according to the weight coefficient corresponding to the training round to obtain a model loss value corresponding to the initial feature extraction model.

6. The method according to claim 3, characterized in that: The step of obtaining a plurality of sample document images comprises: Determine a positive sample document image in a first color space corresponding to the sample portrait base image; Converting the positive sample document image into a reference document image in a second color space; The values ​​of each color channel in the reference document image are adjusted to obtain a negative sample document image.

7. The method according to any one of claims 1 to 6, characterized in that: The second image feature corresponding to the historical document image includes the third image feature of the historical document image with color cast and the fourth image feature of the historical document image without color cast; The determining, according to the first image feature and the second image feature corresponding to the historical document image, a color cast recognition result of the document image includes: determining a first feature similarity between the first image feature and the third image feature; determining a second feature similarity between the first image feature and the fourth image feature; The color cast recognition result of the document image is determined according to the first feature similarity and the second feature similarity.

8. A color cast identification device based on regional weight, characterized in that: include: An acquisition module, used to acquire an image of a certificate to be identified and multiple historical certificate images; A segmentation module, used for processing the document image through the first network layer in the trained feature extraction model to obtain a plurality of segmented area images in the document image; A processing module, used for processing the plurality of segmented region images through the second network layer in the feature extraction model to obtain a first image feature corresponding to the document image; The recognition module is used to determine the color cast recognition result of the document image according to the first image feature and a second image feature corresponding to the historical document image.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and are configured to be executed by the processor to implement the color cast identification method based on regional weights as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the color cast identification method based on regional weights as described in any one of claims 1 to 7.

Images