Eyebrow Occlusion Detection Method Based on Image Analysis and Color Features

Abstract

This invention discloses an eyebrow occlusion detection algorithm based on image analysis and color features. This algorithm belongs to the field of image processing technology and is mainly used to detect whether eyebrows are occluded in ID photo images. The algorithm first preprocesses the input image, specifically performing face detection and keypoint detection operations to obtain a set of facial keypoints. Based on this set, a facial feature mask image is generated, and image analysis is performed to obtain an image analysis mask image, defining an eyebrow occlusion judgment value. Next, based on the generated results, it determines whether there is an intersection between the hair and eyebrow regions to initially identify whether the eyebrows are occluded; based on the keypoints, the bounding rectangle of the eyebrows is calculated, and the proportion of hair pixels is checked to determine whether the eyebrows are completely unoccluded. If the determination cannot be completed, the color space is converted, a fine mask is extracted through clustering, and the color difference between the eyebrows and the surrounding areas is calculated to further evaluate the eyebrow occlusion situation.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to an eyebrow occlusion detection method based on image analysis and color features. Background Technology

[0002] With the continuous advancement of image processing technology, its application in areas such as ID photo taking has become increasingly important. ID photos refer to official photographs used on various documents, such as ID cards, passports, driver's licenses, student IDs, and other official documents. Relevant document-issuing departments have clear requirements regarding the presentation of an individual's facial features, and eyebrows, as one of the most important facial features, are crucial for accurately detecting whether the eyebrow area is obscured, thus ensuring the photo meets the requirements for ID photos.

[0003] Traditional eyebrow occlusion detection methods have limitations in accuracy, efficiency, and adaptability, especially when dealing with complex scenes or detailed features. While color clustering methods can initially distinguish between hair and skin areas, their accuracy is affected by complex background colors or special hair colors (such as dyed hair), potentially failing to accurately separate eyebrow and hair areas and thus impacting the accuracy of occlusion assessment. Furthermore, traditional methods rely on facial landmarks for eyebrow region localization and analysis. However, when facial occlusion is severe or landmark detection is inaccurate, the entire eyebrow occlusion detection process is easily disrupted, leading to reduced reliability of the results. Summary of the Invention

[0004] The purpose of this invention is to provide an eyebrow occlusion detection method based on image analysis and color features, which can solve the problem of misjudgment that may occur when the boundary between eyebrows and hair is blurred or when encountering other interfering objects, while reducing the dependence on facial key points.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] In a first aspect, the present invention provides a method for detecting eyebrow occlusion based on image analysis and color features, comprising the following steps:

[0007] The input image is preprocessed to obtain a set of facial key points, a facial feature mask, and a portrait parsing mask, and the eyebrow occlusion judgment value is initially defined.

[0008] Based on the facial feature mask image and the portrait parsing mask image, determine whether there is obvious occlusion. If so, determine the eyebrow occlusion score.

[0009] Based on the set of facial key points and the image parsing mask, determine whether the eyebrows are completely unobstructed. If they are completely unobstructed, determine the eyebrow occlusion score.

[0010] If the eyebrow occlusion judgment value does not change after the above judgment, the input image is converted to the target color space to obtain the processed image;

[0011] Extract a fine eyebrow mask from the processed image;

[0012] A mask image of the outer region of the eyebrows is generated based on the set of facial key points, and a fine mask image of the outer region of the eyebrows is obtained by processing the mask image of the outer region of the eyebrows.

[0013] Based on the fine eyebrow mask and the fine eyebrow outer region mask image, color differences are calculated to determine the eyebrow occlusion score, which is used to quantitatively evaluate whether the ID photo meets the usage standards.

[0014] In one possible implementation, the preprocessing of the input image includes:

[0015] Perform face detection on the input image to obtain face detection bounding boxes;

[0016] Based on the face detection box, face key point detection is performed on the input image to obtain the face key point set;

[0017] From the set of facial key points, select the point sets corresponding to the facial features, and divide the contour point sets of different parts;

[0018] Based on the set of contour points, draw corresponding convex polygons, assign unique labels to different convex polygons, and generate the facial feature mask image.

[0019] Perform portrait parsing processing on the input image to obtain the portrait parsing mask image labeled with multiple semantic categories;

[0020] Initialize the eyebrow occlusion judgment value and set it to 0.

[0021] In one possible implementation, the step of filtering the set of points corresponding to the facial features from the set of facial key points includes:

[0022] Extract the point sets corresponding to the left and right eyebrows from the set of facial key points, and determine them as the eyebrow outer contour point set;

[0023] Extract the point set corresponding to other facial features and use it as the outline point set of other facial features.

[0024] In one possible implementation, determining whether there is obvious occlusion based on the facial feature mask image and the portrait parsing mask image includes:

[0025] Extract the eyebrow region from the facial feature mask;

[0026] Extract the hair region from the image parsing mask;

[0027] Calculate the intersection of the eyebrow region and the hair region;

[0028] If the area of ​​the intersection exceeds the proportion of the total area of ​​the eyebrow region to a first preset threshold, it is determined that there is obvious occlusion.

[0029] Set the eyebrow occlusion score to 0 and update the eyebrow occlusion judgment value to 1.

[0030] In one possible implementation, determining whether the eyebrows are completely unobstructed based on the set of facial key points and the image parsing mask includes:

[0031] The bounding rectangle of the eyebrow region is determined based on the eyebrow outer contour point set of the facial key point set.

[0032] Based on the bounding rectangle, extract the eyebrow region parsing mask of the corresponding area from the portrait parsing mask;

[0033] Count the number of pixels belonging to the hair category in the parsed mask image of the eyebrow region;

[0034] Calculate the proportion of the number of pixels to the total number of pixels in the eyebrow region parsed mask image;

[0035] If the ratio does not exceed the second preset threshold, it is determined that the eyebrows are not obscured at all;

[0036] Set the eyebrow occlusion score to 1 and update the eyebrow occlusion judgment value to 1.

[0037] In one possible implementation, converting the input image to the target color space includes:

[0038] The input image is converted from the first color space to the second color space to obtain the processed image.

[0039] In one possible implementation, extracting the fine eyebrow mask from the processed image includes:

[0040] Based on the eyebrow convex polygon in the facial feature mask image, the eyebrow area is selected on the processed image.

[0041] Extract color features from all pixels in the eyebrow region;

[0042] A clustering algorithm is performed on the color features to divide the pixels into two categories;

[0043] Calculate the average value of the brightness channel of the two types of pixels in the second color space respectively;

[0044] The category with the smaller average value of the brightness channels is identified as the target eyebrow pixels;

[0045] The fine eyebrow mask is generated based on the position information of the target eyebrow pixels.

[0046] In one possible implementation, generating the eyebrow outer region mask map based on the facial key point set includes:

[0047] Extract the left and right eyebrow outer contour point sets from the facial key point set;

[0048] Calculate the center point of the outer contour point set of the left and right eyebrows respectively;

[0049] Using the center point as a reference, extend the outer contour points of the left and right eyebrows outward by a preset distance;

[0050] Draw the outer region based on the expanded point set and generate an initial outer region mask;

[0051] The initial outer region mask is compared with the eyebrow region of the facial feature mask to remove overlapping parts;

[0052] Perform a downward expansion operation on the mask after removing the overlapping parts to obtain the mask image of the outer region of the eyebrow.

[0053] In one possible implementation, the process of processing the eyebrow outer region mask image to obtain a refined eyebrow outer region mask image includes:

[0054] The mask image of the outer region of the eyebrow is divided into a left sub-mask and a right sub-mask along the vertical midline;

[0055] On the processed image, the effective regions of the left sub-mask and the right sub-mask are extracted respectively;

[0056] Calculate the histogram information of the two effective regions in the luminance channel of the second color space;

[0057] Based on the histogram information, the average brightness of the two effective regions is calculated respectively;

[0058] The sub-mask with the smaller average brightness value is determined as the mask map of the outer region of the fine eyebrow.

[0059] In one possible implementation, calculating the color difference based on the fine eyebrow mask and the fine eyebrow outer region mask image includes:

[0060] On the processed image, the effective area of ​​the eyebrow is determined based on the fine eyebrow mask;

[0061] Calculate the histogram information of the effective area of ​​the eyebrow in the three channels of the second color space;

[0062] Based on the histogram information, the mean values ​​of the three channels are calculated and combined to form the first color feature vector;

[0063] On the processed image, the effective outer region is determined based on the fine eyebrow outer region mask map;

[0064] Calculate the histogram information of the effective peripheral region in the three channels of the second color space;

[0065] Based on the histogram information, the mean values ​​of the three channels are calculated and combined to form a second color feature vector;

[0066] Using a preset color difference formula, the difference between the first color feature vector and the second color feature vector is calculated to obtain the color difference value of one eyebrow.

[0067] Perform the same operation on the other eyebrow to obtain the color difference value of the other eyebrow.

[0068] The smaller value among the color differences between the two eyebrows is selected as the eyebrow occlusion score.

[0069] In a second aspect, the present invention also provides an electronic device, including a processor and a memory;

[0070] The memory is used to store programs;

[0071] The processor executes the program to implement the method described above.

[0072] Thirdly, the present invention also provides a computer-readable storage medium storing a program that is executed by a processor to implement the methods described above.

[0073] Fourthly, the present invention also provides a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0074] Compared with the prior art, the advantages of this invention are as follows:

[0075] This invention comprehensively employs face analysis, facial landmark localization, and color feature analysis technologies to form a multi-level detection logic. By combining the image mask obtained from face analysis with the facial feature mask generated based on facial landmarks, it can first determine whether there is an intersection between the hair and eyebrow regions, quickly identifying obvious bangs occlusion. Then, based on facial landmarks, it calculates the bounding rectangle of the eyebrow region and combines this with the image mask to statistically analyze the pixel proportion of the hair region, determining whether the eyebrows are completely unoccluded. For cases where the first two steps cannot determine the issue, the image is further converted to a specific color space, and a clustering algorithm is used to extract a fine eyebrow mask. The degree of occlusion is then assessed by calculating the color difference between the eyebrows and the surrounding area.

[0076] Traditional eyebrow occlusion detection methods heavily rely on facial landmarks. When facial occlusion is severe or landmark detection is inaccurate, the detection process is easily disrupted. This algorithm, while relying on facial landmark localization, incorporates face analysis and color feature analysis techniques: face analysis can independently identify semantic categories such as hair and skin, providing additional evidence for occlusion judgment; color feature analysis, through color space transformation and clustering algorithms, can still accurately assess occlusion based on color differences even when there are slight deviations in landmark localization.

[0077] This invention optimizes the detection process through layered detection logic: first, preliminary judgment and complete unoccluded judgment are completed through face analysis and face key point localization. These two steps can be completed quickly based on the existing analysis and localization results without complex calculations, and can quickly filter out images with obvious occlusion and complete unocclusion; only images that cannot be judged in the first two steps are subjected to fine calculations such as color feature analysis.

[0078] The invention fully considers the differences in different application scenarios in its technical design: the portrait analysis model can accurately identify semantic categories such as background, clothing, hair and skin, and can cope with ID photo detection in different background environments; the color feature analysis stage can adapt to eyebrow region recognition in different hair color scenarios (including special hair colors such as dyed hair) through color space transformation and clustering algorithms; the layered detection logic can quickly handle conventional obvious occlusion or completely unoccluded scenarios, and can also accurately handle complex scenarios such as blurred boundaries.

[0079] The detection process of this invention simplifies operation through technological integration: from input image to output occlusion detection result, the entire process requires no manual intervention, only four automated steps: preprocessing, preliminary judgment, region judgment, and fine judgment. The preprocessing stage can automatically complete face detection, key point detection, portrait parsing, and mask image generation; each subsequent judgment stage is automatically calculated and judged based on the data generated in the previous steps, without the need for manually setting complex parameters or performing manual auxiliary recognition. Attached Figure Description

[0080] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0081] Figure 1 This is a flowchart illustrating the eyebrow occlusion detection method based on image analysis and color features in an embodiment of the present invention.

[0082] Figure 2 This is another flowchart illustrating the eyebrow occlusion detection method based on image analysis and color features in an embodiment of the present invention.

[0083] Figure 3 This is a visual representation of the facial feature mask generated in an embodiment of the present invention.

[0084] Figure 4 This is a visualization of the image mask for portrait analysis under obvious occlusion conditions in an embodiment of the present invention;

[0085] Figure 5 This is a visualization of the generated eyebrow region parsing mask in an embodiment of the present invention;

[0086] Figure 6 This is a comparison image of the eyebrow area mask before and after the fine eyebrow operation in an embodiment of the present invention.

[0087] Figure 7 This is a comparison of the mask images of the outer eyebrow region before and after the fine eyebrow outer region operation in an embodiment of the present invention. Detailed Implementation

[0088] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0089] Example:

[0090] It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0091] Figure 1 This is a flowchart illustrating the eyebrow occlusion detection method based on image analysis and color features in an embodiment of the present invention; see also Figure 1 This invention provides a method for detecting eyebrow occlusion based on image analysis and color features, including the following steps:

[0092] Step 1: Preprocess the input image to obtain a set of facial key points, a facial feature mask, and a portrait parsing mask, and preliminarily define the eyebrow occlusion judgment value.

[0093] In specific execution, the preprocessing of the input image includes: performing face detection on the input image to obtain a face detection bounding box; performing face key point detection on the input image based on the face detection bounding box to obtain the face key point set; selecting point sets corresponding to facial features from the face key point set and dividing the contour point sets of different parts; drawing corresponding convex polygons based on the contour point sets, assigning exclusive labels to different convex polygons, and generating the facial feature mask image; performing portrait parsing processing on the input image to obtain the portrait parsing mask image labeled with multiple semantic categories; and initializing the eyebrow occlusion judgment value by setting it to 0.

[0094] Furthermore, the step of selecting point sets corresponding to facial features from the set of facial key points includes: extracting point sets corresponding to the left and right eyebrows from the set of facial key points and determining them as the eyebrow outer contour point sets; extracting point sets corresponding to other facial features and using them as the other facial feature contour point sets.

[0095] Step 2: Based on the facial feature mask image and the portrait parsing mask image, determine whether there is obvious occlusion. If so, determine the eyebrow occlusion score.

[0096] In specific execution, determining whether there is obvious occlusion based on the facial feature mask image and the portrait parsing mask image includes: extracting the eyebrow region from the facial feature mask image; extracting the hair region from the portrait parsing mask image; calculating the intersection of the eyebrow region and the hair region; if the area of ​​the intersection accounts for more than a first preset threshold of the total area of ​​the eyebrow region, then it is determined that there is obvious occlusion; setting the eyebrow occlusion score to 0 and updating the eyebrow occlusion judgment value to 1.

[0097] Step 3: Based on the facial key point set and the portrait parsing mask, determine whether the eyebrows are completely unobstructed. If they are completely unobstructed, determine the eyebrow occlusion score.

[0098] In specific execution, determining whether the eyebrows are completely unobstructed based on the facial key point set and the portrait parsing mask includes: determining the circumscribed rectangle of the eyebrow region based on the eyebrow outer contour point set of the facial key point set; extracting the corresponding eyebrow region parsing mask from the portrait parsing mask based on the circumscribed rectangle; counting the number of pixels belonging to the hair category in the eyebrow region parsing mask; calculating the proportion of the number of pixels to the total number of pixels in the eyebrow region parsing mask; if the proportion does not exceed a second preset threshold, determining that the eyebrows are completely unobstructed; setting the eyebrow occlusion score to 1, and updating the eyebrow occlusion judgment value to 1.

[0099] Step 4: If the eyebrow occlusion judgment value does not change after the above judgment, the input image is converted to the target color space to obtain the processed image.

[0100] In specific execution, converting the input image to the target color space includes: converting the input image from the first color space to the second color space to obtain the processed image.

[0101] Furthermore, the first color space is the RGB color space, and the second color space is the LAB color space.

[0102] Step 5: Extract fine eyebrow masks from the processed image.

[0103] In specific execution, the step of extracting a fine eyebrow mask from the processed image includes: selecting an eyebrow region on the processed image based on the eyebrow convex polygon in the facial feature mask image; extracting color features from all pixels in the eyebrow region; performing a clustering algorithm on the color features to divide the pixels into two categories; calculating the average value of the brightness channel of the two categories of pixels in the second color space; determining the category with the smaller average value of the brightness channel as the target eyebrow pixel; and generating the fine eyebrow mask based on the position information of the target eyebrow pixel.

[0104] Furthermore, the clustering algorithm is the K-means clustering algorithm.

[0105] Step 6: Generate an eyebrow outer region mask image based on the facial key point set, and process the eyebrow outer region mask image to obtain a refined eyebrow outer region mask image.

[0106] In specific execution, generating the eyebrow outer region mask based on the facial key point set includes: extracting left and right eyebrow outer contour point sets from the facial key point set; calculating the center point of each left and right eyebrow outer contour point set; extending the left and right eyebrow outer contour point sets outward by a preset distance based on the center point; drawing the outer region based on the extended point set to generate an initial outer region mask; comparing the initial outer region mask with the eyebrow region of the facial feature mask image and removing overlapping parts; and performing a downward expansion operation on the mask after removing overlapping parts to obtain the eyebrow outer region mask image.

[0107] Furthermore, the preset distance is a preset number of pixel units.

[0108] Step 7: Based on the fine eyebrow mask and the fine eyebrow outer area mask image, calculate the color difference to determine the eyebrow occlusion score. The eyebrow occlusion score is used to quantitatively evaluate whether the ID photo meets the usage standards.

[0109] In specific execution, the process of processing the eyebrow outer region mask image to obtain a refined eyebrow outer region mask image includes: dividing the eyebrow outer region mask image into a left sub-mask and a right sub-mask along the vertical midline; extracting the effective regions of the left sub-mask and the right sub-mask respectively on the processed image; calculating the histogram information of the two effective regions in the luminance channel of the second color space; calculating the average luminance of the two effective regions based on the histogram information; and determining the sub-mask with the smaller average luminance value as the refined eyebrow outer region mask image.

[0110] Further, the calculation of color difference based on the fine eyebrow mask and the fine eyebrow outer region mask image includes: determining the effective eyebrow region on the processed image based on the fine eyebrow mask; calculating the histogram information of the effective eyebrow region in the three channels of the second color space; calculating the mean of the three channels based on the histogram information and combining them to form a first color feature vector; determining the outer effective region on the processed image based on the fine eyebrow outer region mask image; calculating the histogram information of the outer effective region in the three channels of the second color space; calculating the mean of the three channels based on the histogram information and combining them to form a second color feature vector; using a preset color difference formula, calculating the difference between the first color feature vector and the second color feature vector to obtain the color difference value of one eyebrow; performing the same operation on the other eyebrow to obtain the color difference value of the other eyebrow; selecting the smaller value among the color difference values ​​of the two eyebrows and determining it as the eyebrow occlusion score.

[0111] Furthermore, the preset color difference formula is the CIEDE2000 color difference formula.

[0112] As a preferred example, further explanation is provided below with reference to the accompanying drawings:

[0113] A method for eyebrow occlusion detection based on image analysis and color features. Figure 2 This is another flowchart illustrating the eyebrow occlusion detection method based on image analysis and color features according to an embodiment of the present invention, which includes the following steps:

[0114] Step 10 (Preprocessing): Perform facial landmark detection and portrait parsing operations on the user image, and obtain the facial feature mask and portrait parsing mask based on the set of facial landmarks.

[0115] Step 20 (Preliminary judgment): Determine whether there is a situation where bangs are obviously covering the eyebrows based on the facial feature mask image and the portrait analysis mask image.

[0116] Step 30 (Region Judgment): Calculate the bounding rectangle of the eyebrow region based on the left and right eyebrow outer contour point sets of the facial key point set, and determine whether the eyebrow is completely unobstructed based on the rectangular region.

[0117] Step 40 (Fine Judgment): If the eyebrow occlusion value E1 is still 0, the user image Image0 is converted to the LAB color space, the fine eyebrow region is extracted based on the color clustering algorithm, the color histogram of the inner and outer regions of the eyebrow is calculated, and the color features are analyzed to assess the degree of eyebrow occlusion.

[0118] Figure 3 This is a visualization of the generated facial feature mask in this embodiment. In step 10, the preprocessing operation in this embodiment further includes:

[0119] Step 11: Perform face detection on the user image Image0, and perform facial landmark detection based on the face detection results to obtain the facial landmark set P0.

[0120] Step 12: Divide the contour point set of different parts according to the facial key point set P0, draw the corresponding convex polygon according to the contour point set, each polygon corresponds to a different facial feature, and finally generate a facial feature mask M0 with different labels.

[0121] For example, in this scheme, a detection model containing 126 key points is used. Among them, the facial key point set numbered 84 to 103 is identified as the outer contour point set of the mouth. A convex polygon of the lips is drawn based on this contour point set and assigned a value of 2. The facial key point set numbered 75 to 82 is identified as the outer contour point set of the right eye, and the facial key point set numbered 66 to 73 is identified as the outer contour point set of the left eye. A convex polygon of the eye is drawn based on the contour point set of both eyes and assigned a value of 3. The facial key point set numbered 42 to 51 is identified as the outer contour point set of the right eyebrow, and the facial key point set numbered 33 to 42 is identified as the outer contour point set of the left eyebrow. A convex polygon of the eyebrow is drawn based on the contour point set of both eyebrows and assigned a value of 4.

[0122] Step 13: Perform a portrait parsing operation on the user image Image0 to generate a portrait parsing mask image M1.

[0123] For example, the portrait analysis model used in this solution can perform in-depth analysis of the input image, accurately identify and label the four main semantic categories: background, clothing, hair, and skin.

[0124] Step 14: Initially define the eyebrow occlusion judgment value E1 as 0.

[0125] Figure 4 This is a visualization of the image mask in the case of obvious occlusion in this embodiment. In step 20, the preliminary judgment operation in this embodiment further includes:

[0126] Step 21: Based on the facial feature mask image M0 and the portrait analysis mask image M1, determine whether there is a situation where bangs are obviously obscuring the eyebrows. If so, directly determine that the eyebrows are obscured, set the final eyebrow occlusion score E2 to 0, and define the eyebrow occlusion value E1 as 1.

[0127] Furthermore, the judgment method specifically involves checking whether the hair area of ​​the portrait parsing mask image M1 intersects with the eyebrow area of ​​the facial feature mask image M0. If there is an intersection, it is directly determined that the eyebrows are occluded.

[0128] Figure 5 This is a visualization of the generated eyebrow region parsing mask image in this embodiment. In step 30, the region determination operation in this embodiment further includes:

[0129] Step 31: Calculate the bounding rectangle of the eyebrow region based on the outer contour point set of the left and right eyebrows of the facial key point set P0;

[0130] Step 32: Based on the portrait parsing mask image M1, extract the eyebrow region parsing mask image M2 of the bounding rectangle belonging to the eyebrow region, determine whether the eyebrows are completely unobstructed, set the final eyebrow occlusion score E2 to 1, and define the eyebrow occlusion value E1 to 1.

[0131] Furthermore, the determination method specifically involves statistically analyzing whether the percentage of pixels in the hair region within the eyebrow region parsing mask M2 exceeds a certain condition. If it does not exceed the condition, the eyebrows are determined to be completely unobstructed. For example, in this scheme, the condition is defined as the percentage of pixels in the hair region within the eyebrow region parsing mask M2 being 0.1.

[0132] Figure 6 This is a comparison example of the masked images of the eyebrow area before and after the fine eyebrow operation in this embodiment. Figure 7 This is a comparison example of the mask images of the outer eyebrow region before and after the fine eyebrow outer region operation in this embodiment. In step 40, the fine judgment operation in this embodiment further includes:

[0133] Step 41: If the eyebrow occlusion value E1 is still 0 after steps 20 and 30 above, convert the user image Image0 from the BGR color space to the LAB color space to obtain the processed image Image1.

[0134] Step 42: Extract the fine eyebrow mask M3 from the processed image Image1 using a color clustering algorithm.

[0135] Furthermore, the color clustering algorithm mainly extracts the eyebrow region from the eyebrow convex polygon region in the face feature mask image M0 on the processed image Image1, and uses the K-means clustering algorithm to analyze and divide the pixels of the eyebrow region into two categories. Then, it calculates the average value of the L channel of each cluster in the LAB color space, determines the cluster label of the darker color, and generates a fine eyebrow mask image M3 based on the set of pixels belonging to the darker color cluster label.

[0136] Step 43: Based on the left and right eyebrow outer contour point sets of the facial key point set P0, generate an eyebrow outer region mask map M4 by expanding the left and right eyebrow outer contour point sets.

[0137] Furthermore, generating the eyebrow outer region mask map M4 mainly involves first calculating the center points C1 and C2 of the eyebrows on both sides based on the outer contour point sets of the left and right eyebrows. Then, based on the center points C1 and C2, the outer contour point sets of the left and right eyebrows are expanded outwards, and the expanded outer region mask is drawn. The expanded outer region mask is then processed by removing the portion that overlaps with the eyebrow region of the facial feature mask map M0, and then expanding downwards to obtain the final eyebrow outer region mask map M4.

[0138] Step 44: Process the eyebrow outer region mask image M4 to generate a fine eyebrow outer region mask image M5.

[0139] Furthermore, taking the left eyebrow as an example, generating a detailed eyebrow outer region mask image M5 mainly involves dividing the left eyebrow outer region mask image M4 into two parts, masks M6 and M7. Based on the effective areas of M6 and M7, the histogram information of the L channel is calculated on the processed image Image1. The mean value of the corresponding histograms of M6 and M7 is calculated to measure the brightness of the two outer region mask parts. The mask with the smaller mean value is selected as the detailed eyebrow outer region mask image of the left eyebrow, because the darker area is more likely to be the occluded eyebrow area.

[0140] Step 45: Calculate the color histograms of the effective areas of the fine eyebrow mask M3 and the fine eyebrow outer region mask M5, analyze the color characteristics of the left and right eyebrows and their outer regions, and thus assess the degree to which the eyebrows are obscured.

[0141] Further, taking the left eyebrow as an example, based on the effective area of ​​the fine eyebrow mask M3, the histogram information of the L, A, and B channels is calculated on the processed image Image1, and the mean values ​​L1, A1, and B1 of the corresponding channels are calculated. The three mean values ​​are combined to form the color feature vector lab1 of the effective area of ​​the fine eyebrow mask M3. Based on the effective area of ​​the fine eyebrow outer region mask M5, the histogram information of the L, A, and B channels is calculated on the processed image Image1, and the mean values ​​L2, A2, and B2 of the corresponding channels are calculated. The three mean values ​​are combined to form the color feature vector lab2 of the effective area of ​​the fine eyebrow outer region mask M5. The color difference between the color feature vector lab1 and the color feature vector lab2 is calculated using the CIEDE2000 color difference formula to obtain the color difference value of the left eyebrow. Similarly, the color difference value of the right eyebrow can be calculated. The final eyebrow occlusion score E2 is the value with the smaller color difference value between the left and right eyebrows.

[0142] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, the electronic device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to realize the eyebrow occlusion detection method based on image analysis and color features as described above.

[0143] It is understood that the memory may include random access memory (RAM) or read-only memory. Optionally, the memory may include non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the stored data area may store data created according to the use of the server, etc.

[0144] A processor may include one or more processing cores. The processor connects to various parts of the server via various interfaces and lines, performing various server functions and processing data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by accessing data stored in memory. Optionally, the processor may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate one or more of a Central Processing Unit (CPU) and a modem. The CPU primarily handles the operating system and applications; the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.

[0145] Since this electronic device is the electronic device corresponding to the eyebrow occlusion detection method based on image analysis and color features in the embodiments of the present invention, and the principle of solving the problem by this electronic device is similar to that of this method, the implementation of this electronic device can refer to the implementation process of the above method embodiments, and the repeated parts will not be described again.

[0146] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the eyebrow occlusion detection method based on image analysis and color features as described above.

[0147] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0148] Since this storage medium is the storage medium corresponding to the eyebrow occlusion detection method based on image analysis and color features in the embodiments of the present invention, and the principle of solving the problem by this storage medium is similar to that of this method, the implementation of this storage medium can refer to the implementation process of the above method embodiments, and the repeated parts will not be described again.

[0149] In some possible implementations, various aspects of the methods of the embodiments of the present invention can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of the eyebrow occlusion detection method based on image parsing and color features according to various exemplary embodiments of this application as described above. The executable computer program code or "code" for performing the various embodiments can be written in high-level programming languages ​​such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0150] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0151] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0152] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting eyebrow occlusion based on image analysis and color features, characterized in that, Including the following steps: The input image is preprocessed to obtain a set of facial key points, a facial feature mask, and a portrait parsing mask, and the eyebrow occlusion judgment value is initially defined. Based on the facial feature mask image and the portrait parsing mask image, determine whether there is obvious occlusion. If so, determine the eyebrow occlusion score. Based on the set of facial key points and the image parsing mask, determine whether the eyebrows are completely unobstructed. If they are completely unobstructed, determine the eyebrow occlusion score. The eyebrow occlusion judgment value is initialized to 0. When it is determined that there is obvious occlusion, the eyebrow occlusion judgment value is updated to 1. When it is determined that the eyebrow is not occluded at all, the eyebrow occlusion judgment value is updated to 1. If the eyebrow occlusion judgment value does not change after the above judgment, the input image is converted to the target color space to obtain the processed image. Extract a fine eyebrow mask from the processed image; A mask image of the outer region of the eyebrows is generated based on the set of facial key points, and a fine mask image of the outer region of the eyebrows is obtained by processing the mask image of the outer region of the eyebrows. On the processed image, the effective eyebrow region is determined based on the fine eyebrow mask; the histogram information of the effective eyebrow region in the three channels of the target color space is calculated; based on the histogram information, the mean of the three channels is calculated and combined to form a first color feature vector; on the processed image, the outer effective region is determined based on the fine eyebrow outer region mask; the histogram information of the outer effective region in the three channels of the target color space is calculated; based on the histogram information, the mean of the three channels is calculated and combined to form a second color feature vector; using a preset color difference formula, the difference between the first color feature vector and the second color feature vector is calculated to obtain the color difference value of one eyebrow; the same operation is performed on the other eyebrow to obtain the color difference value of the other eyebrow; the smaller value of the color difference values ​​of the two eyebrows is selected and determined as the eyebrow occlusion score, which is used to quantitatively evaluate whether the ID photo meets the usage standards; The process of processing the mask image of the outer region of the eyebrow to obtain a refined mask image of the outer region of the eyebrow includes: For the outer area mask of the left and right eyebrows respectively, divide the mask image of the outer area of ​​the eyebrow into a left sub-mask and a right sub-mask along the vertical midline of the corresponding mask image itself; On the processed image, the effective regions of the left sub-mask and the right sub-mask are extracted respectively; Calculate the histogram information of the two effective regions in the luminance channel of the target color space; Based on the histogram information, the average brightness of the two effective regions is calculated respectively; The sub-mask with the smaller average brightness value is determined as the mask map of the outer region of the fine eyebrow.

2. The eyebrow occlusion detection method based on image analysis and color features according to claim 1, characterized in that, The preprocessing of the input image includes: Perform face detection on the input image to obtain face detection bounding boxes; Based on the face detection box, face key point detection is performed on the input image to obtain the face key point set; From the set of facial key points, select the point sets corresponding to the facial features, and divide the contour point sets of different parts; Based on the set of contour points, draw corresponding convex polygons, assign unique labels to different convex polygons, and generate the facial feature mask image. The input image is subjected to portrait parsing processing to obtain the portrait parsing mask map labeled with multiple semantic categories.

3. The eyebrow occlusion detection method based on image analysis and color features according to claim 2, characterized in that, The step of selecting the point set corresponding to the facial features from the set of facial key points includes: Extract the point sets corresponding to the left and right eyebrows from the set of facial key points, and determine them as the eyebrow outer contour point set; Extract the point set corresponding to other facial features and use it as the outline point set of other facial features.

4. The eyebrow occlusion detection method based on image analysis and color features according to claim 1, characterized in that, The determination of whether there is obvious occlusion based on the facial feature mask image and the portrait parsing mask image includes: Extract the eyebrow region from the facial feature mask; Extract the hair region from the image parsing mask; Calculate the intersection of the eyebrow region and the hair region; If the area of ​​the intersection exceeds the proportion of the total area of ​​the eyebrow region to a first preset threshold, it is determined that there is obvious occlusion. Set the eyebrow occlusion score to 0 and update the eyebrow occlusion judgment value to 1.

5. The eyebrow occlusion detection method based on image analysis and color features according to claim 1, characterized in that, The step of determining whether the eyebrows are completely unobstructed based on the set of facial key points and the image parsing mask includes: The bounding rectangle of the eyebrow region is determined based on the eyebrow outer contour point set of the facial key point set. Based on the bounding rectangle, extract the eyebrow region parsing mask of the corresponding area from the portrait parsing mask; Count the number of pixels belonging to the hair category in the parsed mask image of the eyebrow region; Calculate the proportion of the number of pixels to the total number of pixels in the eyebrow region parsed mask image; If the ratio does not exceed the second preset threshold, it is determined that the eyebrows are not obscured at all; Set the eyebrow occlusion score to 1 and update the eyebrow occlusion judgment value to 1.

6. The eyebrow occlusion detection method based on image analysis and color features according to claim 1, characterized in that, The step of converting the input image to the target color space includes: The input image is converted from the first color space to the second color space to obtain the processed image.

7. The eyebrow occlusion detection method based on image analysis and color features according to claim 1, characterized in that, The step of extracting a fine eyebrow mask from the processed image includes: Based on the eyebrow convex polygon in the facial feature mask image, the eyebrow area is selected on the processed image. Extract color features from all pixels in the eyebrow region; A clustering algorithm is performed on the color features to divide the pixels into two categories; Calculate the average value of the brightness channel of the two types of pixels in the target color space respectively; The category with the smaller average value of the brightness channels is identified as the target eyebrow pixels; The fine eyebrow mask is generated based on the position information of the target eyebrow pixels.

8. The eyebrow occlusion detection method based on image analysis and color features according to claim 1, characterized in that, The process of generating a mask image of the eyebrow outer region based on the set of facial key points includes: Extract the left and right eyebrow outer contour point sets from the facial key point set; Calculate the center point of the outer contour point set of the left and right eyebrows respectively; Using the center point as a reference, extend the outer contour points of the left and right eyebrows outward by a preset distance; Draw the outer region based on the expanded point set and generate an initial outer region mask; The initial outer region mask is compared with the eyebrow region of the facial feature mask to remove overlapping parts; Perform a downward expansion operation on the mask after removing the overlapping parts to obtain the mask image of the outer region of the eyebrow.

Images