Full-coverage face partitioning method and system

Abstract

The invention relates to the technical field of image processing, in particular to a full-coverage face partitioning method and system. The method comprises the steps that multiple groups of face images are collected, and the face images comprise a front face image and a side face image; face key points and face skin area masks are marked on all the face images, an image-point set-mask triple data set is formed, and the face key points comprise preset anatomical positions; taking the triple data set as sample data, and generating an initial analysis model through a deep neural network model; a predefined grid partition generation rule is integrated to the initial analysis model, a face partition model is generated, and the face partition model is used for outputting face key point coordinates, a face skin area mask and grid partitions generated based on the key points. According to the method and the device, the accuracy of dividing the opposite part can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a full-coverage facial sub-region method and system. BACKGROUND

[0002] With the continuous development of image processing technology, analysis and processing of facial images have been widely applied in many fields, such as face recognition, facial expression analysis, medical cosmetology, etc. In these applications, accurate facial sub-region division is very crucial.

[0003] Traditional facial sub-region division methods mainly focus on processing of frontal facial images, which results in that the generated composite images cannot be effectively covered. Secondly, when determining the sub-region boundary, the traditional sub-region division method lacks in-depth consideration of the anatomical structure of the face, which leads to inaccurate sub-region division and is difficult to meet the application requirements in complex scenarios. Therefore, there is an urgent need for a sub-region division method that can fully cover the face. SUMMARY

[0004] In order to solve the above technical problems, the present application provides a full-coverage facial sub-region division method and system, thereby improving the accuracy of facial sub-region division.

[0005] In a first aspect, the present application provides a full-coverage facial sub-region division method, comprising: collecting a plurality of sets of facial images, the facial images including frontal facial images and side facial images; labeling facial key points and facial skin region masks on each of the facial images to form an image-point set-mask triple data set, the facial key points including predetermined anatomical positions; generating an initial analysis model by a deep neural network model using the triple data set as sample data; integrating a pre-defined grid sub-region generation rule into the initial analysis model to generate a facial sub-region model, the facial sub-region model being used to output facial key point coordinates, facial skin region masks, and a grid sub-region generated based on the key points.

[0006] Optionally, in some embodiments, the grid sub-region generation rule includes: determining a horizontal baseline based on the facial key points; determining a plurality of horizontal reference points and vertical reference points based on the facial key points; generating a grid composed of a plurality of sub-regions by the horizontal reference points and vertical reference points with the horizontal baseline as a reference; wherein the boundary of the sub-region is parallel or perpendicular to the horizontal baseline.

[0007] Optionally, in some embodiments, determining the horizontal baseline based on the facial key points includes: for a frontal facial image, extracting facial key points corresponding to the left outer corner of the eye and the right outer corner of the eye, and determining the line connecting the left outer corner of the eye and the right outer corner of the eye as the horizontal baseline; for a side facial image, extracting facial key points corresponding to the outer corner of the eye and the starting point on the same side of the ear, and determining the line connecting the outer corner of the eye and the starting point on the same side of the ear as the horizontal baseline.

[0008] Optionally, in some embodiments, determining multiple horizontal and vertical reference points based on the facial key points includes: determining multiple horizontal reference points through vector calculation based on a predefined first set of key points; and determining multiple vertical reference points based on a predefined second set of key points.

[0009] Optionally, in some embodiments, the first set of key points corresponds to the following anatomical locations: forehead apex, upper forehead boundary, forehead center, midpoint of the upper edge of the brow ridge, midpoint of the lower edge of the brow ridge, midpoint of the line connecting the supraorbital foramina, midpoint of the line connecting the outer corners of the eyes, midpoint of the line connecting the upper edges of the zygomatic arch, midpoint of the line connecting the infraorbital foramina, lower edge of the nasal ala, midpoint of the philtrum, midpoint of the upper edge of the upper lip, midpoint of the lower edge of the lower lip, midpoint of the lip and chin, and chin tip; the second set of key points corresponds to the following anatomical locations: right infraorbital foramen, right inner canthus, midpoint of the nasal root, left inner canthus, and left infraorbital foramen.

[0010] Optionally, in some embodiments, generating a grid consisting of multiple sub-regions based on the horizontal baseline and through the horizontal and vertical reference points includes: generating a horizontal line parallel to the horizontal baseline through each of the horizontal reference points; vertically projecting each of the vertical reference points onto each of the horizontal lines to form grid intersections; and connecting adjacent grid intersections to form multiple quadrilateral sub-regions.

[0011] Optionally, in some embodiments, the deep neural network model includes: a feature extractor for extracting general visual features from the input face image; and two sub-network branches for outputting the coordinates of the facial key points and the facial skin region mask based on the general visual features.

[0012] Optionally, in some embodiments, the deep neural network model includes: a facial keypoint coordinate regression loss function.

[0013] ;

[0014] In the formula, The regression loss value is the facial landmark coordinates. These are the coordinates of facial key points labeled during data preprocessing. For face detection networks, The predicted face coordinates are output by the face detection network branch. This represents the total number of facial landmarks. To sum the errors of the coordinates of all facial key points;

[0015] Skin region mask segmentation loss function:

[0016] ;

[0017] In the formula, This represents the loss value for mask segmentation of the skin region. For channel-dimensional cross-entropy, To input the width of the face image, The input is the height of the face image, where x and y are the coordinates of the pixels in the face image. The mask prediction value output by the neural network. The true label is obtained by using a region change mask in data preprocessing. The percentage of genuine labels. For the softmax function, The maximum index of the softmax function based on the channel dimension is used to output the mask to the network, which is converted into a one-hot predicted label.

[0018] Optionally, in some embodiments, after generating the mesh partition, the method further includes: region validity verification, specifically including: calculating the percentage of effective pixel area identified as skin in each sub-region of the mesh partition based on the facial skin region mask output by the facial partitioning model; and marking sub-regions with the percentage of effective pixel area lower than a preset threshold as invalid regions.

[0019] Secondly, this application also provides a facial partitioning system applicable to any of the facial partitioning methods described above, the facial partitioning system comprising:

[0020] The image acquisition module is used to acquire multiple sets of facial images, including frontal facial images and side facial images;

[0021] The data annotation module is used to annotate facial key points and facial skin region masks for each of the facial images, forming an image-point set-mask triplet dataset, wherein the facial key points include predetermined anatomical locations;

[0022] The initial analysis model construction module is used to generate an initial analysis model using the triplet dataset as sample data through a deep neural network model.

[0023] The face partitioning model construction module is used to integrate predefined mesh partitioning generation rules into the initial analysis model to generate a face partitioning model. The face partitioning model is used to output the coordinates of facial key points, facial skin region masks, and mesh partitions generated based on the key points.

[0024] The technical solution provided in this application has the following advantages compared with the prior art:

[0025] The full-coverage face partitioning method provided in this application collects multiple sets of face images including frontal and side views, and annotates the images with facial key points and skin region masks to form an image-point set-mask triplet dataset, which can achieve full-range coverage of the face (frontal and side views). Furthermore, using the triplet dataset as sample data, an initial analysis model is generated through a deep neural network model, and then predefined grid partitioning generation rules are integrated into the initial analysis model to generate a face partitioning model. By integrating the grid partitioning generation rules, the grid partitioning output by the face partitioning model is more accurate.

[0026] Furthermore, the full-coverage facial segmentation method provided in this application embodiment also uses a skin mask filtering mechanism to eliminate interference from non-skin areas, thereby improving the accuracy of extracting features such as skin color and texture within each grid segment. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0029] Figure 1 A schematic flowchart of the full-coverage facial partitioning method provided in the embodiments of this application;

[0030] Figure 2 This is a schematic diagram illustrating a specific application of the full-coverage facial partitioning method provided in the embodiments of this application;

[0031] Figure 3 This is a schematic diagram of the structure of the full-coverage facial partitioning system provided in the embodiments of this application. Detailed Implementation

[0032] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0033] Many specific details are set forth in the following description in order to provide a full understanding of this application, but this application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of this application, and not all embodiments.

[0034] The full-coverage facial partitioning method and system provided in this application will be described exemplarily below with reference to the accompanying drawings. Figure 1 This is a schematic flowchart illustrating the full-coverage facial segmentation method provided in an embodiment of this application. (Refer to...) Figure 1 The full-coverage facial segmentation method 100 includes the following steps:

[0035] S101. Collect multiple sets of facial images, including frontal facial images and side facial images.

[0036] Specifically, firstly, using a capture device, such as an RGB camera with a resolution of 1920x1080 pixels, under a lighting environment (such as a standard D65), captures facial images of the user from the front and sides (left or right 90°). These facial images can be scaled down to a width of 1024 pixels and illuminated using a histogram equalization algorithm to improve image contrast and detail visibility.

[0037] It is understood that the above description of acquiring multiple sets of facial images is merely exemplary. Those skilled in the art can select and set the acquisition of multiple sets of facial images according to actual applications, as long as the technical principles of this application can be achieved.

[0038] S102. Label the facial key points and facial skin region masks for each of the facial images to form an image-point set-mask triplet dataset, wherein the facial key points include predetermined anatomical locations.

[0039] Specifically, based on the facial image acquired in step S101, key facial points can be marked. These key facial points can include not only the outline of the face, but also facial features and their edges, such as eyes, inner corners of the eyes, outer corners of the eyes, nose, mouth, eyebrows, ears, etc. These key facial points include predetermined anatomical positions, which can be standard positions based on human anatomy.

[0040] Furthermore, in addition to annotating facial key points, facial skin area masks can also be annotated, thereby clearly distinguishing skin areas and non-skin areas (such as hair, eyebrows, and background) in facial images, ensuring that only skin-related information is processed in image processing.

[0041] By combining labeled facial keypoints and facial skin region masks, a triplet dataset containing facial images, point sets, and skin masks can be formed, which can provide accurate data for facial partitioning.

[0042] S103. Using the triplet dataset as sample data, generate an initial analysis model through a deep neural network model.

[0043] Specifically, based on the constructed image-point-mask triplet dataset—that is, multiple data pairs including facial images, point sets, and skin masks—the triplet dataset is used as training samples and input into a deep neural network model to generate an initial analysis model. This deep neural network model is used for facial image analysis and can learn to extract useful features from the input data and identify key anatomical landmarks and skin regions of the face. Through iterative training, the deep neural network model continuously optimizes its internal parameters, generating the initial analysis model.

[0044] Optionally, in one embodiment, the deep neural network model may include: a feature extractor for extracting general visual features from the input face image; and two sub-network branches for outputting the coordinates of the facial key points and the facial skin region mask based on the general visual features.

[0045] Specifically, the deep neural network model can include: one feature extractor and two sub-network branches. The sub-network branches output the coordinates of facial key points and the facial skin region mask, respectively. The feature extractor shares parameters between the two sub-network branches and can consist of 12 facial feature extraction modules, with four convolutional layers that are downsampled by 2x. The structure of the facial feature extraction module can be as follows: it starts with a 1x1 convolution kernel + PreLU activation function to double the number of input channels, because the subsequent modules will be divided into two branches. If downsampling by 1x is required, the first kernel on the left is a 3x3 convolution with a stride of 2; the right side is set with one average pooling layer. Otherwise, the stride on the left is 1 and no average pooling is set. The left branch is subsequently set with multiple 3x3 convolution kernels and branches because the input views are diverse and the facial statistical regions are diverse. In order to improve the extraction capability of the facial feature extraction module, different feature maps are added, and finally the channels of the four feature map branches are merged together.

[0046] Optionally, in one embodiment, the deep neural network model may include: a facial keypoint coordinate regression loss function.

[0047] ;

[0048] In the formula, The regression loss value is the facial landmark coordinates. These are the coordinates of facial key points labeled during data preprocessing. For face detection networks, The predicted face coordinates are output by the face detection network branch. This represents the total number of facial landmarks. This is used to sum the errors of the coordinates of all facial key points.

[0049] Skin region mask segmentation loss function:

[0050] ;

[0051] In the formula, This represents the loss value for mask segmentation of the skin region. For channel-dimensional cross-entropy, To input the width of the face image, The input is the height of the face image, where x and y are the coordinates of the pixels in the face image. The mask prediction value output by the neural network. The true label is obtained by using a region change mask in data preprocessing. The percentage of genuine labels. For the softmax function, The maximum index of the softmax function based on the channel dimension is used to output the mask to the network, which is converted into a one-hot predicted label.

[0052] Specifically, the deep neural network model has two network branch outputs. Therefore, corresponding to the facial key point coordinate regression loss function and the skin region mask segmentation loss function, through successive training, the loss of the above two functions is minimized to complete the training of the deep neural network model.

[0053] S104. Integrate the predefined mesh partitioning generation rules into the initial analysis model to generate a face partitioning model. The face partitioning model is used to output the coordinates of facial key points, facial skin region masks, and mesh partitions generated based on the key points.

[0054] Specifically, predefined mesh partitioning rules are integrated into the initial analysis model to generate a more accurate face partitioning model. This face partitioning model is used to process and analyze facial images, capable of identifying and outputting facial keypoint coordinates and automatically generating mesh partitions based on these coordinates. Furthermore, the face partitioning model can also generate a facial skin region mask, which can distinguish between facial skin and non-skin regions.

[0055] The predefined mesh partitioning generation rules can be rules or algorithms designed based on facial anatomical features, geometric principles, and specific application requirements. When setting the mesh partitioning generation rules, it is necessary to ensure that the natural anatomical structure of the face, such as bones, muscles, and skin, as well as the position and shape of the facial features, are taken into account, while incorporating geometric principles such as symmetry, proportion, and angles, to guarantee the uniformity and regularity of facial partitioning.

[0056] Furthermore, when setting the grid partitioning generation rules, the design can be tailored to the facial differences of different individuals, such as race, age, and gender, to improve the flexibility and scalability of facial partitioning.

[0057] Optionally, in one embodiment, the grid partitioning generation rule may include: determining a horizontal baseline based on the facial key points; determining multiple horizontal and vertical reference points based on the facial key points; using the horizontal baseline as a reference, generating a grid composed of multiple sub-regions through the horizontal and vertical reference points; wherein the boundaries of the sub-regions are parallel or perpendicular to the horizontal baseline.

[0058] Specifically, a horizontal baseline is determined based on key facial points. This horizontal baseline can be determined by connecting the left and right outer corners of the eyes, thereby providing a horizontal reference for facial zoning.

[0059] Multiple horizontal and vertical reference points can be determined based on facial key points. For example, in a frontal view of the face, multiple horizontal reference points can be determined based on anatomical landmarks, and multiple auxiliary horizontal baselines of the same length as the horizontal baseline can be obtained by passing through these multiple horizontal reference points.

[0060] Horizontal and vertical reference points provide precise vertical and horizontal positioning for facial partitioning. Furthermore, the selection of these reference points takes into account the natural features and anatomical structure of the face, ensuring the rationality and scientific basis of facial partitioning.

[0061] Using a horizontal baseline as a reference, a mesh consisting of multiple sub-regions is generated by connecting these horizontal and vertical reference points. This mesh covers the entire face (including the front and side views), and the boundaries of each sub-region are parallel or perpendicular to the horizontal baseline, making facial segmentation more refined and accurate.

[0062] For example, determining the horizontal baseline based on the facial key points may include: for a frontal facial image, extracting facial key points corresponding to the left outer corner of the eye and the right outer corner of the eye, and determining the line connecting the left outer corner of the eye and the right outer corner of the eye as the horizontal baseline; for a side facial image, extracting facial key points corresponding to the outer corner of the eye and the starting point on the same side of the ear, and determining the line connecting the outer corner of the eye and the starting point on the same side of the ear as the horizontal baseline.

[0063] Specifically, for a frontal facial image, the horizontal baseline can be determined, for example, by connecting the points of the left and right outer corners of the eyes. The outer corner of the eye is one of the key anatomical landmarks of the face, and is particularly important for facial symmetry. Therefore, the straight line formed by connecting the two points of the left and right outer corners of the eyes can provide an accurate reference line.

[0064] For lateral facial images, the determination of the horizontal baseline should adapt to the variations in facial features from a lateral view. In this case, the horizontal baseline can be determined by connecting the outer corner of the eye to the starting point on the same side of the ear. The starting point on the ear typically refers to the point where the upper part of the ear meets the side of the head. The line connecting this point to the outer corner of the eye better reflects the anatomical structure of the face from a lateral view, providing an accurate benchmark for partitioning lateral facial images.

[0065] Optionally, in one embodiment, determining multiple horizontal and vertical reference points based on the facial key points may include: determining multiple horizontal reference points by vector calculation based on a predefined first set of key points; and determining multiple vertical reference points based on a predefined second set of key points.

[0066] Specifically, the first set of predefined key points corresponds to, for example, the following anatomical locations: forehead apex, upper forehead boundary, forehead center, midpoint of the upper edge of the brow ridge, midpoint of the lower edge of the brow ridge, midpoint of the line connecting the supraorbital foramina, midpoint of the line connecting the outer corners of the eyes, midpoint of the line connecting the upper edges of the zygomatic arch, midpoint of the line connecting the infraorbital foramina, lower edge of the nasal ala, midpoint of the philtrum groove, midpoint of the upper edge of the upper lip, midpoint of the lower edge of the lower lip, midpoint of the lip and chin, and chin tip.

[0067] Based on the relative positions of the first set of key points, multiple horizontal reference points can be calculated, which can serve as the horizontal boundaries of the grid partitions.

[0068] Multiple vertical reference points are determined based on a predefined second set of key points. These key points may correspond to, for example, the following anatomical locations: the right infraorbital foramen, the right inner canthus, the midpoint of the nasal root, the left inner canthus, and the left infraorbital foramen. By analyzing the distribution of these key points, multiple vertical reference points can be determined, which can serve as the vertical boundaries of the grid partitions.

[0069] It is understandable that the first set of key points and the second set of key points are based on the correspondence between facial features and the internal organs and meridians, so that the facial zoning method can provide support for TCM diagnosis.

[0070] Optionally, in one embodiment, generating a grid consisting of multiple sub-regions based on the horizontal baseline and through the horizontal and vertical reference points may include: generating a horizontal line parallel to the horizontal baseline through each of the horizontal reference points; vertically projecting each of the vertical reference points onto each of the horizontal lines to form grid intersections; and connecting adjacent grid intersections to form multiple quadrilateral sub-regions.

[0071] Specifically, a horizontal line parallel to the horizontal baseline is generated using various horizontal reference points. Based on the determined horizontal baseline, a horizontal line of the same length as the horizontal baseline can be obtained using multiple horizontal reference points.

[0072] Each vertical reference point is projected vertically onto each horizontal line to form a grid intersection. The vertical reference points are determined based on a predefined second set of key points and are distributed along the width of the face. By projecting the vertical reference points onto each horizontal line, multiple grid intersections can be obtained on each horizontal plane, and these grid intersections define the vertical boundaries of the grid partitions.

[0073] By connecting the intersections of adjacent grid lines, multiple quadrilateral sub-regions are formed. These sub-regions, such as rectangles or squares, cover every part of the facial image, and each sub-region is parallel or perpendicular to a horizontal baseline. This method of generating a regular grid allows the facial image to be divided into multiple sub-regions with clearly defined boundaries and relative positions.

[0074] Optionally, in one embodiment, after generating the mesh partition, the method further includes: region validity verification, specifically including: calculating the percentage of effective pixel area identified as skin in each sub-region of the mesh partition based on the facial skin region mask output by the facial partitioning model; and marking sub-regions with the percentage of effective pixel area lower than a preset threshold as invalid regions.

[0075] Specifically, after completing the mesh partitioning, the method may further include region validity verification. Region validity verification ensures that each sub-region contains a sufficient number of valid skin pixels. Specifically, region validity verification relies on the facial skin region mask output by the facial partitioning model. The facial skin region mask is a two-dimensional array where each pixel value indicates whether the pixel at that location belongs to the skin region. Using the facial skin region mask, the percentage of valid pixels identified as skin within each sub-region can be calculated.

[0076] For example, a threshold (such as 20%) can be set to determine the validity of a sub-region. If the percentage of valid pixels within a sub-region is lower than the set threshold, that sub-region will be marked as invalid. By marking invalid regions, the accuracy and reliability of facial image analysis can be further improved.

[0077] Figure 2 The following is a schematic diagram illustrating a specific application of the full-coverage facial segmentation method provided in the embodiments of this application. Figure 2 The specific application of the full-coverage facial zoning method is described in detail.

[0078] ① Image Acquisition and Preprocessing: Using an RGB camera with a resolution of 1920x1080 pixels, images of the user's face from the front and a 90-degree angle to the left (or right) side were acquired under standard D65 lighting conditions. The images were uniformly scaled to 1024px (width), and a histogram equalization algorithm was used for illumination compensation. Multiple (e.g., 130) facial key points were annotated on the acquired facial images, including the facial contours and the edges of facial features. Simultaneously, facial skin region masks were annotated, forming data pairs of image-point set-skin region mask.

[0079] ② Training the detection and segmentation model: The image is scaled to 256x256 and input into a deep neural network model. The deep neural network model consists of one feature extractor and two sub-network branches (i.e., output branches). The sub-network branches output the coordinates of facial key points and the facial skin region mask, respectively. The feature extractor shares parameters between the two sub-network branches and consists of 12 facial feature extraction modules, with four convolutional layers downsampled by 2x. The structure of the facial feature extraction module is as follows: It starts with a 1x1 convolution kernel and a PreLU activation function, doubling the number of input channels. Since the module will be divided into two branches, if downsampling by 1x is required, the first kernel on the left is a 3x3 convolution with a stride of 2; an average pooling layer is set on the right. Otherwise, the stride on the left is 1 and no average pooling is set. The left branch then sets multiple 3x3 convolution kernels and branches, and finally the four feature map branches are merged together.

[0080] ③ Inference process after training: The trained initial analysis model has the ability to regress the coordinates of key points on the front and side of the face and the mask of the facial skin region. At this point, inputting the facial image into the initial analysis model will yield multiple facial key points and facial skin region masks.

[0081] ④ Predefined grid partitioning generation rules:

[0082] a. For example, the coordinates of the outer corners of the right and left eyes can be extracted, calculated, and connected into a line, which is defined as the horizontal baseline AB. The outline points of the left and right ears can be extracted and combined with the outer corner points to determine the horizontal baseline of the side view.

[0083] Front view: The pixel coordinates of multiple horizontal reference points are calculated using vector coordinates. For example, the midpoint of the upper edge of the eyebrow can be obtained by averaging the centers of the upper edges of the left and right eyebrows; the lower boundary of the tear trough can be obtained by averaging the lower points of the left and right eyelids.

[0084] Side view: Similarly, multiple horizontal reference points (e.g., 15) can be calculated based on 130 facial key points. For example, the midpoint between the bridge of the nose and the opening of the external auditory canal can be obtained by taking the midpoint between the vertical coordinate of the tip of the nose and the vertical coordinate of the ear point.

[0085] b. Mesh generation and facial skin mask filtering:

[0086] Front view: Based on the Y coordinates of 15 horizontal reference points, draw 15 horizontal lines. Project the X coordinates of multiple vertical reference points onto each horizontal line to form grid intersections, and then use computational geometry to divide the region into multiple quadrilateral grids.

[0087] Side view: Generates multiple quadrilateral meshes based on multiple horizontal and multiple vertical reference points.

[0088] c. Traverse each sub-region: If the threshold ratio is set to 20%, if the proportion of skin pixels is lower than this threshold, mark it as invalid in the statistical list of the grid; otherwise, extract the pixels of the skin region in the grid from the original image for subsequent analysis (such as RGB mean, colorimetric statistics, etc.).

[0089] The facial segmentation method described in this application can provide treatment plans for skin problems in different segments. For example, after segmenting the face using the facial segmentation method, a skin problem knowledge base is invoked to diagnose skin problems based on prior knowledge of the regions, and treatment plans are provided in combination with the skin problems in different segments.

[0090] In summary, the full-coverage facial segmentation method 100 provided in this application collects multiple sets of facial images including frontal and side views, and annotates facial key points and skin region masks on the images to form an image-point set-mask triplet dataset, which can achieve full-range coverage of the face (frontal and side views). Furthermore, using the triplet dataset as sample data, an initial analysis model is generated through a deep neural network model, and then predefined grid segmentation generation rules are integrated into the initial analysis model to generate a facial segmentation model. By integrating the grid segmentation generation rules, the grid segmentation output by the facial segmentation model is more accurate.

[0091] Furthermore, the full-coverage facial segmentation method provided in this application embodiment also uses a skin mask filtering mechanism to eliminate interference from non-skin areas, thereby improving the accuracy of extracting features such as skin color and texture within each grid segment.

[0092] Based on the same inventive concept, this application also provides a full-coverage facial partitioning system 200. Figure 2 This is a schematic diagram of the structure of the full-coverage facial partitioning system provided in an embodiment of this application. Figure 2 As shown, the full-coverage facial partitioning system 200 may include:

[0093] The image acquisition module 201 is used to acquire multiple sets of facial images, including frontal facial images and side facial images;

[0094] The data annotation module 202 is used to annotate facial key points and facial skin region masks for each of the facial images to form an image-point set-mask triplet dataset, wherein the facial key points include predetermined anatomical locations;

[0095] The initial analysis model construction module 203 is used to generate an initial analysis model using the triplet dataset as sample data through a deep neural network model.

[0096] The face partitioning model construction module 204 is used to integrate predefined mesh partitioning generation rules into the initial analysis model to generate a face partitioning model. The face partitioning model is used to output the coordinates of facial key points, facial skin region masks, and mesh partitions generated based on the key points.

[0097] The full-coverage facial partitioning system provided in the above embodiments can execute the full-coverage facial partitioning method provided in the above embodiments and has the same or corresponding beneficial effects, which will not be described in detail here.

[0098] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments.

[0099] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0100] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0101] In this application, unless otherwise stated, directional terms such as "up" and "down" are generally used in relation to the direction shown in the accompanying drawings, or in relation to the vertical, perpendicular, or gravitational direction; similarly, for ease of understanding and description, "left" and "right" are generally used in relation to the left and right shown in the accompanying drawings; "inner" and "outer" refer to the inner and outer contours of each component itself, but the above directional terms are not intended to limit this application.

[0102] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A full-coverage facial partial zone method, characterized in that, The method comprises the following steps: Collecting a plurality of facial images, the facial images including front facial images and side facial images; Labeling facial key points and facial skin region masks on each facial image to form an image-point-mask triple data set, the facial key points including predetermined anatomical positions; Using the triple data set as sample data to generate an initial analysis model through a deep neural network model; Integrating a pre-defined grid partition generation rule into the initial analysis model to generate a facial grid partition model, the facial grid partition model being used to output facial key point coordinates, a facial skin region mask, and a grid partition generated based on the key points.

2. The facial sub-zone method of claim 1, wherein, The grid partition generation rule comprises: Determining a horizontal baseline based on the facial key points; Determining a plurality of horizontal reference points and vertical reference points based on the facial key points; Generating a grid composed of a plurality of sub-regions based on the horizontal baseline, the horizontal reference points, and the vertical reference points. The boundary of the sub-region is parallel or perpendicular to the horizontal baseline.

3. The facial sub-zone method of claim 2, wherein, The method for determining the horizontal baseline based on the facial key points comprises: For a front facial image, extracting facial key points corresponding to the left outer corner of the eye and the right outer corner of the eye, and determining the line connecting the left outer corner of the eye and the right outer corner of the eye as the horizontal baseline; For a side facial image, extracting facial key points corresponding to the outer corner of the eye and the ipsilateral upper ear starting point, and determining the line connecting the outer corner of the eye and the ipsilateral upper ear starting point as the horizontal baseline.

4. The facial sub-zone method of claim 2, wherein, The method for determining a plurality of horizontal reference points and vertical reference points based on the facial key points comprises: Determining a plurality of horizontal reference points through vector calculation based on a pre-defined first group of key points; Determining a plurality of vertical reference points based on a pre-defined second group of key points.

5. The facial grid partition method according to claim 4, wherein The first group of key points correspond to the following anatomical positions: the vertex of the forehead, the upper forehead boundary point, the center of the forehead, the midpoint of the upper eyebrow arch, the midpoint of the lower eyebrow arch, the midpoint of the superior orbital foramen connecting line, the midpoint of the outer corner of the eye connecting line, the midpoint of the upper zygomatic arch connecting line, the midpoint of the inferior orbital foramen connecting line, the lower nose wing point, the midpoint of the philtrum groove, the midpoint of the upper lip upper edge, the midpoint of the lower lip lower edge, the lip-chin midpoint, and the chin tip point. The second group of key points correspond to the following anatomical positions: the right inferior orbital foramen point, the right inner corner of the eye point, the nasal root midpoint, the left inner corner of the eye point, and the left inferior orbital foramen point.

6. The facial sub-zone method of claim 5, wherein, The method for generating a grid composed of a plurality of sub-regions based on the horizontal baseline, the horizontal reference points, and the vertical reference points comprises: Generating a horizontal line parallel to the horizontal baseline through each horizontal reference point; Vertically projecting each vertical reference point onto each horizontal line to form a grid intersection point; Connecting adjacent grid intersection points to form a plurality of quadrilateral sub-regions.

7. The facial sub-zone method of claim 1, wherein, The deep neural network model comprises: 1 feature extractor for extracting general visual features from the input facial image; 2 sub-network branches respectively used to output facial key point coordinates and facial skin region masks based on the general visual features.

8. The facial sub-zone method of claim 7, wherein, The deep neural network model comprises: A facial key point coordinate regression loss function: ； In the formula, is a face key point coordinate regression loss value, is a face key point coordinate labeled in data preprocessing, is a face detection network, is a predicted face point coordinate output by the face detection network branch, is the total number of face key points, is the sum of errors of all face key point coordinates; A skin region mask segmentation loss function: ； wherein, is the skin region mask segmentation loss value, is the cross-entropy based on the channel dimension, is the width of the input face image, is the height of the input face image, x, y are the coordinate positions of the pixels in the face image, is the mask prediction value output by the neural network, is the real label through the region change mask in the data preprocessing, is the proportion of the real label, is the softmax function, is the maximum value index of the softmax function based on the channel dimension of the network output mask, that is, the predicted label converted into one-hot form.

9. The facial sub-zone method of claim 1, wherein, After generating the grid partition, the method further comprises: region validity verification, specifically comprising: Based on the face skin region mask output by the face partition model, calculating the proportion of valid pixel area in each sub-region in the grid partition that is identified as skin; Marking a sub-region with a valid pixel area proportion lower than a preset threshold as an invalid region.

10. A face partition system suitable for the face partition method of any one of claims 1 to 9, the face partition system comprising: An image acquisition module for acquiring a plurality of face images, the face images including front face images and side face images; A data labeling module for labeling face key points and face skin region masks on each of the face images to form an image-point-mask triple data set, the face key points including predetermined anatomical positions; An initial analysis model construction module for generating an initial analysis model through a deep neural network model using the triple data set as sample data; A face partition model construction module for integrating a predefined grid partition generation rule set into the initial analysis model to generate a face partition model, the face partition model being configured to output face key point coordinates, a face skin region mask, and a grid partition generated based on the key points.

Images