Full coverage facial subregion method and system
By collecting and labeling multiple sets of facial image data, and combining deep neural networks and grid partitioning generation rules, the problem that traditional facial partitioning methods cannot fully cover and accurately partition faces has been solved, achieving high-precision partitioning of facial images.
Patent Information
- Application Number
- CN202610156666.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-04
- Publication Date
- 2026-05-15
- Estimated Expiration
- 2046-02-04
AI Technical Summary
Traditional facial segmentation methods mainly focus on processing frontal facial images, resulting in composite images that cannot effectively cover the face. Furthermore, they lack in-depth consideration of facial anatomy, leading to inaccurate segmentation and difficulty in meeting the application needs of complex scenarios.
Multiple sets of facial images are collected, facial key points and skin region masks are labeled to form an image-point set-mask triplet dataset. An initial analysis model is generated through a deep neural network model, and predefined grid partitioning generation rules are integrated to generate a facial partitioning model. The output includes facial key point coordinates, skin region masks and grid partitions.
It achieves full coverage of the face, improves the accuracy of partitioning, and eliminates interference from non-skin areas through a skin mask filtering mechanism, thereby improving the accuracy of skin color and texture feature extraction within the grid partition.
Smart Images
Figure CN121639716B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a method and system for full-coverage facial segmentation. Background Technology
[0002] With the continuous development of image processing technology, the analysis and processing of facial images has been widely applied in many fields, such as face recognition, facial expression analysis, and medical aesthetics. Accurate facial segmentation is crucial in these applications.
[0003] Traditional facial segmentation methods primarily focus on processing frontal facial images, resulting in composite images that fail to provide adequate coverage. Furthermore, traditional segmentation methods lack in-depth consideration of facial anatomy when determining segment boundaries, leading to imprecise segmentation that is unsuitable for complex scenarios. Therefore, a segmentation method capable of comprehensively covering the face is urgently needed. Summary of the Invention
[0004] To address the aforementioned technical issues, this application provides a full-coverage facial segmentation method and system, thereby improving the accuracy of facial segmentation.
[0005] In a first aspect, this application provides a full-coverage facial partitioning method, comprising: acquiring multiple sets of facial images, including frontal facial images and side facial images; annotating facial key points and facial skin region masks for each of the facial images to form an image-point set-mask triplet dataset, wherein the facial key points include predetermined anatomical locations; using the triplet dataset as sample data, generating an initial analysis model through a deep neural network model; integrating predefined grid partitioning generation rules into the initial analysis model to generate a facial partitioning model, wherein the facial partitioning model is used to output the coordinates of facial key points, facial skin region masks, and grid partitions generated based on the key points.
[0006] Optionally, in some embodiments, the grid partitioning generation rules include: determining a horizontal baseline based on the facial key points; determining multiple horizontal and vertical reference points based on the facial key points; using the horizontal baseline as a reference, generating a grid composed of multiple sub-regions through the horizontal and vertical reference points; wherein the boundaries of the sub-regions are parallel or perpendicular to the horizontal baseline.
[0007] Optionally, in some embodiments, determining the horizontal baseline based on the facial key points includes: for a frontal facial image, extracting facial key points corresponding to the left outer corner of the eye and the right outer corner of the eye, and determining the line connecting the left outer corner of the eye and the right outer corner of the eye as the horizontal baseline; for a side facial image, extracting facial key points corresponding to the outer corner of the eye and the starting point on the same side of the ear, and determining the line connecting the outer corner of the eye and the starting point on the same side of the ear as the horizontal baseline.
[0008] Optionally, in some embodiments, determining multiple horizontal and vertical reference points based on the facial key points includes: determining multiple horizontal reference points through vector calculation based on a predefined first set of key points; and determining multiple vertical reference points based on a predefined second set of key points.
[0009] Optionally, in some embodiments, the first set of key points corresponds to the following anatomical locations: forehead apex, upper forehead boundary, forehead center, midpoint of the upper edge of the brow ridge, midpoint of the lower edge of the brow ridge, midpoint of the line connecting the supraorbital foramina, midpoint of the line connecting the outer corners of the eyes, midpoint of the line connecting the upper edges of the zygomatic arch, midpoint of the line connecting the infraorbital foramina, lower edge of the nasal ala, midpoint of the philtrum, midpoint of the upper edge of the upper lip, midpoint of the lower edge of the lower lip, midpoint of the lip and chin, and chin tip; the second set of key points corresponds to the following anatomical locations: right infraorbital foramen, right inner canthus, midpoint of the nasal root, left inner canthus, and left infraorbital foramen.
[0010] Optionally, in some embodiments, generating a grid consisting of multiple sub-regions based on the horizontal baseline and through the horizontal and vertical reference points includes: generating a horizontal line parallel to the horizontal baseline through each of the horizontal reference points; vertically projecting each of the vertical reference points onto each of the horizontal lines to form grid intersections; and connecting adjacent grid intersections to form multiple quadrilateral sub-regions.
[0011] Optionally, in some embodiments, the deep neural network model includes: a feature extractor for extracting general visual features from the input face image; and two sub-network branches for outputting the coordinates of the facial key points and the facial skin region mask based on the general visual features.
[0012] Optionally, in some embodiments, the deep neural network model includes: a facial keypoint coordinate regression loss function.
[0013] ;
[0014] In the formula, The regression loss value is the facial landmark coordinates. These are the coordinates of facial key points labeled during data preprocessing. For face detection networks, The predicted face coordinates are output by the face detection network branch. This represents the total number of facial landmarks. To sum the errors of the coordinates of all facial key points;
[0015] Skin region mask segmentation loss function:
[0016] ;
[0017] In the formula, This represents the loss value for mask segmentation of the skin region. For channel-dimensional cross-entropy, To input the width of the face image, The input is the height of the face image, where x and y are the coordinates of the pixels in the face image. The mask prediction value output by the neural network. The true label is obtained by using a region change mask in data preprocessing. The percentage of genuine labels. For the softmax function, The maximum index of the softmax function based on the channel dimension is used to output the mask to the network, which is converted into a one-hot predicted label.
[0018] Optionally, in some embodiments, after generating the mesh partition, the method further includes: region validity verification, specifically including: calculating the percentage of effective pixel area identified as skin in each sub-region of the mesh partition based on the facial skin region mask output by the facial partitioning model; and marking sub-regions with the percentage of effective pixel area lower than a preset threshold as invalid regions.
[0019] Secondly, this application also provides a facial partitioning system applicable to any of the facial partitioning methods described above, the facial partitioning system comprising:
[0020] The image acquisition module is used to acquire multiple sets of facial images, including frontal facial images and side facial images;
[0021] The data annotation module is used to annotate facial key points and facial skin region masks for each of the facial images, forming an image-point set-mask triplet dataset, wherein the facial key points include predetermined anatomical locations;
[0022] The initial analysis model construction module is used to generate an initial analysis model using the triplet dataset as sample data through a deep neural network model.
[0023] The face partitioning model construction module is used to integrate predefined mesh partitioning generation rules into the initial analysis model to generate a face partitioning model. The face partitioning model is used to output the coordinates of facial key points, facial skin region masks, and mesh partitions generated based on the key points.
[0024] The technical solution provided in this application has the following advantages compared with the prior art:
[0025] The full-coverage face partitioning method provided in this application collects multiple sets of face images including frontal and side views, and annotates the images with facial key points and skin region masks to form an image-point set-mask triplet dataset, which can achieve full-range coverage of the face (frontal and side views). Furthermore, using the triplet dataset as sample data, an initial analysis model is generated through a deep neural network model, and then predefined grid partitioning generation rules are integrated into the initial analysis model to generate a face partitioning model. By integrating the grid partitioning generation rules, the grid partitioning output by the face partitioning model is more accurate.
[0026] Furthermore, the full-coverage facial segmentation method provided in this application embodiment also uses a skin mask filtering mechanism to eliminate interference from non-skin areas, thereby improving the accuracy of extracting features such as skin color and texture within each grid segment. Attached Figure Description
[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.
[0029] Figure 1 A schematic flowchart of the full-coverage facial partitioning method provided in the embodiments of this application;
[0030] Figure 2 This is a schematic diagram illustrating a specific application of the full-coverage facial partitioning method provided in the embodiments of this application;
[0031] Figure 3 This is a schematic diagram of the structure of the full-coverage facial partitioning system provided in the embodiments of this application. Detailed Implementation
[0032] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0033] Many specific details are set forth in the following description in order to provide a full understanding of this application, but this application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of this application, and not all embodiments.
[0034] The full-coverage facial partitioning method and system provided in this application will be described exemplarily below with reference to the accompanying drawings. Figure 1 This is a schematic flowchart illustrating the full-coverage facial segmentation method provided in an embodiment of this application. (Refer to...) Figure 1 The full-coverage facial segmentation method 100 includes the following steps:
[0035] S101. Collect multiple sets of facial images, including frontal facial images and side facial images.
[0036] Specifically, firstly, using a capture device, such as an RGB camera with a resolution of 1920x1080 pixels, under a lighting environment (such as a standard D65), captures facial images of the user from the front and sides (left or right 90°). These facial images can be scaled down to a width of 1024 pixels and illuminated using a histogram equalization algorithm to improve image contrast and detail visibility.
[0037] It is understood that the above description of acquiring multiple sets of facial images is merely exemplary. Those skilled in the art can select and set the acquisition of multiple sets of facial images according to actual applications, as long as the technical principles of this application can be achieved.
[0038] S102. Label the facial key points and facial skin region masks for each of the facial images to form an image-point set-mask triplet dataset, wherein the facial key points include predetermined anatomical locations.
[0039] Specifically, based on the facial image acquired in step S101, key facial points can be marked. These key facial points can include not only the outline of the face, but also facial features and their edges, such as eyes, inner corners of the eyes, outer corners of the eyes, nose, mouth, eyebrows, ears, etc. These key facial points include predetermined anatomical positions, which can be standard positions based on human anatomy.
[0040] Furthermore, in addition to annotating facial key points, facial skin area masks can also be annotated, thereby clearly distinguishing skin areas and non-skin areas (such as hair, eyebrows, and background) in facial images, ensuring that only skin-related information is processed in image processing.
[0041] By combining labeled facial keypoints and facial skin region masks, a triplet dataset containing facial images, point sets, and skin masks can be formed, which can provide accurate data for facial partitioning.
[0042] S103. Using the triplet dataset as sample data, generate an initial analysis model through a deep neural network model.
[0043] Specifically, based on the constructed image-point-mask triplet dataset—that is, multiple data pairs including facial images, point sets, and skin masks—the triplet dataset is used as training samples and input into a deep neural network model to generate an initial analysis model. This deep neural network model is used for facial image analysis and can learn to extract useful features from the input data and identify key anatomical landmarks and skin regions of the face. Through iterative training, the deep neural network model continuously optimizes its internal parameters, generating the initial analysis model.
[0044] Optionally, in one embodiment, the deep neural network model may include: a feature extractor for extracting general visual features from the input face image; and two sub-network branches for outputting the coordinates of the facial key points and the facial skin region mask based on the general visual features.
[0045] Specifically, the deep neural network model can include: one feature extractor and two sub-network branches. The sub-network branches output the coordinates of facial key points and the facial skin region mask, respectively. The feature extractor shares parameters between the two sub-network branches and can consist of 12 facial feature extraction modules, with four convolutional layers that are downsampled by 2x. The structure of the facial feature extraction module can be as follows: it starts with a 1x1 convolution kernel + PreLU activation function to double the number of input channels, because the subsequent modules will be divided into two branches. If downsampling by 1x is required, the first kernel on the left is a 3x3 convolution with a stride of 2; the right side is set with one average pooling layer. Otherwise, the stride on the left is 1 and no average pooling is set. The left branch is subsequently set with multiple 3x3 convolution kernels and branches because the input views are diverse and the facial statistical regions are diverse. In order to improve the extraction capability of the facial feature extraction module, different feature maps are added, and finally the channels of the four feature map branches are merged together.
[0046] Optionally, in one embodiment, the deep neural network model may include: a facial keypoint coordinate regression loss function.
[0047] ;
[0048] In the formula, The regression loss value is the facial landmark coordinates. These are the coordinates of facial key points labeled during data preprocessing. For face detection networks, The predicted face coordinates are output by the face detection network branch. This represents the total number of facial landmarks. This is used to sum the errors of the coordinates of all facial key points.
[0049] Skin region mask segmentation loss function:
[0050] ;
[0051] In the formula, This represents the loss value for mask segmentation of the skin region. For channel-dimensional cross-entropy, To input the width of the face image, The input is the height of the face image, where x and y are the coordinates of the pixels in the face image. The mask prediction value output by the neural network. The true label is obtained by using a region change mask in data preprocessing. The percentage of genuine labels. For the softmax function, The maximum index of the softmax function based on the channel dimension is used to output the mask to the network, which is converted into a one-hot predicted label.
[0052] Specifically, the deep neural network model has two network branch outputs. Therefore, corresponding to the facial key point coordinate regression loss function and the skin region mask segmentation loss function, through successive training, the loss of the above two functions is minimized to complete the training of the deep neural network model.
[0053] S104. Integrate the predefined mesh partitioning generation rules into the initial analysis model to generate a face partitioning model. The face partitioning model is used to output the coordinates of facial key points, facial skin region masks, and mesh partitions generated based on the key points.
[0054] Specifically, predefined mesh partitioning rules are integrated into the initial analysis model to generate a more accurate face partitioning model. This face partitioning model is used to process and analyze facial images, capable of identifying and outputting facial keypoint coordinates and automatically generating mesh partitions based on these coordinates. Furthermore, the face partitioning model can also generate a facial skin region mask, which can distinguish between facial skin and non-skin regions.
[0055] The predefined mesh partitioning generation rules can be rules or algorithms designed based on facial anatomical features, geometric principles, and specific application requirements. When setting the mesh partitioning generation rules, it is necessary to ensure that the natural anatomical structure of the face, such as bones, muscles, and skin, as well as the position and shape of the facial features, are taken into account, while incorporating geometric principles such as symmetry, proportion, and angles, to guarantee the uniformity and regularity of facial partitioning.
[0056] Furthermore, when setting the grid partitioning generation rules, the design can also be based on the facial differences of different individuals, such as race, age, gender, etc., to improve the flexibility and scalability of facial partitioning.
[0057] Optionally, in one embodiment, the grid partitioning generation rule may include: determining a horizontal baseline based on the facial key points; determining multiple horizontal and vertical reference points based on the facial key points; using the horizontal baseline as a reference, generating a grid composed of multiple sub-regions through the horizontal and vertical reference points; wherein the boundaries of the sub-regions are parallel or perpendicular to the horizontal baseline.
[0058] Specifically, a horizontal baseline is determined based on key facial points. This horizontal baseline can be determined by connecting the left and right outer corners of the eyes, thereby providing a horizontal reference for facial zoning.
[0059] Multiple horizontal and vertical reference points can be determined based on facial key points. For example, in a frontal view of the face, multiple horizontal reference points can be determined based on anatomical landmarks, and multiple auxiliary horizontal baselines of the same length as the horizontal baseline can be obtained by passing through these multiple horizontal reference points.
[0060] Horizontal and vertical reference points provide precise vertical and horizontal positioning for facial partitioning. Furthermore, the selection of these reference points takes into account the natural features and anatomical structure of the face, ensuring the rationality and scientific basis of facial partitioning.
[0061] Using a horizontal baseline as a reference, a mesh consisting of multiple sub-regions is generated by connecting these horizontal and vertical reference points. This mesh covers the entire face (including the front and side views), and the boundaries of each sub-region are parallel or perpendicular to the horizontal baseline, making facial segmentation more refined and accurate.
[0062] For example, determining the horizontal baseline based on the facial key points may include: for a frontal facial image, extracting facial key points corresponding to the left outer corner of the eye and the right outer corner of the eye, and determining the line connecting the left outer corner of the eye and the right outer corner of the eye as the horizontal baseline; for a side facial image, extracting facial key points corresponding to the outer corner of the eye and the starting point on the same side of the ear, and determining the line connecting the outer corner of the eye and the starting point on the same side of the ear as the horizontal baseline.
[0063] Specifically, for a frontal facial image, the horizontal baseline can be determined, for example, by connecting the points of the left and right outer corners of the eyes. The outer corner of the eye is one of the key anatomical landmarks of the face, and is particularly important for facial symmetry. Therefore, the straight line formed by connecting the two points of the left and right outer corners of the eyes can provide an accurate reference line.
[0064] For lateral facial images, the determination of the horizontal baseline should adapt to the variations in facial features from a lateral view. In this case, the horizontal baseline can be determined by connecting the outer corner of the eye to the starting point on the same side of the ear. The starting point on the ear typically refers to the point where the upper part of the ear meets the side of the head. The line connecting this point to the outer corner of the eye better reflects the anatomical structure of the face from a lateral view, providing an accurate benchmark for partitioning lateral facial images.
[0065] Optionally, in one embodiment, determining multiple horizontal and vertical reference points based on the facial key points may include: determining multiple horizontal reference points by vector calculation based on a predefined first set of key points; and determining multiple vertical reference points based on a predefined second set of key points.
[0066] Specifically, the first set of predefined key points corresponds to, for example, the following anatomical locations: forehead apex, upper forehead boundary, forehead center, midpoint of the upper edge of the brow ridge, midpoint of the lower edge of the brow ridge, midpoint of the line connecting the supraorbital foramina, midpoint of the line connecting the outer corners of the eyes, midpoint of the line connecting the upper edges of the zygomatic arch, midpoint of the line connecting the infraorbital foramina, lower edge of the nasal ala, midpoint of the philtrum groove, midpoint of the upper edge of the upper lip, midpoint of the lower edge of the lower lip, midpoint of the lip and chin, and chin tip.
[0067] Based on the relative positions of the first set of key points, multiple horizontal reference points can be calculated, which can serve as the horizontal boundaries of the grid partitions.
[0068] Multiple vertical reference points are determined based on a predefined second set of key points. These key points may correspond to, for example, the following anatomical locations: the right infraorbital foramen, the right inner canthus, the midpoint of the nasal root, the left inner canthus, and the left infraorbital foramen. By analyzing the distribution of these key points, multiple vertical reference points can be determined, which can serve as the vertical boundaries of the grid partitions.
[0069] It is understandable that the first set of key points and the second set of key points are based on the correspondence between facial features and the internal organs and meridians, so that the facial zoning method can provide support for TCM diagnosis.
[0070] Optionally, in one embodiment, generating a grid consisting of multiple sub-regions based on the horizontal baseline and through the horizontal and vertical reference points may include: generating a horizontal line parallel to the horizontal baseline through each of the horizontal reference points; vertically projecting each of the vertical reference points onto each of the horizontal lines to form grid intersections; and connecting adjacent grid intersections to form multiple quadrilateral sub-regions.
[0071] Specifically, a horizontal line parallel to the horizontal baseline is generated using various horizontal reference points. Based on the determined horizontal baseline, a horizontal line of the same length as the horizontal baseline can be obtained using multiple horizontal reference points.
[0072] Each vertical reference point is projected vertically onto each horizontal line to form a grid intersection. The vertical reference points are determined based on a predefined second set of key points and are distributed along the width of the face. By projecting the vertical reference points onto each horizontal line, multiple grid intersections can be obtained on each horizontal plane, and these grid intersections define the vertical boundaries of the grid partitions.
[0073] By connecting the intersections of adjacent grid lines, multiple quadrilateral sub-regions are formed. These sub-regions, such as rectangles or squares, cover every part of the facial image, and each sub-region is parallel or perpendicular to a horizontal baseline. This method of generating a regular grid allows the facial image to be divided into multiple sub-regions with clearly defined boundaries and relative positions.
[0074] Optionally, in one embodiment, after generating the mesh partition, the method further includes: region validity verification, specifically including: calculating the percentage of effective pixel area identified as skin in each sub-region of the mesh partition based on the facial skin region mask output by the facial partitioning model; and marking sub-regions with the percentage of effective pixel area lower than a preset threshold as invalid regions.
[0075] Specifically, after completing the mesh partitioning, the method may further include region validity verification. Region validity verification ensures that each sub-region contains a sufficient number of valid skin pixels. Specifically, region validity verification relies on the facial skin region mask output by the facial partitioning model. The facial skin region mask is a two-dimensional array where each pixel value indicates whether the pixel at that location belongs to the skin region. Using the facial skin region mask, the percentage of valid pixels identified as skin within each sub-region can be calculated.
[0076] For example, a threshold (such as 20%) can be set to determine the validity of a sub-region. If the percentage of valid pixels within a sub-region is lower than the set threshold, that sub-region will be marked as invalid. By marking invalid regions, the accuracy and reliability of facial image analysis can be further improved.
[0077] Figure 2 The following is a schematic diagram illustrating a specific application of the full-coverage facial segmentation method provided in the embodiments of this application. Figure 2 The specific application of the full-coverage facial zoning method is described in detail.
[0078] ① Image Acquisition and Preprocessing: Using an RGB camera with a resolution of 1920x1080 pixels, images of the user's face from the front and a 90-degree angle to the left (or right) side were acquired under standard D65 lighting conditions. The images were uniformly scaled to 1024px (width), and a histogram equalization algorithm was used for illumination compensation. Multiple (e.g., 130) facial key points were annotated on the acquired facial images, including the facial contours and the edges of facial features. Simultaneously, facial skin region masks were annotated, forming data pairs of image-point set-skin region mask.
[0079] ② Training the detection and segmentation model: The image is scaled to 256x256 and input into a deep neural network model. The deep neural network model consists of one feature extractor and two sub-network branches (i.e., output branches). The sub-network branches output the coordinates of facial key points and the facial skin region mask, respectively. The feature extractor shares parameters between the two sub-network branches and consists of 12 facial feature extraction modules, with four convolutional layers downsampled by 2x. The structure of the facial feature extraction module is as follows: It starts with a 1x1 convolution kernel and a PreLU activation function, doubling the number of input channels. Since the module will be divided into two branches, if downsampling by 1x is required, the first kernel on the left is a 3x3 convolution with a stride of 2; an average pooling layer is set on the right. Otherwise, the stride on the left is 1 and no average pooling is set. The left branch then sets multiple 3x3 convolution kernels and branches, and finally the four feature map branches are merged together.
[0080] ③ Inference process after training: The trained initial analysis model has the ability to regress the coordinates of key points on the front and side of the face and the mask of the facial skin region. At this point, inputting the facial image into the initial analysis model will yield multiple facial key points and facial skin region masks.
[0081] ④ Predefined grid partitioning generation rules:
[0082] a. For example, the coordinates of the outer corners of the right and left eyes can be extracted, calculated, and connected into a line, which is defined as the horizontal baseline AB. The outline points of the left and right ears can be extracted and combined with the outer corner points to determine the horizontal baseline of the side view.
[0083] Front view: The pixel coordinates of multiple horizontal reference points are calculated using vector coordinates. For example, the midpoint of the upper edge of the eyebrow can be obtained by averaging the centers of the upper edges of the left and right eyebrows; the lower boundary of the tear trough can be obtained by averaging the lower points of the left and right eyelids.
[0084] Side view: Similarly, multiple horizontal reference points (e.g., 15) can be calculated based on 130 facial key points. For example, the midpoint between the bridge of the nose and the opening of the external auditory canal can be obtained by taking the midpoint between the vertical coordinate of the tip of the nose and the vertical coordinate of the ear point.
[0085] b. Mesh generation and facial skin mask filtering:
[0086] Front view: Based on the Y coordinates of 15 horizontal reference points, draw 15 horizontal lines. Project the X coordinates of multiple vertical reference points onto each horizontal line to form grid intersections, and then use computational geometry to divide the region into multiple quadrilateral grids.
[0087] Side view: Generates multiple quadrilateral meshes based on multiple horizontal and multiple vertical reference points.
[0088] c. Traverse each sub-region: If the threshold ratio is set to 20%, if the proportion of skin pixels is lower than this threshold, mark it as invalid in the statistical list of the grid; otherwise, extract the pixels of the skin region in the grid from the original image for subsequent analysis (such as RGB mean, colorimetric statistics, etc.).
[0089] The facial segmentation method described in this application can provide treatment plans for skin problems in different segments. For example, after segmenting the face using the facial segmentation method, a skin problem knowledge base is invoked to diagnose skin problems based on prior knowledge of the regions, and treatment plans are provided in combination with the skin problems in different segments.
[0090] In summary, the full-coverage face partitioning method 100 provided in this application collects multiple sets of face images including frontal and side views, and annotates the images with facial key points and skin region masks to form an image-point set-mask triplet dataset, which can achieve full-range coverage of the face (frontal and side views). Furthermore, using the triplet dataset as sample data, an initial analysis model is generated through a deep neural network model, and then predefined grid partitioning generation rules are integrated into the initial analysis model to generate a face partitioning model. By integrating the grid partitioning generation rules, the grid partitioning output by the face partitioning model is more accurate.
[0091] Furthermore, the full-coverage facial segmentation method provided in this application embodiment also uses a skin mask filtering mechanism to eliminate interference from non-skin areas, thereby improving the accuracy of extracting features such as skin color and texture within each grid segment.
[0092] Based on the same inventive concept, this application also provides a full-coverage facial partitioning system 200. Figure 2 This is a schematic diagram of the structure of the full-coverage facial partitioning system provided in an embodiment of this application. Figure 2 As shown, the full-coverage facial partitioning system 200 may include:
[0093] The image acquisition module 201 is used to acquire multiple sets of facial images, including frontal facial images and side facial images;
[0094] The data annotation module 202 is used to annotate facial key points and facial skin region masks for each of the facial images to form an image-point set-mask triplet dataset, wherein the facial key points include predetermined anatomical locations;
[0095] The initial analysis model construction module 203 is used to generate an initial analysis model using the triplet dataset as sample data through a deep neural network model.
[0096] The face partitioning model construction module 204 is used to integrate predefined mesh partitioning generation rules into the initial analysis model to generate a face partitioning model. The face partitioning model is used to output the coordinates of facial key points, facial skin region masks, and mesh partitions generated based on the key points.
[0097] The full-coverage facial partitioning system provided in the above embodiments can execute the full-coverage facial partitioning method provided in the above embodiments and has the same or corresponding beneficial effects, which will not be described in detail here.
[0098] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments.
[0099] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.
[0100] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.
[0101] In this application, unless otherwise stated, directional terms such as "up" and "down" are generally used in relation to the direction shown in the accompanying drawings, or in relation to the vertical, perpendicular, or gravitational direction; similarly, for ease of understanding and description, "left" and "right" are generally used in relation to the left and right shown in the accompanying drawings; "inner" and "outer" refer to the inner and outer contours of each component itself, but the above directional terms are not intended to limit this application.
[0102] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for full-coverage facial segmentation, characterized in that, include: Multiple sets of facial images are acquired, including frontal facial images and side facial images; Each of the aforementioned facial images is labeled with facial key points and facial skin region masks to form an image-point set-mask triplet dataset, wherein the facial key points include predetermined anatomical locations; Using the triplet dataset as sample data, an initial analysis model is generated through a deep neural network model; The predefined mesh partitioning generation rules are integrated into the initial analysis model to generate a face partitioning model. The face partitioning model is used to output the coordinates of facial key points, facial skin region masks, and mesh partitions generated based on the key points. Based on the facial skin region mask output by the facial segmentation model, the proportion of effective pixel area identified as skin in each sub-region of the grid partition is calculated; sub-regions with an effective pixel area proportion lower than a preset threshold are marked as invalid regions and removed, and the remaining effective sub-regions are used for feature extraction.
2. The facial segmentation method according to claim 1, characterized in that, The grid partitioning generation rules include: A horizontal baseline is determined based on the aforementioned facial key points; Multiple horizontal and vertical reference points are determined based on the aforementioned facial key points; Using the horizontal baseline as a reference, a grid consisting of multiple sub-regions is generated through the horizontal reference point and the vertical reference point; The boundaries of the sub-regions are either parallel or perpendicular to the horizontal baseline.
3. The facial segmentation method according to claim 2, characterized in that, The determination of the horizontal baseline based on the facial key points includes: For a frontal facial image, extract the facial key points corresponding to the left and right outer corners of the eyes, and determine the line connecting the left and right outer corners of the eyes as the horizontal baseline; For a side view image, extract the facial key points corresponding to the outer corner of the eye and the starting point on the same side of the ear, and determine the line connecting the outer corner of the eye and the starting point on the same side of the ear as the horizontal baseline.
4. The facial segmentation method according to claim 2, characterized in that, The determination of multiple horizontal and vertical reference points based on the facial key points includes: Based on a predefined first set of key points, multiple horizontal reference points are determined through vector calculation. Based on a predefined second set of key points, multiple vertical reference points are determined.
5. The facial segmentation method according to claim 4, characterized in that, The first set of key points corresponds to the following anatomical locations: forehead vertex, upper forehead boundary, forehead center point, midpoint of the upper edge of the brow ridge, midpoint of the lower edge of the brow ridge, midpoint of the line connecting the supraorbital foramina, midpoint of the line connecting the outer corners of the eyes, midpoint of the line connecting the upper edges of the zygomatic arch, midpoint of the line connecting the infraorbital foramina, lower edge of the nasal ala, midpoint of the philtrum groove, midpoint of the upper edge of the upper lip, midpoint of the lower edge of the lower lip, midpoint of the lip and chin, and chin tip. The second set of key points corresponds to the following anatomical locations: right infraorbital foramen, right inner canthus, midpoint of the root of the nose, left inner canthus, and left infraorbital foramen.
6. The facial segmentation method according to claim 5, characterized in that, The step of generating a grid consisting of multiple sub-regions based on the horizontal baseline and using the horizontal and vertical reference points includes: A horizontal line parallel to the horizontal baseline is generated using each of the aforementioned horizontal reference points; Each of the vertical reference points is vertically projected onto each of the horizontal lines to form grid intersections; Connect the intersections of adjacent grids to form multiple quadrilateral sub-regions.
7. The facial segmentation method according to claim 1, characterized in that, The deep neural network model includes: One feature extractor is used to extract general visual features from the input face image; Two sub-network branches are used to output the coordinates of the facial key points and the facial skin region mask based on the general visual features.
8. The facial segmentation method according to claim 7, characterized in that, The deep neural network model includes: Facial landmark coordinate regression loss function: ； In the formula, The regression loss value is the facial landmark coordinates. These are the coordinates of facial key points labeled during data preprocessing. For face detection networks, The predicted face coordinates are output by the face detection network branch. This represents the total number of facial landmarks. To sum the errors of the coordinates of all facial key points; Skin region mask segmentation loss function: ； In the formula, This represents the loss value for mask segmentation of the skin region. For channel-dimensional cross-entropy, To input the width of the face image, The input is the height of the face image, where x and y are the coordinates of the pixels in the face image. The mask prediction value output by the neural network. The true label is obtained by using a region change mask in data preprocessing. The percentage of genuine labels. For the softmax function, The maximum index of the softmax function based on the channel dimension is used to output the mask to the network, which is converted into a one-hot predicted label.
9. A facial partitioning system, applicable to the facial partitioning method of any one of claims 1 to 8, said facial partitioning system comprising: The image acquisition module is used to acquire multiple sets of facial images, including frontal facial images and side facial images; The data annotation module is used to annotate facial key points and facial skin region masks for each of the facial images, forming an image-point set-mask triplet dataset, wherein the facial key points include predetermined anatomical locations; The initial analysis model construction module is used to generate an initial analysis model using the triplet dataset as sample data through a deep neural network model. A face partitioning model construction module is used to integrate predefined mesh partitioning generation rules into the initial analysis model to generate a face partitioning model. The face partitioning model is used to output the coordinates of facial key points, facial skin region masks, and mesh partitions generated based on the key points. The region validity verification module is used to calculate the percentage of effective pixels identified as skin in each sub-region of the grid partition based on the facial skin region mask output by the facial partitioning model; sub-regions with an effective pixel area percentage lower than a preset threshold are marked as invalid regions and removed, and the remaining effective sub-regions are used for feature extraction.