A method, apparatus, electronic device, and storage medium for matching hairstyles.

Abstract

This application provides a hairstyle matching method, apparatus, electronic device, and storage medium. The method includes: obtaining an actual hair mask of a face image using a face segmentation algorithm; extracting multiple source hair masks by rendering a face model file exported from a face-shaping platform, wherein each face model file corresponds to a hairstyle; aligning the actual hair mask and the source hair masks for face matching; comparing the aligned actual hair mask and the source hair masks, and selecting the source hair mask with the highest overlap as the matching hairstyle for the face image. This application allows for the determination of a matching hairstyle even after the hairstyle source material has been updated, reducing computational load.

Description

Technical Field

[0001] This application relates to the field of AI face-shaping technology, and more particularly to a hairstyle matching method, device, electronic device, and storage medium. Background Technology

[0002] With the development of science and technology and the needs of modern industrial technology, 3D virtual digital humans have become a common research hotspot in academia and industry. This technology has wide applications in virtual characters and other interactive games and entertainment. Among these applications, AI facial sculpting is an important technological direction, which involves inputting a facial image and having an algorithm reconstruct a digital image of that person, with a high degree of similarity to the input photo. In particular, the choice of hairstyle greatly affects the degree of personalization; different hairstyles (long or short hair, with or without bangs, etc.) make the same person look completely different.

[0003] The common approach to hairstyle matching in existing technologies is to train a hairstyle classification model to classify the input image. However, the hairstyle classification model requires that the types of hairstyles be defined and requires a large number of labeled images. If the hairstyle library of the face-shaping platform is updated, then the model needs to be re-labeled and retrained, which is too computationally intensive. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this application provides a hairstyle matching method, apparatus, electronic device and storage medium.

[0005] In a first aspect, this application provides a method for matching hairstyles, the method comprising:

[0006] The actual hair mask of the face image is obtained using a face segmentation algorithm;

[0007] By rendering the face model files exported from the face-shaping platform, multiple hair masks are extracted, with each face model file corresponding to a hairstyle.

[0008] Align the actual hair mask and the source hair mask for a human face;

[0009] The actual hair mask after face alignment is compared with the source hair mask, and the source hair mask with the highest degree of overlap is taken as the matching hairstyle of the face image.

[0010] Optionally, by rendering the face model file exported from the face-shaping platform, multiple source hair masks can be extracted, including:

[0011] For the face model file exported from the face-shaping platform, 3D modeling software is used for rendering to obtain a preset rendering result, wherein the preset rendering result is a digital face model with hair;

[0012] After deleting the hair mesh from the face model file, the model is rendered to obtain the target rendering result, wherein the target rendering result is a digital face model without hair.

[0013] The preset rendering result and the target rendering result are aligned to form a face;

[0014] Subtract the pixels of the preset rendering result from the pixels of the target rendering result after aligning the face;

[0015] If the pixel difference after subtraction is greater than the preset tolerance value, then the pixel corresponding to the pixel difference is used as the hair mask of the material.

[0016] Optionally, aligning the preset rendering result and the target rendering result for faces includes:

[0017] Select either the preset rendering result or the target rendering result to perform key point detection and obtain the key point coordinates;

[0018] Align the key point coordinates with the preset coordinates in the preset template to determine the mapping matrix;

[0019] By applying the mapping matrix to the preset rendering result and the target rendering result, the preset rendering result and the target rendering result are aligned to the same position.

[0020] Optionally, aligning the actual hair mask and the source hair mask for a face includes:

[0021] In the actual hair mask and the material hair mask, at least three key points on the facial contour and the tip of the nose are determined respectively;

[0022] Based on the mapping relationship between the corresponding key points, the affine matrix is ​​obtained;

[0023] The affine matrix is ​​applied to the actual hair mask to align the face between the actual hair mask and the source hair mask.

[0024] Optionally, applying the affine matrix to the actual hair mask to align the face between the actual hair mask and the source hair mask includes:

[0025] When performing an affine transformation on the actual hair mask, the key point coordinates of the actual hair mask are shifted up or down according to a set offset, so that the hairline of the transformed actual hair mask and the hairline of the source hair mask can be aligned.

[0026] Optionally, the comparison between the actual hair mask after face alignment and the source hair mask, and the selection of the source hair mask with the highest overlap as the matching hairstyle of the face image, includes:

[0027] For the actual hair mask and the source hair mask after face alignment, calculate the intersection-union ratio (CUI), where the CUI indicates the degree of overlap between the actual hair mask and the source hair mask;

[0028] The hair mask with the largest intersection and comparison is used as the matching hairstyle for the face image.

[0029] Optionally, the calculation of the intersection-union ratio (IU / R) for the actual hair mask after face alignment and the source hair mask includes:

[0030] The actual hair mask after face alignment and the source hair mask are divided into multiple regions respectively;

[0031] Calculate the intersection-union ratio of regions within the same area;

[0032] The crossover-union ratio is obtained by calculating the weighted average of the region crossover-union ratio and the corresponding region weight.

[0033] Secondly, a hairstyle matching device is provided, the device comprising:

[0034] The module is used to obtain the actual hair mask of the face image using a face segmentation algorithm;

[0035] The extraction module is used to extract multiple hair masks from face model files exported from the face modeling platform by rendering the model. Each face model file corresponds to a hairstyle.

[0036] An alignment module is used to align the actual hair mask and the source hair mask for a human face.

[0037] The comparison module is used to compare the actual hair mask after face alignment with the source hair mask, and select the source hair mask with the highest degree of overlap as the matching hairstyle of the face image.

[0038] Thirdly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0039] Memory, used to store computer programs;

[0040] A processor, when executing a program stored in memory, implements any of the steps of the described hairstyle matching method.

[0041] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements any of the steps of the hairstyle matching method described above.

[0042] The technical solutions provided in this application have the following advantages compared with the prior art:

[0043] The method provided in this application provides a method where each face model file corresponds to a hairstyle. By rendering the face model file exported from a face-shaping platform, multiple source hair masks are extracted. The actual hair mask of the face image is compared with these multiple source hair masks for face alignment, and the source hair mask with the highest overlap is used as the matching hairstyle for the face image. This application does not require a neural network model, and even after the hairstyle source materials are updated, this method can still be used to determine the matching hairstyle, eliminating the need for re-annotating training data and retraining the model, thus reducing computational load. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the hardware environment for a hairstyle matching method provided in an embodiment of this application;

[0047] Figure 2 A flowchart illustrating a hairstyle matching method provided in this application embodiment;

[0048] Figure 3 A schematic diagram of a face image and a segmented mask provided in an embodiment of this application;

[0049] Figure 4 A flowchart of a hairstyle matching method provided in an embodiment of this application;

[0050] Figure 5 This is a schematic diagram of the structure of a hairstyle matching device provided in an embodiment of this application;

[0051] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustration and has no specific meaning in itself. Therefore, "module" and "part" may be used interchangeably.

[0054] To address the problems mentioned in the background art, according to one aspect of the embodiments of this application, an embodiment of a hairstyle matching method is provided.

[0055] Optionally, in the embodiments of this application, the above-described hairstyle matching method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal 101 and server 103. Figure 1 As shown, server 103 is connected to terminal 101 via a network and can be used to provide services to the terminal or clients installed on the terminal. Database 105 can be set up on the server or independently of the server to provide data storage services for server 103. The network mentioned above includes, but is not limited to, wide area network, metropolitan area network or local area network. Terminal 101 includes, but is not limited to, PC, mobile phone, tablet computer, etc.

[0056] One hairstyle matching method in this application embodiment can be executed by server 103, which can quickly match hairstyles even after the hairstyle material is updated, thereby reducing the amount of computation.

[0057] The following will describe in detail a hairstyle matching method provided in the embodiments of this application, with reference to specific implementation methods. Figure 2 As shown, the specific steps are as follows:

[0058] Step 201: Use a face segmentation algorithm to obtain the actual hair mask of the face image.

[0059] In this embodiment, the terminal sends a face image to the server. The server obtains the contour feature points (landmarks) of the face image based on the face image. The contour feature points indicate the contours of eyebrows, eyes, nose, mouth, and face, and each contour feature point has its own semantic information. This application can use a 106-point keypoint detection module to obtain the contour feature points, or it can use other detection models to obtain the contour feature points.

[0060] After obtaining the facial contour feature points, the server selects at least three reference points for face alignment. For example, the centers of the left and right pupils and the tip of the nose are selected as reference points. Specifically, a transformation matrix is ​​obtained by calculating the positional mapping relationship between these three reference points and the corresponding points on the reference template. Applying this transformation matrix to the entire face image aligns the face. The aligned face can be considered a frontal face. Applying the transformation matrix to the contour feature points yields the contour feature points in the aligned face image. If the received input is a side-view image, this algorithm can be used to detect the angle of the side view and align the input face image to a frontal view based on this angle, achieving better matching results.

[0061] The server uses a face segmentation algorithm that can segment various parts of the face (including face, mouth, nose, eyes, hair, etc.) in the input face image. The face segmentation algorithm can use the BiseNet face segmentation model. This application does not impose specific restrictions on the face segmentation algorithm.

[0062] Compared to facial features, hair has a coarser texture, so a hair mask segmented from the hair can accurately represent the shape of the hair. Figure 3 This is a diagram of a face image and its segmented mask, where the mask represents the hair. As you can see, the actual hair mask includes the facial contours and the hair shape.

[0063] Step 202: Render the face model file exported from the face-shaping platform and extract multiple hair masks.

[0064] Each face model file corresponds to a hairstyle.

[0065] In this embodiment, a variety of different hairstyles are predefined in the face-shaping platform. Since the face-shaping platform has multiple styles, the segmentation algorithm may have errors in segmenting the hair area. This application performs model rendering on the face model file exported from the face-shaping platform and extracts multiple material hair masks. The material hair masks contain the facial contour and hair shape of the face model.

[0066] Step 203: Align the actual hair mask and the source hair mask for the face.

[0067] In this embodiment, the position and size of the input face image may differ from the source hair mask. Therefore, it is necessary to first align the actual hair mask and the source hair mask. Generally, a facial landmark detection algorithm is used to detect the key points of the face, and the center point of the eyes and the tip of the nose are taken as alignment templates. However, the face-shaping platform may contain cartoon characters rather than real people. The eyes of cartoon characters are generally enlarged, so using the center point of the eyes for alignment will cause the face image and the cartoon material to not be aligned to the same template. Therefore, this application uses at least three points, including the two endpoints of the outer contour of the face and a point on the tip of the nose, for alignment and scaling. During alignment, it is necessary to ensure that the hair as a whole is in the image.

[0068] Step 204: Compare the actual hair mask after face alignment with the source hair mask, and take the source hair mask with the highest overlap as the matching hairstyle of the face image.

[0069] In this embodiment of the application, after the server completes face alignment, it compares the actual hair mask and the source hair mask. This comparison can be a similarity comparison or an intersection-union ratio calculation. The source hair mask with the highest overlap is taken as the matching hairstyle of the face image.

[0070] In this application, each face model file corresponds to a hairstyle. The face model file exported from the face-shaping platform is rendered, and multiple source hair masks are extracted. The actual hair mask of the face image is then compared with these multiple source hair masks for face alignment. The source hair mask with the highest overlap is used as the matching hairstyle for the face image. This application does not require a neural network model; even if the hairstyle source material is updated, this method can still determine the matching hairstyle, eliminating the need for re-annotating training data and retraining the model, thus reducing computational load.

[0071] As an optional implementation, the process of rendering a face model file exported from a face-shaping platform and extracting multiple source hair masks includes: rendering the face model file exported from the face-shaping platform using 3D modeling software to obtain a preset rendering result, wherein the preset rendering result is a digital face model with hair; deleting the hair mesh from the face model file and then rendering to obtain a target rendering result, wherein the target rendering result is a digital face model without hair; aligning the preset rendering result and the target rendering result; subtracting the pixels of the preset rendering result from the pixels of the target rendering result after face alignment; if the pixel difference after subtraction is greater than a preset tolerance value, then the pixels corresponding to the pixel difference are used as source hair masks.

[0072] In this embodiment, on the one hand, the face model file exported from the face-shaping platform is rendered using 3D modeling software to obtain a preset rendering result R. hairThe preset rendering result is a digital face model with hair. The 3D modeling software can be Blender; this application does not limit the specific type of 3D modeling software. On the other hand, after exporting the face model file from the face-shaping platform, the hair mesh in the face model file is deleted, and then rendered using 3D modeling software to obtain the target rendering result R. head The target rendering result is a digital face model without hair.

[0073] The server aligns the preset rendering result and the target rendering result for face recognition. Specifically, the server selects either the preset rendering result or the target rendering result for keypoint detection, obtaining at least three keypoint coordinates. These keypoint coordinates are the coordinates of the endpoints of the facial contour and the tip of the nose. The server aligns these keypoint coordinates with preset coordinates in the preset template, calculates the mapping matrix, and then applies the result to the R... hair and R head Apply this mapping matrix to align the preset rendering result and the target rendering result to the same position.

[0074] After the server aligns the face, R hair pixels and R head Pixels are subtracted. If the difference is less than or equal to a preset tolerance value, the pixel is less likely to be hair and is marked as a non-hair pixel. If the difference is greater than the preset tolerance value (e.g., 10), the pixel is more likely to be hair and is marked as a hair pixel. The pixel corresponding to the difference is then used as the hair mask. The mask can be considered as an image of the same size as the original image, where pixel values ​​are only 0 and 1, where 0 represents non-hair and 1 represents hair.

[0075] Preferably, the server applies a closing operation to the hair mask of the material, that is, first dilates the image and then erodes it to close the small cracks in the hair mask of the material, making the hair mask of the material more continuous and complete.

[0076] In this application, the server obtains a preset rendering result with hair and a target rendering result without hair by rendering. After aligning the two images, the pixels corresponding to the pixel difference values ​​are used as the material hair mask. Regardless of the hairstyle, the above method can be used to obtain an accurate material hair mask. Compared with the segmentation error caused by the segmentation algorithm in the prior art, this application improves the accuracy of hair region segmentation.

[0077] As an optional implementation, face alignment between the actual hair mask and the source hair mask includes: the server determining at least three key points on the facial contour and the tip of the nose in both the actual hair mask and the source hair mask, wherein the three key points can define a facial contour, and the key points on the facial contour are at least two endpoints of the outer contour of the face; obtaining an affine matrix based on the mapping relationship between the corresponding key points; applying the affine matrix to the actual hair mask to realize the position and size adjustment, or rotation, skewing, and other operations of the actual hair mask, thereby aligning the face between the actual hair mask and the source hair mask.

[0078] During the alignment process, facial alignment has a lower priority; the key is to align the hairline, otherwise, the comparison calculation of the bangs will be inaccurate. If a facial keypoint algorithm with forehead contour detection is applied, the keypoints of the hairline can be aligned directly. Otherwise, an offset can be set based on empirical values, and the keypoint coordinates of the actual hair mask can be moved up or down according to the pre-set offset to align the hairline of the actual hair mask with that of the source hair mask.

[0079] In this application, to improve the accuracy of image comparison, the actual hair mask and the source hair mask need to be scaled and aligned beforehand. This application uses the endpoints of the facial contour and the tip of the nose for face alignment, which can effectively prevent alignment differences caused by the alignment of the eye center point. To further avoid deviations in the comparison of the bangs, the key points of the hairline are used for alignment.

[0080] As an optional implementation, comparing the actual hair mask after face alignment with the source hair mask and selecting the source hair mask with the highest overlap as the matching hairstyle of the face image includes: calculating the overlap between the actual hair mask and the source hair mask using the intersection-over-union (IoU) ratio. Here, the intersection-over-union (IoU) ratio is the area of ​​the intersection of two rectangles / the area of ​​the union of the two rectangles. Therefore, after obtaining the face image mask, the IoU can be calculated with the source mask, and the source with the largest IoU can be considered to have the highest similarity.

[0081] Calculating the IoU of the entire image directly can easily lead to false detections. For example, with short hair and long hair, the IoU of short hair is higher than that of long hair above the neck. This results in the registration result being short hair, but the actual input image is long hair, which will lead to a degraded user experience.

[0082] This application divides the actual hair mask and the source hair mask after face alignment into multiple regions, for example, five regions: the entire face, above the eyebrows, to the left of the left eye, to the right of the right eye, and below the chin. The intersection-union ratio (IU) is calculated for each region, with each region having a pre-set corresponding weight. A weighted average is then calculated based on the IU and the corresponding region weights to obtain the IU.

[0083] For example, the actual hair mask M of the face image is obtained. input And the hair mask M from the resource library. temp Based on facial landmarks, the coordinates of the eyebrows, eyes, and chin can be obtained; therefore, M... input and M temp The model is divided into five parts: overall, above the eyebrows, to the left of the left eye, to the right of the right eye, and below the chin. The IoU is calculated for each of the five parts separately, and then weighted and averaged according to the weights of 1, 1.2, 1, 1, and 1.5 to obtain the final average IoU (mIoU). The image with the highest mIoU is the image with the closest registered hairstyle.

[0084] After registration, the face-shaping platform obtains the registered material. This material is then combined with other personalized materials and displayed on the client to create a personalized 3D cartoon model. The digital human image generated using AI face-shaping in this application is more personalized, with hairstyles more closely resembling the input facial image.

[0085] This application divides the hair mask into multiple regions for intersection-union (IoU) calculation, which can avoid false detections caused by IoU of the entire image and improve matching accuracy.

[0086] Based on the same technical concept, embodiments of this application also provide a hairstyle matching flowchart, such as... Figure 4 As shown.

[0087] Based on the same technical concept, embodiments of this application also provide a hairstyle matching device, such as... Figure 5 As shown.

[0088] Module 501 is used to obtain the actual hair mask of the face image using a face segmentation algorithm;

[0089] The extraction module 502 is used to extract multiple hair masks by rendering the face model files exported from the face modeling platform. Each face model file corresponds to a hairstyle.

[0090] Alignment module 503 is used to align the actual hair mask and the source hair mask for face alignment;

[0091] The comparison module 504 is used to compare the actual hair mask after face alignment with the source hair mask, and take the source hair mask with the highest overlap as the matching hairstyle of the face image.

[0092] Optionally, the extraction module 502 is used for:

[0093] For the face model files exported from the face-shaping platform, 3D modeling software is used for rendering to obtain a preset rendering result, which is a digital face model with hair.

[0094] After removing the hair mesh from the face model file, the rendering is performed to obtain the target rendering result, which is a digital face model without hair.

[0095] Align the preset rendering result and the target rendering result for the face;

[0096] Subtract the pixels of the preset rendering result from the pixels of the target rendering result after aligning the face;

[0097] If the difference in pixels after subtraction is greater than the preset tolerance value, then the pixel corresponding to the difference in pixels will be used as the hair mask of the material.

[0098] Optionally, the extraction module 502 is used for:

[0099] Aligning faces between the preset rendering result and the target rendering result includes:

[0100] Select either the preset rendering result or the target rendering result to perform keypoint detection and obtain the keypoint coordinates;

[0101] Align the key point coordinates with the preset coordinates in the preset template to determine the mapping matrix;

[0102] By applying a mapping matrix to the preset rendering result and the target rendering result, the preset rendering result and the target rendering result are aligned to the same position.

[0103] Optionally, the alignment module 503 is used for:

[0104] Identify at least three key points on the face contour and nose tip in both the actual hair mask and the source hair mask;

[0105] Based on the mapping relationship between the corresponding key points, the affine matrix is ​​obtained;

[0106] The affine matrix is ​​applied to the actual hair mask to align the face between the actual hair mask and the source hair mask.

[0107] Optionally, the alignment module 503 is used for:

[0108] When performing an affine transformation on the actual hair mask, the key point coordinates of the actual hair mask are shifted up or down according to a set offset, so that the hairline of the transformed actual hair mask and the hairline of the source hair mask can be aligned.

[0109] Optionally, the comparison module 504 is used for:

[0110] For the actual hair mask after face alignment and the source hair mask, calculate the intersection-union ratio (CIU), where CIU indicates the degree of overlap between the actual hair mask and the source hair mask;

[0111] The hair mask with the largest intersection and comparison is used as the matching hairstyle for the face image.

[0112] Optionally, the comparison module 504 is used for:

[0113] The actual hair mask after face alignment and the source hair mask are divided into multiple regions respectively;

[0114] Calculate the intersection-union ratio of regions within the same area;

[0115] The crossover-union ratio (CUNR) is obtained by calculating the weighted average of the regional crossover-union ratio and the corresponding regional weights.

[0116] Based on the same technical concept, embodiments of the present invention also provide an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0117] Memory 603 is used to store computer programs;

[0118] The processor 601 performs the above steps when executing the program stored in the memory 603.

[0119] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0120] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0121] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0122] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0123] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements the steps of any of the above methods.

[0124] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the methods described above.

[0125] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0126] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method of matching a hairstyle, characterized by, The method comprises: An actual hair mask of a face image is obtained by using a face segmentation algorithm; A plurality of material hair masks are extracted by performing model rendering on face model files exported by a face sculpting platform, wherein each face model file corresponds to a hairstyle; The actual hair mask and the material hair masks are aligned; The actual hair mask and the material hair masks after face alignment are compared, and the material hair mask with the highest coincidence degree is taken as the matching hairstyle of the face image; The method comprises: A preset rendering result of a digital face model with hair is obtained by rendering the face model files exported by the face sculpting platform using a three-dimensional modeling software; A target rendering result of a digital face model without hair is obtained by deleting the hair mesh in the face model files and then performing rendering; The preset rendering result and the target rendering result are aligned; Pixel points of the preset rendering result and pixel points of the target rendering result after face alignment are subtracted; If the pixel difference after subtraction is greater than a preset tolerance value, the pixel points corresponding to the pixel difference are taken as the material hair mask.

2. The method of claim 1, wherein, The method comprises: Any one of the preset rendering result or the target rendering result is selected for key point detection to obtain key point coordinates; The key point coordinates and preset coordinates in a preset template are aligned to determine a mapping matrix; The preset rendering result and the target rendering result are aligned to the same position by applying the mapping matrix to the preset rendering result and the target rendering result.

3. The method of claim 1, wherein, The method comprises: At least three key points on the face contour and the nose tip in the actual hair mask and the material hair mask are determined respectively; An affine matrix is obtained based on the mapping relationship between the corresponding key points; The affine matrix is applied to the actual hair mask to realize face alignment of the actual hair mask and the material hair mask.

4. The method of claim 3, wherein, The method comprises: When performing affine transformation on the actual hair mask, the key point coordinates of the actual hair mask are moved up or down by a set offset amount, so that the hairline of the transformed actual hair mask can be aligned with the hairline of the material hair mask.

5. The method of claim 2, wherein, The method comprises: The intersection-over-union of the actual hair mask and the material hair mask after face alignment is calculated, wherein the intersection-over-union indicates the coincidence degree between the actual hair mask and the material hair mask; The material hair mask with the maximum intersection-over-union is taken as the matching hairstyle of the face image.

6. The method of claim 5, wherein, The intersection-over-union ratio is calculated based on the actual hair mask and the material hair mask after face alignment, comprising: The actual hair mask and the material hair mask after face alignment are divided into multiple regions respectively; The region intersection-over-union ratio of the same region is calculated respectively; The weighted average value is obtained according to the region intersection-over-union ratio and the corresponding region weight, and the intersection-over-union ratio is obtained.

7. A hair style matching apparatus characterized by comprising: The device comprises: An obtaining module is configured to obtain an actual hair mask of a face image by using a face segmentation algorithm; An extracting module is configured to extract multiple material hair masks by performing model rendering on face model files exported by a face sculpting platform, wherein each face model file corresponds to a hairstyle; An aligning module is configured to perform face alignment on the actual hair mask and the material hair mask; A comparing module is configured to compare the actual hair mask and the material hair mask after face alignment, and take the material hair mask with the highest coincidence degree as the matching hairstyle of the face image; The extracting module is configured to: Render the face model files exported by the face sculpting platform by using a three-dimensional modeling software to obtain a preset rendering result, wherein the preset rendering result is a digital face model with hair; Render the face model files after deleting the hair mesh to obtain a target rendering result, wherein the target rendering result is a digital face model without hair; Perform face alignment on the preset rendering result and the target rendering result; Subtract the pixel points of the face alignment preset rendering result from the pixel points of the target rendering result; If the pixel difference after subtraction is greater than a preset tolerance value, the pixel points corresponding to the pixel difference are taken as the material hair mask.

8. An electronic device, comprising: The device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; The memory is configured to store a computer program; The processor is configured to execute the program stored on the memory to implement the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-6. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-6.

Images