Virtual hair generation method and device, computer readable medium and electronic device

Through image segmentation and feature extraction technology, the target model is matched from the hair model library based on hair style characteristics, and three-dimensional virtual hair is generated, which solves the problem of low realism of virtual hair in the existing technology, and improves the matching and generation of virtual hair efficiency.

CN113744286BActive Publication Date: 2024-12-03GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111075181.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-14
Publication Date
2024-12-03
Estimated Expiration
2041-09-14

AI Technical Summary

Technical Problem

The existing technology is difficult to generate three-dimensional virtual hair with strong sense of reality, and there is a lack of effective methods to build virtual hair models that match the actual hair profile.

Method used

By obtaining an image containing the portrait area, image segmentation is performed to obtain a hair outline image, and then feature extraction is performed. The target hair model is determined from the hair model feature library based on the extracted hair style features, and a three-dimensional virtual hair is generated.

Benefits of technology

It improves the realism and accuracy of three-dimensional virtual hair, realizes automatic conversion from two-dimensional hair images to three-dimensional hair models, and improves generation efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113744286B_ABST
    Figure CN113744286B_ABST
Patent Text Reader

Abstract

The present disclosure provides a method and device for generating virtual hair, a computer-readable medium and an electronic device, and relates to the field of virtual reality technology. The method comprises: obtaining a current image, wherein the current image comprises a portrait region; performing image segmentation on the portrait region to obtain a hair contour image; performing feature extraction on the hair contour image to obtain a hair style feature; determining a target hair model from a hair model feature library according to the hair style feature, so as to generate three-dimensional virtual hair based on the target hair model. The present disclosure can automatically generate three-dimensional virtual hair that conforms to the hairstyle features of the original portrait according to the hair image contained in the current image, thereby ensuring the authenticity of the three-dimensional virtual hair and achieving effects such as virtual hairstyle try-on and virtual avatar.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] With the continuous improvement of people's living standards, Virtual Reality (VR) technology, as a new field of science and technology, has an increasing demand in all walks of life for VR technology.

[0003] Since different hair styles can significantly reflect the characteristics of a person, constructing realistic virtual hair becomes the key to the virtual avatar construction scheme. However, currently, the relevant virtual avatar construction schemes do not mention how to construct realistic three-dimensional virtual hair. Summary of the invention

[0004] The purpose of the present disclosure is to provide a virtual hair generation method, a virtual hair generation device, a computer-readable medium and an electronic device, thereby at least to a certain extent solving the problem that related technical solutions cannot generate three-dimensional virtual hair with a strong sense of reality, and effectively improving the realism and accuracy of the generated three-dimensional virtual hair.

[0005] According to a first aspect of the present disclosure, a method for generating virtual hair is provided, comprising:

[0006] Get the current image, where the current image includes a portrait area:

[0007] Performing image segmentation on the portrait area to obtain a hair contour image;

[0008] Extracting features from the hair contour image to obtain hair style features;

[0009] A target hair model is determined from a hair model feature library according to the hair style feature, so as to generate three-dimensional virtual hair based on the target hair model.

[0010] According to a second aspect of the present disclosure, there is provided a virtual hair generating device, comprising:

[0011] The image acquisition module is used to acquire the current image, wherein the current image includes a portrait area:

[0012] An image segmentation module, used to perform image segmentation on the portrait area to obtain a hair contour image;

[0013] A hair style feature extraction module, used to extract features from the hair contour image to obtain hair style features;

[0014] The three-dimensional virtual hair generation module is used to determine a target hair model from a hair model feature library according to the hair style feature, so as to generate three-dimensional virtual hair based on the target hair model.

[0015] According to a third aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0016] According to a fourth aspect of the present disclosure, there is provided an electronic device, characterized in that it includes:

[0017] Processor; and

[0018] The memory is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0019] A virtual hair generation method provided by an embodiment of the present disclosure obtains a current image including a portrait area, performs image segmentation on the portrait area to obtain a hair contour image, then performs feature extraction on the hair contour image to obtain hair style features, and determines a target hair model from a hair model feature library according to the hair style features, so as to generate three-dimensional virtual hair based on the target hair model. On the one hand, image segmentation is performed on the portrait area to obtain a hair contour image, and then hair style features are extracted from the hair contour image, and then the target hair model is matched according to the hair style features, which can effectively ensure the matching of the generated three-dimensional virtual hair with the hair contour image and improve the realism of the three-dimensional virtual hair; on the other hand, the target hair model is determined from the hair model feature library according to the hair style features, and then the three-dimensional virtual hair is generated based on the target hair model, which can realize the automatic conversion from the two-dimensional hair image to the three-dimensional hair model and improve the generation efficiency of the three-dimensional hair model.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0022] Figure 1 A schematic diagram showing an exemplary system architecture to which embodiments of the present disclosure may be applied;

[0023] Figure 2 A schematic diagram showing an electronic device to which the embodiments of the present disclosure can be applied;

[0024] Figure 3A schematic diagram schematically illustrates a flow chart of a method for generating virtual hair in an exemplary embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of a process of performing image segmentation on a portrait area to obtain a hair contour image in an exemplary embodiment of the present disclosure is schematically shown;

[0026] Figure 5 A schematic diagram of a process of obtaining a hair contour image based on image semantic segmentation in an exemplary embodiment of the present disclosure is schematically shown;

[0027] Figure 6 A schematic diagram of a process of training a hair style extractor in an exemplary embodiment of the present disclosure is schematically shown;

[0028] Figure 7 A schematic diagram of a process of determining a target hair model from a hair model feature library in an exemplary embodiment of the present disclosure is schematically shown;

[0029] Figure 8 The following is a schematic diagram schematically showing the composition of a virtual hair generating device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0031] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0032] Figure 1 A schematic diagram of a system architecture of an exemplary application environment in which a method and apparatus for generating virtual hair according to an embodiment of the present disclosure can be applied is shown.

[0033] like Figure 1As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with image processing functions, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0034] The virtual hair generation method provided in the embodiment of the present disclosure is generally executed by the terminal devices 101, 102, and 103, and accordingly, the virtual hair generation device is generally set in the terminal devices 101, 102, and 103. However, it is easy for those skilled in the art to understand that the virtual hair generation method provided in the embodiment of the present disclosure can also be executed by the server 105, and accordingly, the virtual hair generation device can also be set in the server 105, which is not particularly limited in the present exemplary embodiment. For example, in an exemplary embodiment, the user can collect the current image through the image sensor for collecting images included in the terminal devices 101, 102, and 103, and then upload the current image to the server 105. After the server generates three-dimensional virtual hair through the virtual hair generation method provided in the embodiment of the present disclosure, the server renders the three-dimensional virtual hair into the display area of ​​the terminal devices 101, 102, and 103.

[0035] An exemplary embodiment of the present disclosure provides an electronic device for implementing a virtual hair generation method, which may be Figure 1 The terminal device 101, 102, 103 or the server 105 in the embodiment of the present invention comprises at least a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to execute the virtual hair generation method by executing the executable instructions.

[0036] Below Figure 2 Taking the mobile terminal 200 in FIG. 1 as an example, the structure of the electronic device is exemplarily described. It should be understood by those skilled in the art that, in addition to the components specifically used for mobile purposes, Figure 2The structure in the figure can also be applied to fixed type devices. In other embodiments, the mobile terminal 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between the components is only shown schematically and does not constitute a structural limitation of the mobile terminal 200. In other embodiments, the mobile terminal 200 may also adopt the same Figure 2 Different interface connection methods, or a combination of multiple interface connection methods.

[0037] like Figure 2 As shown, the mobile terminal 200 may specifically include: a processor 210, an internal memory 221, an external memory interface 222, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 271, a receiver 272, a microphone 273, an earphone interface 274, a sensor module 280, a display screen 290, a camera module 291, an indicator 292, a motor 293, a button 294, and a subscriber identification module (SIM) card interface 295, etc. The sensor module 280 may include a depth sensor 2801, a pressure sensor 2802, a gyroscope sensor 2803, etc.

[0038] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural network processor (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0039] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the mobile terminal 200 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0040] The processor 210 is provided with a memory. The memory can store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions, and notification instructions, and the execution is controlled by the processor 210.

[0041] The charging management module 240 is used to receive charging input from the charger. The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240 to power the processor 210, the internal memory 221, the display screen 290, the camera module 291 and the wireless communication module 260.

[0042] The wireless communication function of the mobile terminal 200 can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modulation and demodulation processor and baseband processor. Among them, antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals; mobile communication module 250 can provide solutions for wireless communication including 2G / 3G / 4G / 5G applied on mobile terminal 200; modulation and demodulation processor can include modulator and demodulator; wireless communication module 260 can provide solutions for wireless communication including wireless local area network (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT) and the like applied on mobile terminal 200. In some embodiments, antenna 1 of mobile terminal 200 is coupled with mobile communication module 250, and antenna 2 is coupled with wireless communication module 260, so that mobile terminal 200 can communicate with network and other devices through wireless communication technology.

[0043] The mobile terminal 200 implements the display function through a GPU, a display screen 290, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 290 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs that execute program instructions to generate or change display information.

[0044] The mobile terminal 200 can realize the shooting function through the ISP, the camera module 291, the video codec, the GPU, the display screen 290 and the application processor, etc. Among them, the ISP is used to process the data fed back by the camera module 291; the camera module 291 is used to capture static images or videos; the digital signal processor is used to process digital signals, and can process other digital signals in addition to digital image signals; the video codec is used to compress or decompress digital videos, and the mobile terminal 200 can also support one or more video codecs.

[0045] The external memory interface 222 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile terminal 200. The external memory card communicates with the processor 210 through the external memory interface 222 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0046] The internal memory 221 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 221 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the mobile terminal 200 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory (Universal Flash Storage, UFS), etc. The processor 210 executes various functional applications and data processing of the mobile terminal 200 by running instructions stored in the internal memory 221 and / or instructions stored in a memory provided in the processor.

[0047] The mobile terminal 200 can implement audio functions such as music playing and recording through the audio module 270 , the speaker 271 , the receiver 272 , the microphone 273 , the earphone interface 274 and the application processor.

[0048] The depth sensor 2801 is used to obtain the depth information of the scene. In some embodiments, the depth sensor can be arranged in the camera module 291.

[0049] The pressure sensor 2802 is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 2802 can be arranged on the display screen 290. There are many types of pressure sensors 2802, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc.

[0050] The gyro sensor 2803 can be used to determine the motion posture of the mobile terminal 200. In some embodiments, the angular velocity of the mobile terminal 200 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 2803. The gyro sensor 2803 can be used for shooting anti-shake, navigation, somatosensory game scenes, etc.

[0051] In addition, sensors with other functions may be set in the sensor module 280 according to actual needs, such as an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0052] The mobile terminal 200 may also include other devices that provide auxiliary functions. For example, the key 294 includes a power key, a volume key, etc., and the user can generate key signal input related to the user settings and function control of the mobile terminal 200 through key input. For example, the indicator 292, the motor 293, the SIM card interface 295, etc.

[0053] The following takes the execution of a terminal device as an example to specifically describe the virtual hair generating method and the virtual hair generating apparatus according to the exemplary embodiment of the present disclosure.

[0054] Figure 3 A schematic flow chart of a virtual hair generation method in this exemplary embodiment is shown, including the following steps S310 to S340:

[0055] In step S310, a current image is acquired, where the current image includes a portrait area.

[0056] In an exemplary embodiment, the current image refers to an image that includes a portrait area and is used to generate three-dimensional virtual hair corresponding to the portrait area. For example, the current image can be an image that is acquired in real time by an image sensor of a terminal device and includes a portrait area, or it can be a pre-stored image that includes a portrait area. Of course, it can also be a drawn image that includes portrait lines, etc. This exemplary embodiment does not specifically limit this.

[0057] The portrait area refers to an image area that contains human body features (at least a complete hair area). For example, the portrait area can be a face image contained in the current image, a half-body image contained in the current image, or a full-body image contained in the current image. This example embodiment does not specifically limit this.

[0058] In step S320, the portrait region is segmented to obtain a hair contour image.

[0059] In an exemplary embodiment, image segmentation refers to the process of dividing the current image into several specific regions with unique properties and proposing objects of interest, and the hair contour image refers to the image region containing the hair image texture obtained by performing image segmentation on the portrait region.

[0060] The portrait area can be segmented by a more mature image segmentation algorithm. For example, the image segmentation algorithm can be a DeepLabel algorithm, a Unet algorithm, or a Fully Convolutional Networks (FCN) algorithm. Of course, it can also be other types of image segmentation algorithms that at least include a feature encoding module Encoder and a feature decoding module Decoder. This example embodiment does not make any special limitations on this.

[0061] In step S330, feature extraction is performed on the hair contour image to obtain hair style features.

[0062] In an exemplary embodiment, feature extraction refers to a process of abstractly expressing key features in a hair contour image. For example, feature extraction can be used to determine the shape features, color features, texture features, etc. of the hair in the hair contour image, and to abstractly express the shape features, color features, texture features, etc. as feature vectors.

[0063] Hair style features refer to features that characterize key information used to describe the hair hairstyle style in the hair contour image. For example, hair style features may include hair shape features, hair color features, and hair texture features. Of course, hair style features may also include other features used to describe key information of the hair hairstyle style, but this example embodiment is not limited to this.

[0064] In step S340, a target hair model is determined from a hair model feature library according to the hair style feature, so as to generate three-dimensional virtual hair based on the target hair model.

[0065] In an exemplary embodiment, the hair model feature library refers to a database that is pre-collected and constructed to store hair style features corresponding to three-dimensional hair models of multiple different hair styles and three-dimensional hair models (rendering data) associated with the hair style features.

[0066] According to the similarity of hair style features and hair style features in a hair model feature library, a target hair model that is most similar to the hair contour image can be matched, and then rendering data of the target hair model can be obtained, and three-dimensional virtual hair corresponding to the hair contour image can be generated according to the rendering data.

[0067] Steps S310 to S340 are described in detail below.

[0068] In an exemplary embodiment, the Figure 4 The steps in extract the hair contour image from the portrait area, refer to Figure 4 As shown, it may specifically include:

[0069] Step S410, performing face frame detection on the portrait area to determine a face image frame;

[0070] Step S420, performing image segmentation on the face image in the face image frame to obtain a hair contour image.

[0071] Among them, face frame detection refers to a detection method for determining a face image from a portrait area in the form of a detection frame. For example, face frame detection can be performed on the portrait area by a multi-task convolutional neural network (MTCNN), or by a face detector Retinaface, or by a target detection algorithm YOLO. Of course, other detection algorithms that can determine a face image from a portrait area are also possible, and this example embodiment does not make any special limitation on this.

[0072] The face image frame refers to the minimum bounding box that determines the position of the face image in the portrait area. By determining the position of the face image in the portrait area through the face image frame, the size of the image required for image segmentation can be reduced, and the amount of calculation of the image segmentation algorithm can be reduced. At the same time, by selecting the face image through the face image frame for image segmentation, compared with directly using the image corresponding to the portrait area as the input of the image segmentation algorithm, the accuracy and precision of the segmented hair contour image can be effectively improved.

[0073] In an exemplary embodiment, the Figure 5 The steps in the image segmentation of face images are referenced Figure 5 As shown, it may specifically include:

[0074] Step S510, semantically classifying the pixels of the face image in the face image frame to determine target pixels belonging to the hair area;

[0075] Step S520, performing image segmentation on the face image according to the target pixel points to obtain a hair contour image.

[0076] Among them, the target pixel refers to the pixel in the face image that is identified as the hair area, and semantic classification refers to classifying each pixel in the image and determining the category of each point (such as background, person or car, etc.), thereby dividing the image area.

[0077] By semantically classifying the pixels in the face image and determining the target pixels belonging to the hair area, we can avoid the problem of image segmentation errors that are prone to occur in related image segmentation schemes based on image edge gradient methods, and effectively ensure the accuracy and precision of the hair contour image obtained by segmentation.

[0078] In an exemplary embodiment, a hair contour image may be feature extracted according to a pre-trained hair style extractor to obtain a hair style feature. The hair style extractor may include a deep learning network for image classification, for example, the deep learning network may be a residual neural network Resnet, Vggnet, Facenet, Arcface or other deep learning network framework; the loss function of the hair style extractor may adopt a triplet loss function TripletLoss, the triplet loss function may include an anchor sample (target sample) anchor, a positive sample positive sample and a negative sample negative sample, wherein the positive sample indicates a sample belonging to the same category as the anchor sample, and the negative sample indicates a sample belonging to a different category from the anchor sample, and the purpose of the triplet loss function is to make the embedding feature embedding of the anchor sample as close as possible to the positive sample and as far away from the negative sample as possible. Of course, the loss function of the hair style extractor in this exemplary embodiment may also be other types of loss functions, and there is no special limitation on this.

[0079] Since the intra-class differences between hair contour images may be small, and the advantage of the triplet loss function is that it can distinguish smaller differences, therefore, by extracting hair style features in hair contour images through a hair style extractor based on the triplet loss function, it is possible to effectively distinguish smaller differences in hair contour images and improve the accuracy and precision of hair style features.

[0080] Specifically, the hair contour image can be input into the hair style extractor to output an embedded feature vector corresponding to the hair contour image, wherein the embedded feature vector is high-dimensional feature data and is an abstract expression of the input hair contour image. The embedded feature vector is used as a hair style feature to facilitate subsequent calculations by the computer.

[0081] In an exemplary embodiment, the Figure 6 The steps in the implementation of the hair style extractor training and database construction, refer to Figure 6 As shown, it may specifically include:

[0082] Step S610, obtaining a hair model database and a sample hair outline image;

[0083] Step S620, training the constructed hair style extractor according to the hair model database and the sample hair outline map to obtain a trained hair style extractor; and

[0084] Step S630: construct a hair model feature library based on the hair model database.

[0085] The sample hair contour map refers to the pre-collected hair contour maps of different hair styles used to train the hair style extractor, and the hair model database refers to a database containing three-dimensional hair models corresponding to the sample hair contour maps, which is used to assist in training the hair style extractor.

[0086] For example, assuming that the loss function of the hair style extractor is a triplet loss function, then before the sample hair contour map is input into the constructed hair style extractor, the sample hair contour map can be preprocessed in combination with the hair model database to construct a triplet (target sample, positive sample, negative sample) corresponding to the sample hair contour map, and then the triplet (target sample, positive sample, negative sample) is used as the input data batch of the constructed hair style extractor, and the constructed hair style extractor is trained until the triplet loss function converges, thereby completing the training of the hair style extractor.

[0087] On the other hand, a hair model feature library can be constructed based on the hair model database. Specifically, hair style features corresponding to each three-dimensional hair model can be constructed according to the hair contour map associated with the hair model database. Then, a hair model feature library can be constructed according to the hair style features corresponding to each three-dimensional hair model for subsequent three-dimensional hair model matching.

[0088] In an exemplary embodiment, the Figure 7 The steps in the implementation of the target hair model from the hair model feature library, refer to Figure 7 As shown, it may specifically include:

[0089] Step S710, calculating the similarity between the hair style feature and the candidate hair features in the hair model feature library;

[0090] Step S720, sorting the candidate hair features according to the similarity;

[0091] Step S730: determine the three-dimensional hair model corresponding to the candidate hair feature with the maximum similarity as the target hair model.

[0092] The candidate hair features refer to the hair style features corresponding to each three-dimensional hair model in the hair model feature library, which are used to match the hair style features corresponding to the current image.

[0093] Similarity refers to data used to measure the similarity between the hair style features corresponding to the face image in the current image and the candidate hair features in the hair model feature library. For example, the similarity can be the cosine similarity between the hair style features and the candidate hair features, or the Euclidean distance between the hair style features and the candidate hair features. Of course, it can also be other types of data used to measure the similarity between the hair style features corresponding to the face image in the current image and the candidate hair features in the hair model feature library. This example embodiment does not specifically limit this.

[0094] The candidate hair features may be sorted according to the similarity. For example, the candidate hair features may be arranged in order according to the similarity, or may be arranged in reverse order according to the similarity. This exemplary embodiment does not specifically limit this.

[0095] The three-dimensional hair model corresponding to the candidate hair feature with the maximum similarity can be determined as the target hair model. At this time, it can be considered that the target hair model is closest to the hair style of the hair contour image. Therefore, it can be used as the three-dimensional hair model corresponding to the hair contour image, and then the three-dimensional virtual hair corresponding to the hair contour image is generated.

[0096] In an exemplary embodiment, the generated three-dimensional virtual hair can be used to realize the user's virtual avatar and add the virtual avatar to a virtual reality scene. For example, when the virtual avatar application is started, the current image of the user is acquired in real time through an image sensor, and then the current image is used to generate three-dimensional virtual hair based on the virtual hair generation method in this exemplary embodiment, and then a three-dimensional face model is generated according to key facial features (such as facial features, facial contour features, etc.) in the current image, and then a three-dimensional body model is generated according to the user's selection or the system's random method, and finally the virtual avatar corresponding to the user is constructed based on the generated three-dimensional virtual hair, three-dimensional face model and three-dimensional body model.

[0097] On the other hand, the generated three-dimensional virtual hair can also be used for virtual hairstyle try-on. For example, when the virtual hairstyle try-on application is started, the current image is acquired in real time by the image sensor, or the current image is selected and input by the user, and the current image is used to generate three-dimensional virtual hair based on the virtual hair generation method in this example embodiment. Finally, the three-dimensional virtual hair is rendered to the hair area in the current image, or to the image specified by the user, to realize the function of virtual hairstyle try-on.

[0098] Of course, the three-dimensional virtual hair generated by the virtual hair generation method in this example embodiment can also be used in other application scenarios. This is only a schematic example, and this example embodiment is not limited to this.

[0099] In summary, in this exemplary embodiment, a current image including a portrait region is acquired, the portrait region is segmented to obtain a hair contour image, and then a feature extraction is performed on the hair contour image to obtain a hair style feature, and a target hair model is determined from a hair model feature library according to the hair style feature, so as to generate three-dimensional virtual hair based on the target hair model. On the one hand, the portrait region is segmented to obtain a hair contour image, and then the hair style feature is extracted from the hair contour image, and then the target hair model is matched according to the hair style feature, which can effectively ensure the matching of the generated three-dimensional virtual hair with the hair contour image and improve the realism of the three-dimensional virtual hair; on the other hand, the target hair model is determined from the hair model feature library according to the hair style feature, and then the three-dimensional virtual hair is generated based on the target hair model, which can realize the automatic conversion from the two-dimensional hair image to the three-dimensional hair model and improve the generation efficiency of the three-dimensional hair model.

[0100] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0101] For further reference, Figure 8 As shown, in the embodiment of this example, a virtual hair generation device 800 is also provided, which may include an image acquisition module 810, an image segmentation module 820, a hair style feature extraction module 830 and a three-dimensional virtual hair generation module 840. Among them:

[0102] The image acquisition module 810 is used to acquire a current image, where the current image includes a portrait area:

[0103] The image segmentation module 820 is used to perform image segmentation on the portrait area to obtain a hair contour image;

[0104] The hair style feature extraction module 830 is used to extract features from the hair contour image to obtain hair style features;

[0105] The three-dimensional virtual hair generating module 840 is used to determine a target hair model from a hair model feature library according to the hair style feature, so as to generate three-dimensional virtual hair based on the target hair model.

[0106] In an exemplary embodiment, the image segmentation module 820 may include:

[0107] A face image frame detection unit, used to perform face frame detection on the portrait area to determine the face image frame;

[0108] The face image segmentation unit is used to segment the face image in the face image frame to obtain a hair contour image.

[0109] In an exemplary embodiment, the face image segmentation unit may also be used to:

[0110] Performing semantic classification on the pixels of the face image in the face image frame to determine target pixels belonging to the hair area;

[0111] The face image is segmented according to the target pixel points to obtain a hair contour image.

[0112] In an exemplary embodiment, the hair style feature extraction module 830 may be used to:

[0113] Feature extraction is performed on the hair contour image according to a pre-trained hair style extractor to obtain hair style features. The hair style extractor includes a deep learning network for image classification and a triplet loss function.

[0114] In an exemplary embodiment, the hair style feature extraction module 830 may also be used to:

[0115] Inputting the hair contour image into the hair style extractor, and outputting an embedded feature vector corresponding to the hair contour image;

[0116] The embedded feature vector is used as the hair style feature.

[0117] In an exemplary embodiment, the virtual hair generating apparatus 800 may include a hair style extractor training module, and the hair style extractor training module may be used to:

[0118] Obtain a hair model database and a sample hair contour map;

[0119] Training the constructed hair style extractor according to the hair model database and the sample hair outline map to obtain a trained hair style extractor; and

[0120] A hair model feature library is constructed based on the hair model database.

[0121] In an exemplary embodiment, the 3D virtual hair generation module 840 may be used to:

[0122] Calculating the similarity between the hair style feature and the candidate hair features in the hair model feature library;

[0123] sorting the candidate hair features according to the similarities;

[0124] The three-dimensional hair model corresponding to the candidate hair feature with the maximum similarity is determined as the target hair model.

[0125] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0126] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0127] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification, for example, it can execute Figures 3 to 7 Any one or more steps in .

[0128] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0129] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0130] In addition, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0131] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0132] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating virtual hair, characterized in that: include: Acquire a current image, where the current image includes a portrait area; Performing image segmentation on the portrait area to obtain a hair contour image; Extracting features from the hair contour image to obtain hair style features, including: extracting features from the hair contour image according to a pre-trained hair style extractor to obtain hair style features, wherein the hair style extractor includes a deep learning network for image classification and a triplet loss function; A target hair model is determined from a hair model feature library according to the hair style feature, so as to generate three-dimensional virtual hair based on the target hair model.

2. The method according to claim 1, characterized in that The step of performing image segmentation on the portrait area to obtain a hair contour image includes: Performing face frame detection on the portrait area to determine a face image frame; Perform image segmentation on the face image in the face image frame to obtain a hair contour image.

3. The method according to claim 2, characterized in that The performing image segmentation on the face image in the face image frame to obtain the hair contour image comprises: Performing semantic classification on the pixels of the face image in the face image frame to determine target pixels belonging to the hair area; The face image is segmented according to the target pixel points to obtain a hair contour image.

4. The method according to claim 1, characterized in that The step of extracting features from the hair contour image according to a pre-trained hair style extractor to obtain hair style features includes: Inputting the hair contour image into the hair style extractor, and outputting an embedded feature vector corresponding to the hair contour image; The embedded feature vector is used as the hair style feature.

5. The method according to claim 1, characterized in that The method further comprises: Obtain a hair model database and a sample hair contour map; Training the constructed hair style extractor according to the hair model database and the sample hair outline map to obtain a trained hair style extractor; and A hair model feature library is constructed based on the hair model database.

6. The method according to claim 1 or 5, characterized in that: The step of determining a target hair model from a hair model feature library according to the hair style feature comprises: Calculating the similarity between the hair style feature and the candidate hair features in the hair model feature library; sorting the candidate hair features according to the similarities; The three-dimensional hair model corresponding to the candidate hair feature with the maximum similarity is determined as the target hair model.

7. A virtual hair generating device, characterized in that: include: The image acquisition module is used to acquire the current image, wherein the current image includes a portrait area: An image segmentation module, used to perform image segmentation on the portrait area to obtain a hair contour image; A hair style feature extraction module, used to extract features from the hair contour image to obtain hair style features, including: extracting features from the hair contour image according to a pre-trained hair style extractor to obtain hair style features, wherein the hair style extractor includes a deep learning network for image classification and a triplet loss function; The three-dimensional virtual hair generation module is used to determine a target hair model from a hair model feature library according to the hair style feature, so as to generate three-dimensional virtual hair based on the target hair model.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 6 by executing the executable instructions.