Method and device for generating three-dimensional eyebrow shape, electronic device and storage medium

By annotating a small number of eyebrow reference lines in a two-dimensional eyebrow image and using a depth estimation model to generate a three-dimensional eyebrow shape, the problem of long time consumption and low efficiency in the generation of three-dimensional eyebrow shapes in the existing technology is solved, and automated and efficient three-dimensional eyebrow shape generation is achieved.

CN117422831BActive Publication Date: 2024-11-08BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311435164.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-31
Publication Date
2024-11-08
Estimated Expiration
2043-10-31

AI Technical Summary

Technical Problem

Current technologies for generating 3D eyebrow shapes are time-consuming, inefficient, and costly, relying mainly on manual design and making it difficult to automate the process.

Method used

By annotating a small number of eyebrow reference lines in a two-dimensional eyebrow image, and using a depth estimation model to obtain the depth values ​​of the root point and outer endpoint of the eyebrow reference lines, a corresponding three-dimensional eyebrow shape is generated, thus achieving automated generation.

Benefits of technology

It greatly improves the efficiency of generating 3D eyebrow shapes, reduces the amount of computation, and improves the accuracy and detail of the generated shapes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117422831B_ABST
    Figure CN117422831B_ABST
Patent Text Reader

Abstract

The present disclosure provides a method and device for generating a three-dimensional eyebrow shape, an electronic device and a storage medium, which relate to the field of artificial intelligence technology, specifically computer vision, augmented reality, virtual reality, deep learning and other technical fields, and can be applied to scenes such as artificial intelligence content generation and digital humans. The method for generating a three-dimensional eyebrow shape includes: obtaining an eyebrow image, the eyebrow image is annotated with multiple eyebrow reference lines, each eyebrow reference line includes a root point and an outer end point of the eyebrow; performing depth estimation on the eyebrow image to obtain the depth value of the root point and the depth value of the outer end point of each of the multiple eyebrow reference lines; based on the depth value of the root point and the depth value of the outer end point of each of the multiple eyebrow reference lines, generating a three-dimensional eyebrow shape corresponding to the eyebrow image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to technical fields such as computer vision, augmented reality, virtual reality, and deep learning, and can be applied to scenarios such as artificial intelligence content generation and digital humans. Background Art

[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] Three-dimensional virtual images (i.e., digital humans) have wide application value in user scenarios such as social networking, live broadcasting, and games. Using artificial intelligence technology, customized three-dimensional virtual images can be generated based on facial images to effectively meet the personalized needs of users.

[0004] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0005] The present disclosure provides a method and device for generating a three-dimensional eyebrow shape, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of the present disclosure, a method for generating a three-dimensional eyebrow shape is provided, comprising: acquiring an eyebrow image, wherein the eyebrow image is annotated with a plurality of eyebrow reference lines, each of the plurality of eyebrow reference lines comprising a root point and an outer endpoint of the eyebrow; performing depth estimation on the eyebrow image to obtain depth values ​​of the root points and the outer endpoints of each of the plurality of eyebrow reference lines; and generating a three-dimensional eyebrow shape corresponding to the eyebrow image based on the depth values ​​of the root points and the outer endpoints of each of the plurality of eyebrow reference lines.

[0007] According to one aspect of the present disclosure, a device for generating a three-dimensional eyebrow shape is provided, comprising: an acquisition module, configured to acquire an eyebrow image, wherein the eyebrow image is annotated with a plurality of eyebrow reference lines, each of the plurality of eyebrow reference lines comprising a root point and an outer endpoint of the eyebrow; an estimation module, configured to perform depth estimation on the eyebrow image to obtain depth values ​​of the root points and the outer endpoints of the plurality of eyebrow reference lines; and a generation module, configured to generate a three-dimensional eyebrow shape corresponding to the eyebrow image based on the depth values ​​of the root points and the outer endpoints of the plurality of eyebrow reference lines.

[0008] According to one aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above method.

[0009] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above method.

[0010] According to one aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program can implement the above method when executed by a processor.

[0011] According to one or more embodiments of the present disclosure, a three-dimensional eyebrow shape can be automatically generated based on a two-dimensional eyebrow shape image, thereby improving the generation efficiency of the three-dimensional eyebrow shape.

[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0014] Figure 1 A schematic diagram showing an exemplary system in which various methods described herein may be implemented according to an embodiment of the present disclosure;

[0015] Figure 2 A flow chart of a method for generating a three-dimensional eyebrow shape according to an embodiment of the present disclosure is shown;

[0016] Figure 3 A schematic diagram of an eyebrow shape image according to an embodiment of the present disclosure is shown;

[0017] Figure 4 A schematic diagram showing a process of generating a three-dimensional eyebrow shape according to an embodiment of the present disclosure is shown;

[0018] Figure 5 A structural block diagram of a device for generating a three-dimensional eyebrow shape according to an embodiment of the present disclosure is shown; and

[0019] Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0020] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0021] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0022] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations. "Multiple" refers to two or more.

[0023] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0024] The 3D eyebrow shape is an important part of the 3D virtual image, which can affect the appearance and style of the 3D virtual image. In order to improve the realism and delicacy of the 3D virtual image, it is necessary to create a 3D eyebrow shape that can delicately show the direction and texture of a single eyebrow. In the related art, the 3D eyebrow shape is usually created manually by the designer, which is time-consuming, inefficient and costly.

[0025] In view of the above problems, an embodiment of the present disclosure provides a method for generating a three-dimensional eyebrow shape. According to an embodiment of the present disclosure, a three-dimensional eyebrow shape corresponding to the eyebrow shape image is generated by estimating the depth values ​​of the root point and the outer end point of the eyebrow reference line in the two-dimensional eyebrow shape image. Therefore, only a small number of eyebrow reference lines need to be marked in the two-dimensional eyebrow shape image to automatically generate the three-dimensional eyebrow shape corresponding to the two-dimensional eyebrow shape image, thereby realizing the automatic generation of the three-dimensional eyebrow shape and greatly improving the generation efficiency of the three-dimensional eyebrow shape.

[0026] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0028] In an embodiment of the present disclosure, the client device server 120 may run one or more services or software applications that enable the method for generating a three-dimensional eyebrow shape to be executed.

[0029] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0030] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0031] The user may use client devices 101, 102, 103, 104, 105 and / or 106 to send the initial instruction. The client device may provide an interface that enables the user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0032] Client devices 101, 102, 103, 104, 105 and / or 106 may include various categories of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various categories and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. The client device is capable of executing a variety of different applications, such as various Internet-related applications, communication applications (eg, email applications), short message service (SMS) applications, and may use various communication protocols.

[0033] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, Wi-Fi), and / or any combination of these and / or other networks.

[0034] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0035] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0036] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0037] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0038] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The databases 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different categories. In some embodiments, the databases used by the server 120 may be, for example, relational databases. One or more of these databases may store, update, and retrieve data to and from the databases in response to commands.

[0039] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different categories of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0040] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0041] According to some embodiments, the client devices 101 - 106 may be configured to execute the method for generating a 3D eyebrow shape according to the embodiment of the present disclosure, and generate a 3D eyebrow shape based on a 2D eyebrow image annotated with a small number of eyebrow reference lines.

[0042] According to some other embodiments, the client devices 101-106 may send a 2D eyebrow image with a small number of eyebrow reference lines marked to the server 120. The server 120 is configured to execute the 3D eyebrow generation method of the embodiment of the present disclosure to generate a 3D eyebrow corresponding to the 2D eyebrow image.

[0043] Figure 2 FIG. 2 is a flow chart of a method 200 for generating a three-dimensional eyebrow shape according to an embodiment of the present disclosure. The execution subject of the method 200 may be a client device (eg Figure 1 ), or a server (e.g., a client device 101-106 shown in FIG. Figure 1 , server 120 shown in FIG.

[0044] like Figure 2 As shown, method 200 includes steps S210 - S230 .

[0045] In step S210, an eyebrow image is obtained. The eyebrow image is annotated with multiple eyebrow reference lines, each of which includes a root point and an outer end point of the eyebrow.

[0046] In step S220, depth estimation is performed on the eyebrow image to obtain depth values ​​of root points and outer end points of each of the plurality of eyebrow reference lines.

[0047] In step S230, a three-dimensional eyebrow shape corresponding to the eyebrow shape image is generated based on the depth values ​​of the root points and the depth values ​​of the outer end points of the plurality of eyebrow reference lines.

[0048] According to the embodiments of the present disclosure, by estimating the depth values ​​of the root point and the outer end point of the eyebrow reference line in the two-dimensional eyebrow image, a three-dimensional eyebrow shape corresponding to the eyebrow image is generated. Therefore, only a small number of eyebrow reference lines need to be marked in the two-dimensional eyebrow image to automatically generate the three-dimensional eyebrow shape corresponding to the two-dimensional eyebrow image, thereby realizing the automatic generation of the three-dimensional eyebrow shape and greatly improving the generation efficiency of the three-dimensional eyebrow shape.

[0049] In an embodiment of the present disclosure, the eyebrow shape image is a two-dimensional image including the eyebrow shape in a front view. The eyebrow shape includes a plurality of eyebrows, and is a shape formed by the plurality of eyebrows.

[0050] According to some embodiments, the eyebrow shape image may include and only include the eyebrow shape. For example, the eyebrow shape image may include the eyebrow shape specially designed by the designer, but does not include other face regions. According to other embodiments, the eyebrow shape image may include other face regions in addition to the eyebrow shape, such as eyes, nose, hair, etc. For example, the eyebrow shape image may be a face image (a face photo or a cartoon face).

[0051] In an embodiment of the present disclosure, an eyebrow image is annotated with a plurality of eyebrow reference lines. The eyebrow reference line is used to describe a single eyebrow in the eyebrow image, and is a curve from the root point to the outer end point of a single eyebrow, wherein the root point is the contact point between the eyebrow and the surface of the face. Each eyebrow reference line can be stored as the pixel coordinates of the root point, the pixel coordinates of the outer end point, and the direction (direction angle) of the curve. The eyebrow reference line can be annotated manually, or the curve in the eyebrow area of ​​the eyebrow image can be automatically identified by an image processing tool, thereby achieving automatic annotation.

[0052] The number of eyebrow reference lines is much less than the number of single eyebrows included in the eyebrow image, so only a small number of eyebrow reference lines need to be marked in the two-dimensional eyebrow image to automatically generate the three-dimensional eyebrow shape corresponding to the two-dimensional eyebrow image, which can reduce the amount of calculation and improve the calculation efficiency. The specific number of eyebrow reference lines can be set as needed, for example, set to 5, 10, 20, etc. The present disclosure does not limit the number of eyebrow reference lines.

[0053] According to some embodiments, in order to ensure the generation effect of a three-dimensional eyebrow shape, the multiple eyebrow reference lines include multiple eyebrow reference line groups corresponding to different directions. Each eyebrow reference line group includes at least two eyebrow reference lines, and the at least two eyebrow reference lines have the same direction. According to some embodiments, the angle of the vector pointing from the root point to the outer endpoint can be recorded as the direction angle of the eyebrow reference line. Further, two eyebrow reference lines whose angle of direction angle is less than a threshold (e.g., 5°, 8°, 10°, etc.) are determined to have the same direction. It should be noted that the same eyebrow reference line may be divided into different eyebrow reference line groups. By marking multiple groups of eyebrow reference lines with different directions, the shape and position information of different eyebrows in the two-dimensional eyebrow shape can be fully expressed, thereby improving the accuracy and delicacy of the generated three-dimensional eyebrow shape.

[0054] Figure 3 FIG. 3 shows a schematic diagram of an eyebrow image 300 according to an embodiment of the present disclosure. Figure 3 As shown, the eyebrow image 300 is annotated with multiple eyebrow reference lines 310-380. Each eyebrow reference line includes a root point Ai and an outer end point Bi (i=1, 2, 3, ..., 8). Eyebrow reference lines 310-380 are divided into three eyebrow reference line groups according to their directions. Among them, eyebrow reference lines 310, 320, 340, and 360 have the same direction (the angle between the two direction angles is less than the threshold), and belong to eyebrow reference line group G1. Eyebrow reference lines 330 and 340 have the same direction (the angle between the two direction angles is less than the threshold), and belong to eyebrow reference line group G2. Eyebrow reference lines 350, 370, and 380 have the same direction (the angle between the two direction angles is less than the threshold), and belong to eyebrow reference line group G3.

[0055] In step S220, depth estimation is performed on the eyebrow image obtained in step S210 to obtain the depth value of the root point and the depth value of the outer end point of each eyebrow reference line.

[0056] According to some embodiments, step S220 may include steps S221 and S222.

[0057] In step S221, the eyebrow image is input into a trained depth estimation model to obtain a depth map corresponding to the eyebrow image. The depth map indicates the depth value of each pixel in the eyebrow image.

[0058] In step S222, based on the depth map, the depth value of the root point and the depth value of the outer end point are determined.

[0059] According to the above embodiment, the depth value is estimated using the trained depth estimation model without acquiring additional information, thereby improving the efficiency of depth estimation while ensuring accuracy.

[0060] The depth map has the same size as the eyebrow image, and the pixels in the depth map correspond one-to-one to the pixels in the eyebrow image. Let the eyebrow image be I and the depth map be D. Then the pixel value of the pixel D(x, y) with coordinates (x, y) in the depth map is the depth value of the pixel I(x, y) in the eyebrow image I. The depth value indicates the distance from the pixel to the image acquisition device.

[0061] The depth estimation model can be a neural network model that takes an image as input and outputs a depth map corresponding to the image. The depth estimation model can be trained based on sample images labeled with depth map labels. The sample images and their depth map labels can be obtained, for example, by taking a depth camera (also known as a 3D camera).

[0062] It can be understood that the root point and the outer end point of the eyebrow reference line correspond to a pixel in the eyebrow image respectively. Accordingly, according to some embodiments, in step S222, the depth value of the root point and the depth value of the outer end point can be directly obtained from the depth map. This can improve the efficiency of obtaining the depth value.

[0063] According to some embodiments, step S222 may include steps S2221 and S2222.

[0064] In step S2221, the initial depth value of the root point and the initial depth value of the outer endpoint are obtained from the depth map, that is, the depth value in the depth map is used as the initial depth value of the root point and the outer endpoint.

[0065] In step S2222, the initial depth value of the root point and the initial depth value of the outer endpoint are standardized based on the heights of the root point and the outer endpoint relative to the face surface to obtain the depth value of the root point and the depth value of the outer endpoint.

[0066] It can be understood that the root point of the eyebrow is located on the surface of the face, that is, the height of the root point relative to the face surface is 0. The outer end point of the eyebrow is located above the face, and its height relative to the face surface is greater than 0. The depth values ​​of the root point and the outer end point in the depth map are used as initial depth values, and the initial depth values ​​of the root point and the outer end point are standardized based on the height of the root point and the outer end point relative to the face surface, that is, the standardized depth value is the depth value relative to the face surface. The three-dimensional virtual image is usually stored in the form of a three-dimensional face mesh model, that is, the face surface is represented as multiple meshes, and the three-dimensional coordinates of the vertices of each mesh and the connection relationship between the vertices are stored. According to the above embodiment, the standardized depth value of the root point and the outer end point is the depth value relative to the face surface. This makes it easy to combine the generated three-dimensional eyebrow shape with the existing three-dimensional face mesh model to obtain a three-dimensional virtual image.

[0067] It is understandable that different eyebrow shapes are located in roughly the same position on the face (all located a certain distance above the eyes), occupy a small area of ​​the face, and have similar overall shapes (all are horizontal long strips). The differences between different eyebrow shapes are mostly reflected in subtle details, such as the changes in the curvature of the external contour, the direction and distribution of a single eyebrow, etc.

[0068] Since the position of the eyebrow shape on the face is relatively fixed and the area it occupies is relatively small, the front face surface covered by the eyebrow shape area can be approximated as a vertical plane, that is, the depth value of the face surface is the same everywhere. Accordingly, the depth of the eyebrow point in the eyebrow shape area can be converted into the height of the eyebrow point relative to the face surface. Therefore, the initial depth values ​​of the root point and the outer end point are standardized based on their height relative to the face surface. The standardized depth value can not only facilitate the combination of the generated three-dimensional eyebrow shape with the three-dimensional face mesh model, but also ensure the accuracy of the depth value, thereby ensuring the authenticity and delicacy of the three-dimensional eyebrow shape.

[0069] As for hair, beard and other hair, due to the large differences in the shapes of different hairstyles and beards, the positions and areas on the face are not fixed, and the face surface covered by hair, beard and other hair is usually a curved surface. If the face surface covered by hair, beard and other hair is approximately a plane, and the depth values ​​of the hair and beard are obtained by a method similar to the above embodiment, the errors of these depth values ​​are large, and it is difficult to meet the accuracy requirements of three-dimensional reconstruction, resulting in serious distortion of the generated three-dimensional hair.

[0070] According to some embodiments, in step S2222, the difference d between the initial depth value r of the root point and the initial depth value o of the outer endpoint can be calculated. It can be understood that since the root point of the eyebrow is farther from the image acquisition device than the outer endpoint, the initial depth value of the root point is greater than the initial depth value of the outer endpoint in the depth map. Subsequently, the standardized depth value r' of the root point is set to the preset depth value s of the face surface, and the standardized depth value o' of the outer endpoint is set to the sum of the above-mentioned preset depth value s and the above-mentioned difference d. The preset depth value of the face surface is usually 0 (i.e., s=0), and accordingly, the standardized depth value r' of the root point can be set to 0, and the standardized depth value o' of the outer endpoint can be set to the above-mentioned difference d.

[0071] According to some embodiments, step S230 may include steps S231 - S233 .

[0072] In step S231, for any one of the plurality of eyebrow reference lines, a three-dimensional eyebrow reference line is generated based on the depth value of the root point and the depth value of the outer end point of the eyebrow reference line.

[0073] In step S232, the eyebrow region is extracted from the eyebrow image.

[0074] In step S233, based on a plurality of three-dimensional eyebrow reference lines respectively corresponding to the plurality of eyebrow reference lines, three-dimensional eyebrow interpolation is performed in a three-dimensional space corresponding to the eyebrow shape region to generate a three-dimensional eyebrow shape.

[0075] According to the above embodiment, three-dimensional eyebrow shapes are automatically generated through eyebrow point interpolation and eyebrow interpolation, thereby improving the efficiency of generating three-dimensional eyebrow shapes.

[0076] The eyebrow reference line marked in the eyebrow image is a two-dimensional eyebrow reference line, indicating the two-dimensional image information of the eyebrow. The depth values ​​of the root point and the outer end point indicate the three-dimensional space information of the eyebrow. In step S231, the two-dimensional image information of the eyebrow reference line is combined with the three-dimensional space information to obtain the three-dimensional eyebrow reference line corresponding to the eyebrow reference line. It can be understood that for each eyebrow reference line, a corresponding three-dimensional eyebrow reference line can be obtained.

[0077] According to some embodiments, step S231 may include steps S2311 and S2312.

[0078] In step S2311, based on the depth value of the root point and the depth value of the outer end point of the eyebrow reference line, the depth value of each of the multiple middle points on the eyebrow reference line is obtained by interpolation.

[0079] In step S2312, a three-dimensional eyebrow reference line is generated based on the depth values ​​of each point on the eyebrow reference line.

[0080] In the above step S2311, the depth value of the root point and the depth value of the outer endpoint can be the depth value after normalization based on the face surface (through the above step S2222). The eyebrow reference line is a curve, and multiple middle points on the curve can be obtained by sampling on the curve. Based on the depth value of the root point and the depth value of the outer endpoint, the curve is fitted by interpolation to obtain the depth value of each middle point.

[0081] In the above step S2312, each point on the eyebrow reference line includes a root point, a middle point and an outer end point. It can be understood that each point on the eyebrow reference line includes a two-dimensional pixel coordinate (x, y) and a depth value z relative to the face surface. Each point is represented as three-dimensional information (x, y, z) composed of a two-dimensional pixel coordinate (x, y) and a depth value z. The three-dimensional information of each point is combined to form a three-dimensional eyebrow reference line.

[0082] According to the above embodiment, the depth value is interpolated first, and then the two-dimensional eyebrow reference line is converted into a three-dimensional eyebrow reference line based on the depth value, thereby facilitating the fusion of the three-dimensional eyebrow shape with the existing three-dimensional face mesh model.

[0083] According to some embodiments, the eyebrow image may be a real frontal face image acquired by an image acquisition device. Accordingly, step S231 may include steps S2313 and S2314.

[0084] In step S2313, based on the depth value of the root point of the eyebrow reference line, the depth value of the outer endpoint and the intrinsic parameter matrix of the image acquisition device, the three-dimensional coordinates of the root point and the three-dimensional coordinates of the outer endpoint are determined.

[0085] In step S2314, based on the three-dimensional coordinates of the root point and the three-dimensional coordinates of the outer end point, the three-dimensional coordinates of each of the plurality of middle points on the eyebrow reference line are obtained by interpolation to obtain a three-dimensional eyebrow reference line.

[0086] According to the above embodiment, the three-dimensional coordinates of the root point and the outer end point are first determined, and then the three-dimensional coordinates are interpolated to obtain a three-dimensional eyebrow reference line.

[0087] In the above step S2313, the depth value of the root point and the depth value of the outer endpoint can be the depth value directly obtained from the depth map corresponding to the eyebrow image. The intrinsic parameter matrix of the image acquisition device is used to describe the coordinate transformation relationship from the camera coordinate system to the pixel coordinate system. The depth value of the root point is equivalent to the z coordinate value of the root point in the camera coordinate system, and the depth value of the outer endpoint is equivalent to the z coordinate value of the outer endpoint in the camera coordinate system. Based on the intrinsic parameter matrix, the pixel coordinates (u, v) of the root point in the image and the z coordinate value of the root point in the camera coordinate system, the x coordinate value and y coordinate value of the root point in the camera coordinate system can be solved, thereby obtaining the three-dimensional coordinates (x, y, z) of the root point in the camera coordinate system. Similarly, the three-dimensional coordinates of the outer endpoint in the camera coordinate system can also be obtained.

[0088] According to some embodiments, the three-dimensional coordinates of the root point and the external endpoint in the camera coordinate system can be further converted into three-dimensional coordinates in the world coordinate system based on the external parameter matrix of the image acquisition device. The external parameter matrix is ​​used to describe the coordinate conversion relationship from the world coordinate system to the camera coordinate system.

[0089] According to some embodiments, in the above step S2314, a plurality of intermediate points may be obtained by sampling the eyebrow reference line. Based on the three-dimensional coordinates of the root point and the outer end point, interpolation is performed in the three-dimensional space to obtain the three-dimensional coordinates of each intermediate point. The three-dimensional coordinates corresponding to each point on the eyebrow reference line are combined to form a three-dimensional eyebrow reference line.

[0090] According to some embodiments, in step S232, the trained semantic segmentation model can be used to segment the eyebrow region from the eyebrow image, that is, to obtain a mask of the eyebrow region. The eyebrow image is input into the semantic segmentation model, and the semantic segmentation model outputs the category of each pixel in the eyebrow image. The category of the pixel can be, for example, eyebrows, eyes, nose, face surface, etc. A plurality of pixels of the category "eyebrow" constitute the eyebrow region.

[0091] The semantic segmentation model can be a neural network model that takes an image as input and outputs the category of each pixel in the image. The semantic segmentation model can be trained based on sample images annotated with category labels.

[0092] According to some embodiments, the semantic segmentation model may be a model capable of receiving prompt information, such as a SAM (Segment Anything Model) model. Accordingly, the eyebrow image and the segmentation prompt information (prompt) may be input into the trained semantic segmentation model together to obtain the eyebrow region output by the semantic segmentation model. The segmentation prompt information includes at least one foreground pixel belonging to the eyebrow region and at least one background pixel that does not belong to the eyebrow region. For example, three pixels (foreground pixels) within the eyebrow region and three pixels (background pixels) within the region outside the eyebrow region (e.g., the region outside the eyes, cheeks, and face) may be arbitrarily specified as segmentation prompt information.

[0093] According to the above embodiment, the segmentation hint information can provide knowledge guidance for the semantic segmentation model. The semantic segmentation model extracts the eyebrow shape region in combination with the segmentation hint information, which can improve the accuracy of the eyebrow shape region.

[0094] In step S233, based on the multiple three-dimensional eyebrow reference lines corresponding to the multiple eyebrow reference lines, three-dimensional eyebrow interpolation is performed in the three-dimensional space corresponding to the eyebrow shape area to increase the density of the three-dimensional eyebrows and generate a three-dimensional eyebrow shape.

[0095] According to some embodiments, in step S233, 3D eyebrow interpolation may be performed between adjacent 3D eyebrow reference lines having the same direction to improve the density and authenticity of the 3D eyebrows. Multiple 3D eyebrow reference lines and multiple 3D eyebrows obtained by interpolation are combined to form a 3D eyebrow shape.

[0096] Figure 4 FIG. 4 is a schematic diagram showing a process 400 for generating a three-dimensional eyebrow shape according to an embodiment of the present disclosure. Figure 4 As shown, process 400 includes steps S410-S460.

[0097] In step S410, the curves in the eyebrow region of the 2D face image (eyebrow image) are automatically identified by an image processing tool, thereby automatically marking a plurality of eyebrow reference lines, each of which includes an eyebrow root point and an outer end point.

[0098] In step S420, depth information of the 2D face image is estimated to obtain a depth map of the 2D face image.

[0099] In step S430, semantic segmentation is performed on the 2D face image to extract a mask of the eyebrow region.

[0100] In step S440, based on the depth map obtained in step S420, the depth values ​​of the eyebrow root point and the outer end point of each eyebrow reference line are obtained, and then the three-dimensional coordinates of the root point and the outer end point are obtained. Based on the three-dimensional coordinates of the root point and the outer end point, the 3D direction of the eyebrow reference line can be obtained.

[0101] In step S450, a 3D eyebrow reference line is generated based on the three-dimensional coordinates of the root point and the outer end point of the eyebrow reference line.

[0102] In step S460, 3D eyebrow interpolation is performed between two 3D eyebrow reference lines with the same 3D direction to increase the density of the eyebrows so that the generated 3D eyebrows fill the eyebrow shape area and generate a 3D eyebrow shape.

[0103] According to an embodiment of the present disclosure, a device for generating a three-dimensional eyebrow shape is also provided. Figure 5 FIG. 5 shows a structural block diagram of a device 500 for generating a three-dimensional eyebrow shape according to an embodiment of the present disclosure. Figure 5 As shown, the apparatus 500 includes an acquisition module 510 , an estimation module 520 and a generation module 530 .

[0104] The acquisition module 510 is configured to acquire an eyebrow image, wherein the eyebrow image is annotated with a plurality of eyebrow reference lines, and each of the plurality of eyebrow reference lines includes a root point and an outer end point of the eyebrow.

[0105] The estimation module 520 is configured to perform depth estimation on the eyebrow shape image to obtain depth values ​​of root points and outer end points of each of the plurality of eyebrow reference lines.

[0106] The generating module 530 is configured to generate a three-dimensional eyebrow shape corresponding to the eyebrow shape image based on the depth values ​​of the root points and the depth values ​​of the outer end points of the plurality of eyebrow reference lines.

[0107] According to the embodiments of the present disclosure, by estimating the depth values ​​of the root point and the outer end point of the eyebrow reference line in the two-dimensional eyebrow image, a three-dimensional eyebrow shape corresponding to the eyebrow image is generated. Therefore, only a small number of eyebrow reference lines need to be marked in the two-dimensional eyebrow image to automatically generate the three-dimensional eyebrow shape corresponding to the two-dimensional eyebrow image, thereby realizing the automatic generation of the three-dimensional eyebrow shape and greatly improving the generation efficiency of the three-dimensional eyebrow shape.

[0108] According to some embodiments, the estimation module 520 includes: an estimation unit, configured to input the eyebrow image into a trained depth estimation model to obtain a depth map corresponding to the eyebrow image, wherein the depth map indicates the depth value of each pixel in the eyebrow image; and a determination unit, configured to determine the depth value of the root point and the depth value of the outer endpoint based on the depth map.

[0109] According to some embodiments, the determination unit includes: a first acquisition subunit, configured to acquire the initial depth value of the root point and the initial depth value of the outer endpoint from the depth map; and a standardization subunit, configured to standardize the initial depth value of the root point and the initial depth value of the outer endpoint based on the heights of the root point and the outer endpoint relative to the face surface to obtain the depth value of the root point and the depth value of the outer endpoint.

[0110] According to some embodiments, the normalization subunit is further configured to: calculate the difference between the initial depth value of the root point and the initial depth value of the outer endpoint; set the depth value of the root point to a preset depth value of the face surface; and set the depth value of the outer endpoint to the sum of the preset depth value and the difference.

[0111] According to some embodiments, the determining unit comprises: a second acquiring subunit configured to acquire the depth value of the root point and the depth value of the outer endpoint from the depth map.

[0112] According to some embodiments, the generation module 530 includes: a first generation unit, configured to generate a three-dimensional eyebrow reference line for any one of the multiple eyebrow reference lines based on the depth value of the root point and the depth value of the outer end point of the eyebrow reference line; an extraction unit, configured to extract an eyebrow shape area from the eyebrow shape image; and a second generation unit, configured to perform three-dimensional eyebrow interpolation in the three-dimensional space corresponding to the eyebrow shape area based on multiple three-dimensional eyebrow reference lines respectively corresponding to the multiple eyebrow reference lines to generate the three-dimensional eyebrow shape.

[0113] According to some embodiments, the first generating unit includes: a first interpolation subunit, configured to obtain the depth value of each of the plurality of intermediate points on the eyebrow reference line by interpolation based on the depth value of the root point and the depth value of the outer end point of the eyebrow reference line; and a generating subunit, configured to generate a three-dimensional eyebrow reference line based on the depth value of each point on the eyebrow reference line.

[0114] According to some embodiments, the first generating unit includes: a determining subunit, configured to determine the three-dimensional coordinates of the root point and the three-dimensional coordinates of the outer endpoint based on the depth value of the root point of the eyebrow reference line, the depth value of the outer endpoint and the intrinsic parameter matrix of the image acquisition device; and a second interpolating subunit, configured to obtain the three-dimensional coordinates of each of the multiple intermediate points on the eyebrow reference line by interpolation based on the three-dimensional coordinates of the root point and the three-dimensional coordinates of the outer endpoint to obtain the three-dimensional eyebrow reference line.

[0115] According to some embodiments, the extraction unit is further configured to: input the eyebrow image and segmentation hint information into a trained semantic segmentation model to obtain the eyebrow region output by the semantic segmentation model, wherein the segmentation hint information includes at least one foreground pixel belonging to the eyebrow region and at least one background pixel not belonging to the eyebrow region.

[0116] It should be understood that Figure 5 The modules and units of the apparatus 500 shown in FIG. 5 can be compared with those of the reference Figure 2 The steps in the method 200 described above correspond to each other. Therefore, the operations, features and advantages described above for the method 200 are also applicable to the device 500 and the modules and units included therein. For the sake of brevity, some operations, features and advantages are not repeated here.

[0117] Although specific functionality is discussed above with reference to specific modules, it should be noted that the functionality of the various modules discussed herein may be separated into multiple modules, and / or at least some functionality of multiple modules may be combined into a single module.

[0118] It should also be understood that various techniques may be described herein in the general context of software hardware elements or program modules. Figure 5The various units described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these units can be implemented as computer program code / instructions, which are configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, these units can be implemented as hardware logic / circuits. For example, in some embodiments, one or more of modules 510-530 can be implemented together in a system on chip (System on Chip, SoC). SoC can include an integrated circuit chip (which includes a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces, and / or one or more components in other circuits), and can optionally execute the received program code and / or include embedded firmware to perform functions.

[0119] According to one aspect of the present disclosure, there is also provided an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned three-dimensional eyebrow generation method 200.

[0120] According to one aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is further provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned method 200 for generating a three-dimensional eyebrow shape.

[0121] According to one aspect of the present disclosure, a computer program product is also provided, including a computer program, which implements the above-mentioned method 200 for generating a three-dimensional eyebrow shape when executed by a processor.

[0122] refer to Figure 6 , a block diagram of an electronic device 600 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0123] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0124] Multiple components in the device 600 are connected to the I / O interface 605, including: an input unit 606, an output unit 607, a storage unit 608, and a communication unit 609. The input unit 606 can be any type of device that can input information to the device 600. The input unit 606 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 607 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 608 can include but is not limited to a disk, an optical disk. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, an 802.11 device, a Wi-Fi device, a WiMAX device, a cellular communication device, and / or the like.

[0125] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the method 200 for generating a three-dimensional eyebrow shape. For example, in some embodiments, the method 200 for generating a three-dimensional eyebrow shape may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method 200 for generating a three-dimensional eyebrow shape described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the method 200 for generating a three-dimensional eyebrow shape in any other appropriate manner (for example, by means of firmware).

[0126] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0128] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0131] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0132] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0133] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present disclosure is not limited by these embodiments or examples. Various elements in the embodiments or examples may be omitted or may be replaced by their equivalent elements. In addition, each step may be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples may be combined in various ways. It is important that with the evolution of technology, many elements described herein may be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A method for generating a three-dimensional eyebrow shape, comprising: Acquire an eyebrow image, wherein the eyebrow image is annotated with a plurality of eyebrow reference lines, each of the plurality of eyebrow reference lines is a curve from a root point to an outer end point of a single eyebrow and is used to describe a single eyebrow in the eyebrow image, and each eyebrow reference line has the following information: pixel coordinates of the root point, pixel coordinates of the outer end point, and a direction of the curve; Performing depth estimation on the eyebrow shape image to obtain depth values ​​of root points and outer end points of each of the plurality of eyebrow reference lines; For any eyebrow reference line among the plurality of eyebrow reference lines, generating a three-dimensional eyebrow reference line based on a depth value of a root point and a depth value of an outer end point of the eyebrow reference line; Extracting an eyebrow region from the eyebrow image; and Based on a plurality of three-dimensional eyebrow reference lines respectively corresponding to the plurality of eyebrow reference lines, a three-dimensional eyebrow shape corresponding to the eyebrow shape image is generated, wherein the three-dimensional eyebrow shape includes the plurality of three-dimensional eyebrow reference lines and can show the direction and texture of a single eyebrow.

2. The method according to claim 1, wherein: The performing depth estimation on the eyebrow image to obtain the depth values ​​of the root points and the depth values ​​of the outer end points of the plurality of eyebrow reference lines comprises: Inputting the eyebrow image into a trained depth estimation model to obtain a depth map corresponding to the eyebrow image, wherein the depth map indicates a depth value of each pixel in the eyebrow image; and Based on the depth map, a depth value of the root point and a depth value of the outer endpoint are determined.

3. The method according to claim 2, wherein: The determining, based on the depth map, the depth value of the root point and the depth value of the outer endpoint comprises: Acquire the initial depth value of the root point and the initial depth value of the outer endpoint from the depth map; and Based on the heights of the root point and the outer endpoints relative to the face surface, the initial depth value of the root point and the initial depth value of the outer endpoints are standardized to obtain the depth value of the root point and the depth value of the outer endpoints.

4. The method according to claim 3, wherein: The step of normalizing the initial depth value of the root point and the initial depth value of the outer endpoint based on the heights of the root point and the outer endpoint relative to the face surface to obtain the depth value of the root point and the depth value of the outer endpoint comprises: Calculate the difference between the initial depth value of the root point and the initial depth value of the outer end point; Setting the depth value of the root point to a preset depth value of the face surface; and The depth value of the outer endpoint is set to the sum of the preset depth value and the difference value.

5. The method according to claim 2, wherein: The determining, based on the depth map, the depth value of the root point and the depth value of the outer endpoint comprises: The depth value of the root point and the depth value of the outer endpoint are obtained from the depth map.

6. The method according to any one of claims 1 to 5, wherein: The generating a three-dimensional eyebrow shape corresponding to the eyebrow shape image based on a plurality of three-dimensional eyebrow reference lines respectively corresponding to the plurality of eyebrow reference lines comprises: Based on a plurality of three-dimensional eyebrow reference lines respectively corresponding to the plurality of eyebrow reference lines, three-dimensional eyebrow interpolation is performed in a three-dimensional space corresponding to the eyebrow shape area to generate the three-dimensional eyebrow shape, wherein the three-dimensional eyebrow shape includes the plurality of three-dimensional eyebrow reference lines and a plurality of three-dimensional eyebrows obtained by interpolation.

7. The method according to claim 6, wherein: The step of generating a three-dimensional eyebrow reference line based on the depth value of the root point and the depth value of the outer end point of the eyebrow reference line comprises: Based on the depth value of the root point and the depth value of the outer end point of the eyebrow reference line, obtain the depth value of each of the plurality of intermediate points on the eyebrow reference line by interpolation; and A three-dimensional eyebrow reference line is generated based on the depth values ​​of each point on the eyebrow reference line.

8. The method according to claim 6, wherein: The eyebrow image is a face image acquired by an image acquisition device, and wherein generating a three-dimensional eyebrow reference line based on a depth value of a root point and a depth value of an outer end point of the eyebrow reference line comprises: Determine the three-dimensional coordinates of the root point and the three-dimensional coordinates of the outer endpoint based on the depth value of the root point of the eyebrow reference line, the depth value of the outer endpoint and the intrinsic parameter matrix of the image acquisition device; and Based on the three-dimensional coordinates of the root point and the three-dimensional coordinates of the outer end point, the three-dimensional coordinates of each of the plurality of intermediate points on the eyebrow reference line are obtained by interpolation to obtain the three-dimensional eyebrow reference line.

9. The method according to claim 1, wherein: The extracting the eyebrow region from the eyebrow image comprises: The eyebrow image and segmentation hint information are input into a trained semantic segmentation model to obtain the eyebrow region output by the semantic segmentation model, wherein the segmentation hint information includes at least one foreground pixel belonging to the eyebrow region and at least one background pixel not belonging to the eyebrow region.

10. A device for generating a three-dimensional eyebrow shape, comprising: an acquisition module, configured to acquire an eyebrow image, wherein the eyebrow image is annotated with a plurality of eyebrow reference lines, each of the plurality of eyebrow reference lines is a curve from a root point to an outer end point of a single eyebrow and is used to describe a single eyebrow in the eyebrow image, and each eyebrow reference line has the following information: pixel coordinates of the root point, pixel coordinates of the outer end point, and a direction of the curve; an estimation module configured to perform depth estimation on the eyebrow shape image to obtain depth values ​​of root points and outer end points of each of the plurality of eyebrow reference lines; and Generate modules, including: A first generating unit is configured to generate a three-dimensional eyebrow reference line for any eyebrow reference line among the plurality of eyebrow reference lines based on a depth value of a root point and a depth value of an outer end point of the eyebrow reference line; an extracting unit, configured to extract an eyebrow region from the eyebrow image; and The second generating unit is configured to generate a three-dimensional eyebrow shape corresponding to the eyebrow shape image based on a plurality of three-dimensional eyebrow reference lines respectively corresponding to the plurality of eyebrow reference lines, wherein the three-dimensional eyebrow shape includes the plurality of three-dimensional eyebrow reference lines and can show the direction and texture of a single eyebrow.

11. The device according to claim 10, wherein: The estimation module comprises: an estimation unit configured to input the eyebrow image into a trained depth estimation model to obtain a depth map corresponding to the eyebrow image, wherein the depth map indicates a depth value of each pixel in the eyebrow image; and A determining unit is configured to determine a depth value of the root point and a depth value of the outer endpoint based on the depth map.

12. The device according to claim 11, wherein: The determining unit comprises: A first acquisition subunit is configured to acquire an initial depth value of the root point and an initial depth value of the outer endpoint from the depth map; and The standardization subunit is configured to standardize the initial depth value of the root point and the initial depth value of the outer endpoint based on the heights of the root point and the outer endpoint relative to the face surface to obtain the depth value of the root point and the depth value of the outer endpoint.

13. The device according to claim 12, wherein: The standardization subunit is further configured to: Calculate the difference between the initial depth value of the root point and the initial depth value of the outer end point; Setting the depth value of the root point to a preset depth value of the face surface; as well as The depth value of the outer endpoint is set to the sum of the preset depth value and the difference value.

14. The device according to claim 11, wherein: The determining unit comprises: The second acquisition subunit is configured to acquire the depth value of the root point and the depth value of the outer endpoint from the depth map.

15. The device according to any one of claims 10 to 14, wherein: The second generating unit is further configured to: Based on a plurality of three-dimensional eyebrow reference lines respectively corresponding to the plurality of eyebrow reference lines, three-dimensional eyebrow interpolation is performed in a three-dimensional space corresponding to the eyebrow shape area to generate the three-dimensional eyebrow shape, wherein the three-dimensional eyebrow shape includes the plurality of three-dimensional eyebrow reference lines and a plurality of three-dimensional eyebrows obtained by interpolation.

16. The device according to claim 15, wherein: The first generating unit comprises: A first interpolation subunit is configured to obtain the depth value of each of the plurality of intermediate points on the eyebrow reference line by interpolation based on the depth value of the root point and the depth value of the outer end point of the eyebrow reference line; and The generating subunit is configured to generate a three-dimensional eyebrow reference line based on the depth value of each point on the eyebrow reference line.

17. The device according to claim 15, wherein: The eyebrow image is a face image acquired by an image acquisition device, and wherein the first generating unit comprises: a determination subunit configured to determine the three-dimensional coordinates of the root point and the three-dimensional coordinates of the outer endpoint based on the depth value of the root point of the eyebrow reference line, the depth value of the outer endpoint and the intrinsic parameter matrix of the image acquisition device; and The second interpolation subunit is configured to obtain the three-dimensional coordinates of each of the plurality of intermediate points on the eyebrow reference line through interpolation based on the three-dimensional coordinates of the root point and the three-dimensional coordinates of the outer end point to obtain the three-dimensional eyebrow reference line.

18. The device according to claim 10, wherein: The extraction unit is further configured to: The eyebrow image and segmentation hint information are input into a trained semantic segmentation model to obtain the eyebrow region output by the semantic segmentation model, wherein the segmentation hint information includes at least one foreground pixel belonging to the eyebrow region and at least one background pixel not belonging to the eyebrow region.

19. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-9.

21. A computer program product comprising a computer program, wherein: The computer program implements the method according to any one of claims 1 to 9 when executed by a processor.