A three-dimensional reconstruction method and device for bust

Abstract

The present application provides a method and device for three-dimensional reconstruction of a bust, which is used to solve the problems of high complexity and long reconstruction time. The method includes: obtaining an image of the front face of a bust of a target person, and then using the semantic mask of the head of the front face to unfold the texture, so that at least two organs in the unfolded texture are located at preset positions, and then the texture of the back is completed according to the unfolded texture map including the front. A three-dimensional mesh model can also be constructed based on the front face, and a pre-configured ear model is used to replace the ear in the three-dimensional mesh model, and then the texture of the ear area is obtained from the replaced three-dimensional mesh model, and it is fused to the texture of the completed back, thereby obtaining a three-dimensional mesh model. No professional modeling equipment is required, and the complexity is low. Since there is a certain correlation between the textures of the front and back of the human body, the back texture is supplemented according to the front texture, so that the constructed three-dimensional model fits the person better and achieves a better effect.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a method and device for three-dimensional reconstruction of a bust. Background Art

[0002] At present, 3D bust reconstruction technology has been widely used in the fields of 3D printing, entertainment, and remote augmented reality (AR) calls. Traditional 3D bust reconstruction technology can use a monocular system or a multi-eye system. When using a monocular system, an infrared depth camera is usually used to scan the person in a circle, and the scanned person needs to remain still throughout the scanning process. As a result, this method takes a long time to scan the person, the reconstruction calculation time is long, and the scanning effect may be unsatisfactory or fail. The multi-eye scanning system is a collection system based on multiple perspectives. Although it has the ability of real-time reconstruction, it is expensive, the equipment is large, the complexity is high, and it is not very convenient to operate. Summary of the invention

[0003] The embodiments of the present application provide a method and device for three-dimensional reconstruction of a bust, so as to solve the problems of long reconstruction time and high complexity.

[0004] The specific technical solutions provided by the embodiments of this application are as follows:

[0005] In the first aspect, the present application provides a method for three-dimensional reconstruction of a bust, which can be implemented by an electronic device, such as one or a processing unit of the electronic device. The method may include: obtaining a to-be-processed image of a bust of a target person, wherein the bust includes a front face, and then obtaining a first texture expansion map according to the obtained to-be-processed image, wherein the first texture expansion map is used to characterize the front texture of the bust, and at least two organs (such as a nose and an ear) of the facial features of the bust in the first texture expansion map are located at preset positions; then, according to the front texture of the bust in the first texture expansion map, the back texture of the bust in the first texture expansion map is supplemented to obtain a second texture expansion map, wherein the second texture expansion map is used to characterize the surface texture of the bust; finally, according to the second texture expansion map, a three-dimensional model of the bust is obtained. The reconstruction method provided by the present application does not require professional modeling equipment and has low complexity. Since there is a certain correlation between the front and back textures of the human body, the back texture is supplemented according to the front texture, so that the constructed three-dimensional model fits the person better and achieves a better effect.

[0006] The second texture development map is used to characterize the surface texture of the bust, that is, the second texture development map describes the omnidirectional texture of the surface of the bust. The surface texture includes the front texture of the bust and the back texture of the bust. For example, the front of the bust includes the face, the front neck or the front shoulders, etc. The back of the bust may include the back of the head, the back neck or the back shoulders, etc.

[0007] In a possible design, obtaining the first texture expansion map according to the image to be processed can be achieved in the following manner:

[0008] Removing the background from the image to be processed to obtain a front image of the bust;

[0009] Performing semantic segmentation on the front image to obtain a head semantic mask of the front image;

[0010] Obtaining a three-dimensional mesh model of the bust according to the head semantic mask and the frontal image;

[0011] A first texture expansion map is obtained according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, wherein the at least two organs in the first texture expansion map are located at preset positions.

[0012] Through the above design, the texture unfolding map is obtained according to the head semantic mask, that is, the texture is unfolded according to the semantic pixel coordinates to improve the unfolding accuracy.

[0013] In a possible design, obtaining the first texture unfolded image according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model can be implemented in the following manner:

[0014] Performing texture expansion on the texture corresponding to the three-dimensional mesh model based on a central axis to obtain a third texture expansion image, wherein the central axis is a connecting line from the top end of the head to the bottom end of the head in the three-dimensional mesh model;

[0015] Determining the positions of the at least two organs in the three-dimensional mesh model according to the positions of the at least two organs on the head in the head semantic mask;

[0016] The third texture expansion map is adjusted according to the locations of the at least two organs in the three-dimensional mesh model to obtain the first texture expansion map.

[0017] In the above design, after obtaining the bust 3D mesh model, the bust texture can be circularly expanded to obtain the first texture expansion map in combination with the front texture of the bust. Circular expansion is a non-Atlas texture expansion. Non-atlas expansion is conducive to improving texture continuity and reducing gaps between textures. In addition, it can also avoid block repair (inpainting), while achieving a relatively fixed position of the semantic block on the texture map, which is convenient for machine learning.

[0018] In a possible design, obtaining the three-dimensional model of the bust based on the second texture unfolded image includes:

[0019] The texture stitching line area in the second texture unfolding image is smoothed to obtain a fourth texture unfolding image, and the three-dimensional model of the bust is obtained based on the fourth texture unfolding image. The texture stitching line area is determined according to the unfolding line used when the texture of the three-dimensional mesh model is unfolded.

[0020] The above design smoothes the seam area, which can eliminate the gaps in the seam area and optimize the modeling details.

[0021] In one possible design, the at least two organs include an ear; and the method further comprises:

[0022] Fusing the preconfigured ear model into the ear region in the three-dimensional mesh model to obtain a fused three-dimensional mesh model;

[0023] Obtaining a three-dimensional model of the bust based on the processed second texture development image includes:

[0024] The texture of the ear area on the fused three-dimensional network model is fused to the ear area located at a preset position of the fourth texture expansion map to obtain a fused fourth texture expansion map, and the three-dimensional model of the bust is obtained according to the fused fourth texture expansion map.

[0025] The above design achieves high-precision local texture and geometry, optimizes texture completion and mesh reconstruction details through mesh optimization and texture replacement of the ear.

[0026] In a possible design, smoothing the texture seam line area in the second texture expansion image may be achieved in the following manner:

[0027] Determine a back texture image of the bust according to the front image of the bust;

[0028] The second texture expansion map and the back texture image of the bust are subjected to weighted fusion processing to obtain the fourth texture expansion map.

[0029] In the above design, since there is a certain correlation between the front and back textures of the human body, the back texture is estimated based on the front image. The estimated back texture does not contain gaps, and then it is weighted with the back texture obtained based on the second texture expansion map to optimize the back gap smoothing effect.

[0030] In a possible design, weighted fusion processing is performed on the second texture expansion image and the back texture image of the bust to obtain the fourth texture expansion image, which can be achieved in the following manner:

[0031] Performing weighted fusion processing on the second texture expansion image and the back image of the bust according to a set rule to obtain the fourth texture expansion image;

[0032] The setting rules are:

[0033] I3(i,j)=αI1(i,j)+(1-α)I2(map1(i),map2(j));

[0034] α=I alpha (i,j);

[0035] Wherein, I1 represents the second texture expansion map, I2 represents the back image of the bust, I3 represents the fourth texture expansion map, I alpha represents the weight, map1 represents the mapping function of mapping the pixel points in the back image of the bust to the second texture expansion map in the X-axis direction, map2 represents the mapping function of mapping the pixel points in the back image of the bust to the second texture expansion map in the Y-axis direction, i represents the coordinate value of the pixel point in the X-axis direction, and j represents the coordinate value of the pixel point in the X-axis direction.

[0036] In a possible design, before obtaining the first texture unfolding map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, the method further includes:

[0037] Perform at least one of the following processing on the three-dimensional mesh model:

[0038] Hole filling, mesh homogenization or mesh smoothing.

[0039] In the above design, after the three-dimensional mesh model is filled with holes, the three-dimensional mesh model is made more complete, and the accuracy of the three-dimensional model reconstruction is improved. Performing mesh homogenization processing can prevent the mesh in the obtained three-dimensional mesh model from being too dense or too sparse, which affects the accuracy of the three-dimensional model reconstruction. Performing mesh smoothing processing can remove inaccurate meshes, i.e., noise points, in the three-dimensional mesh model, thereby improving the accuracy and smoothness of the three-dimensional model reconstruction.

[0040] In a second aspect, the present application provides a three-dimensional reconstruction device for a bust, comprising:

[0041] An acquisition unit, used for acquiring an image to be processed, wherein the image to be processed includes a bust of a target person, wherein the bust includes a frontal face;

[0042] A reconstruction unit is used to obtain a first texture expansion map according to the image to be processed, wherein the first texture expansion map is used to characterize the front texture of the bust, and at least two organs among the facial features of the bust in the first texture expansion map are located at preset positions; a second texture expansion map is obtained by supplementing the back texture of the bust in the first texture expansion map according to the front texture of the bust in the first texture expansion map, wherein the second texture expansion map is used to characterize the surface texture of the bust; and a three-dimensional model of the bust is obtained according to the second texture expansion map.

[0043] In a possible design, when the reconstruction unit obtains the first texture expansion map according to the image to be processed, it is specifically used to:

[0044] Removing the background from the image to be processed to obtain a front image of the bust;

[0045] Performing semantic segmentation on the front image to obtain a head semantic mask of the front image;

[0046] Obtaining a three-dimensional mesh model of the bust according to the head semantic mask and the frontal image;

[0047] A first texture expansion map is obtained according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, wherein the at least two organs in the first texture expansion map are located at preset positions.

[0048] In a possible design, when the reconstruction unit obtains the first texture unfolding map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, it is specifically used to:

[0049] Performing texture expansion on the texture corresponding to the three-dimensional mesh model based on a central axis to obtain a third texture expansion image, wherein the central axis is a connecting line from the top end of the head to the bottom end of the head in the three-dimensional mesh model;

[0050] Determining the positions of the at least two organs in the three-dimensional mesh model according to the positions of the at least two organs on the head in the head semantic mask;

[0051] The third texture expansion map is adjusted according to the locations of the at least two organs in the three-dimensional mesh model to obtain the first texture expansion map.

[0052] In a possible design, when obtaining the three-dimensional model of the bust based on the second texture unfolded image, the reconstruction unit is specifically configured to:

[0053] The texture stitching line area in the second texture unfolding image is smoothed to obtain a fourth texture unfolding image, and the three-dimensional model of the bust is obtained based on the fourth texture unfolding image. The texture stitching line area is determined according to the unfolding line used when the texture of the three-dimensional mesh model is unfolded.

[0054] In a possible design, the at least two organs include an ear; the reconstruction unit is further used to fuse the preconfigured ear model into the ear region in the three-dimensional mesh model to obtain a fused three-dimensional mesh model;

[0055] The reconstruction unit, when obtaining the three-dimensional model of the bust based on the processed second texture unfolded image, is specifically used to:

[0056] The texture of the ear area on the fused three-dimensional network model is fused to the ear area located at a preset position of the fourth texture expansion map to obtain a fused fourth texture expansion map, and the three-dimensional model of the bust is obtained according to the fused fourth texture expansion map.

[0057] In a possible design, when the reconstruction unit performs smoothing on the texture seam line region in the second texture expansion image, it is specifically used to:

[0058] Determine a back texture image of the bust according to the front image of the bust;

[0059] The second texture expansion map and the back texture image of the bust are subjected to weighted fusion processing to obtain the fourth texture expansion map.

[0060] In a possible design, when the reconstruction unit performs weighted fusion processing on the second texture expansion map and the back texture image of the bust to obtain the fourth texture expansion map, it is specifically used to:

[0061] Performing weighted fusion processing on the second texture expansion image and the back image of the bust according to a set rule to obtain the fourth texture expansion image;

[0062] The setting rules are:

[0063] I3(i,j)=αI1(i,j)+(1-α)I2(map1(i),map2(j));

[0064] α=I alpha (i,j);

[0065] Wherein, I1 represents the second texture expansion map, I2 represents the back image of the bust, I3 represents the fourth texture expansion map, I alpha represents the weight, map1 represents the mapping function of mapping the pixel points in the back image of the bust to the second texture expansion map in the X-axis direction, map2 represents the mapping function of mapping the pixel points in the back image of the bust to the second texture expansion map in the Y-axis direction, i represents the coordinate value of the pixel point in the X-axis direction, and j represents the coordinate value of the pixel point in the X-axis direction.

[0066] In one possible design, the reconstruction unit, before obtaining the first texture unfolding map based on the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, is also used to: perform at least one of the following processing on the three-dimensional mesh model: hole filling processing, mesh homogenization processing or mesh smoothing processing.

[0067] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory; wherein the processor is coupled to the memory; wherein the memory is used to store program instructions; and the processor is used to read the program instructions stored in the memory to implement the method of the first aspect and any possible design thereof. Optionally, the processor includes an ISP for executing a process of acquiring an image to be processed.

[0068] In a possible design, the electronic device further includes a camera, and the camera is used to collect images to be processed. The processor is used to control the camera to collect images.

[0069] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores program instructions. When the program instructions are executed on an electronic device, the electronic device or a processor executes the method of the first aspect and any possible design thereof.

[0070] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device or a processor to execute the method of the first aspect and any possible design thereof.

[0071] In a sixth aspect, an embodiment of the present application provides a chip, which is coupled to a memory in an electronic device and executes the method of the first aspect and any possible design thereof. Optionally, the chip includes an ISP for executing a process of acquiring an image to be processed.

[0072] In addition, the technical effects brought about by the second to sixth aspects can be found in the description of the first aspect above and will not be repeated here.

[0073] It should be noted that, in the embodiments of the present application, “coupling” refers to two components being directly or indirectly coupled to each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 A schematic diagram of an electronic device in an embodiment of the present application;

[0075] Figure 2 This is a schematic diagram of the process of a three-dimensional reconstruction method of a bust in an embodiment of the present application;

[0076] Figure 3 This is a schematic diagram of a bust in an embodiment of the present application;

[0077] Figure 4 This is a schematic diagram of the method flow of texture expansion in an embodiment of the present application;

[0078] Figure 5 This is a schematic diagram of a three-dimensional reconstruction of a bust in an embodiment of the present application;

[0079] Figure 6 This is a schematic diagram of the central axis in the embodiment of the present application;

[0080] Figure 7 This is a schematic diagram for explaining the circumference expansion in the embodiment of the present application;

[0081] Figure 8 This is a schematic diagram of determining the positions of the nose and ears in an embodiment of the present application;

[0082] Fig. 9 This is a schematic diagram of gap smoothing in an embodiment of the present application;

[0083] Fig.10 Schematic diagram of the smoothing method used for gap leveling in the embodiment of the present application;

[0084] Fig.11 This is a schematic diagram of ear suture in an embodiment of the present application;

[0085] Fig.12 is a schematic diagram of a reconstructed three-dimensional model in an embodiment of the present application;

[0086] Fig.13 This is a schematic diagram of a three-dimensional virtual video call scene in an embodiment of the present application;

[0087] Fig.14 Schematic diagram of device 1400 in an embodiment of the present application. DETAILED DESCRIPTION

[0088] The three-dimensional reconstruction scheme of the bust involved in the present application can be applied in application scenarios such as three-dimensional printing, AR calls, virtual reality (VR) models, etc. The three-dimensional reconstruction method of the bust provided in the present application can be applied to electronic devices. The electronic device can be a personal computer, a server computer, a client, a handheld or laptop device, a microprocessor-based system device, an embedded system device, a set-top box, a programmable consumer electronic product, a network personal computer, and a small computer, a large computer system and a distributed cloud computing technology environment server including any of the above systems, as well as a portable terminal device with functions such as a personal digital assistant and / or image processing, such as a mobile phone, a tablet computer, a wearable device with wireless communication function (such as a smart watch), a vehicle-mounted device, and the like.

[0089] See also Figure 1 As shown, the electronic device involved in the embodiment of the present application may include a processor 110 and a memory 120. The processor 110 may include one or more processing units. For example, the processor 110 may include one or more of a central processing unit (CPU), a graphics processing unit (GPU), an image signal processor (ISP), a digital signal processor (DSP), or a neural processing unit (NPU). Among them, different processing units may be independent devices or integrated in one or more chips or circuit boards. Among them, the digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The neural processor includes but is not limited to a neural network processing unit, such as a deep neural network processing unit or a convolutional neural network processing unit. The neural processor can perform training, calculation or processing using a neural network model. The neural network model includes but is not limited to a deep neural network model or a convolutional neural network model. The above digital signal processor, image processing unit or central processing unit can also perform training, calculation or processing using the neural network model.

[0090] In some embodiments, a memory may be provided in the processor 110 for temporarily storing instructions and data. For example, the memory in the processor 110 may be a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0091] In other embodiments, the processor 110 may further include one or more interfaces. For example, the interface may be a universal serial bus (USB) interface. For another example, the interface may also be an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, etc. It is understandable that the embodiment of the present application may connect different modules including a processor of an electronic device through an interface, so that the electronic device can realize different functions. It should be noted that the embodiment of the present application does not limit the connection method of the interface in the electronic device 100.

[0092] In one example, the NPU included in the processor 110 is a neural network (NN) computing processor, which can quickly process input information and continuously self-learn by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain. The NPU can realize three-dimensional reconstruction of the bust of the electronic device 100.

[0093] In another example, the processor 110 includes an NPU and other processors, and the NPU and other processors can realize three-dimensional reconstruction of the bust of the electronic device 100. The other processors can be, for example, one or more of a CPU, a graphics processing unit (GPU), an image signal processor (ISP), and a digital signal processor (DSP).

[0094] The memory 120 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the memory 120. The memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, a driver software, or an application required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area may store data (such as audio data, a phone book, etc.) created during the use of the electronic device 100. The memory 120 may include at least one of a power-off volatile memory or a non-power-off volatile memory, such as a read-only memory (ROM), a random access memory (RAM), a dynamic random access memory (DRAM), an embedded multimedia card (eMMC), a universal flash storage (UFS), a hard disk or a disk, etc.

[0095] In a possible implementation, the electronic device may further include an image collector 130 for collecting images. The image collector 130 may include a camera, and may further include the ISP mentioned above. The ISP is used to process the image data collected by the camera. For example, when taking a photo, the shutter is opened, and the light is transmitted to the photosensitive element in the camera through the lens in the camera. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP, and the ISP processes the relevant data and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and chromaticity of the image. The ISP can also control the exposure of the image, or optimize parameters such as color temperature according to the shooting scene. In some embodiments, the ISP can be set in the camera. More commonly, the ISP can exist as a part of the processor and be integrated with other processing units such as CPU, GPU or DSP on one or more chips.

[0096] The camera is used to capture static images or dynamic videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the processor 110, which is processed by other processing units. In some embodiments, the electronic device may include one or more cameras.

[0097] In a possible implementation, the electronic device may further include a display screen 140, which is used to display images, videos, etc. In some embodiments, the electronic device may include one or more display screens. The display screen includes but is not limited to a touch screen. In an embodiment of the present application, the display screen may be used to display a three-dimensional model of a person's reconstructed bust.

[0098] The term "at least one" involved in this application refers to one, or more than one, that is, including one, two, three and more; "at least two" refers to two or more than two, that is, including two, three and more. "Multiple" refers to two, or more than two, that is, including two, three and more. In addition, it should be understood that in the description of this application, the words "first" and "second" are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order. "And / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following items (items)" or similar expressions refer to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0099] See also Figure 2 FIG. 1 is a flow chart of a method for 3D reconstruction of a bust according to an embodiment of the present application. The method for 3D reconstruction of a bust can be performed by Figure 2 The electronic device shown is implemented, for example, by one of the electronic devices or a processing unit. Figure 2 As shown, the three-dimensional reconstruction method of the bust mainly includes S201-S204.

[0100] S201, obtaining an image to be processed, where the image to be processed includes a bust of a target person, and the bust includes a frontal face.

[0101] The acquisition action of S201 can be understood as a receiving operation or a processing operation. For example, the NPU can receive the image to be processed sent by other devices, such as an ISP, in S201. For another example, S201 can be executed by the ISP to generate an image by processing the image signal or data collected by the camera. The processing includes but is not limited to various types of color calibration, pixel calibration, white balance or scaling. In an embodiment of the present application, the image to be processed may include a frontal face of a bust.

[0102] For example, the bust may include the front face, neck, shoulders, etc., for example, see Figure 3 shown.

[0103] S202: Obtain a first texture expansion map according to the image to be processed, where the first texture expansion map is used to represent the front texture of the bust.

[0104] Exemplarily, at least two organs among the facial features of the bust in the first texture expansion map are located at preset positions. The facial features include eyebrows, eyes, ears, nose, and mouth. At least two organs, such as ears and nose, or eyes and nose. At least two organs are located at preset positions, which may be the same position of the at least two organs in the texture expansion maps obtained for different images to be processed. For example, the two organs include a nose and an ear, and the position of the ear in the texture expansion maps of different images to be processed is the same, and the position of the nose in the texture expansion maps of different images to be processed is the same. Or at least two organs are located at preset positions, which may be the same relative position of the at least two organs in the texture expansion maps obtained for different images to be processed. Taking two ears and a nose as an example, in the texture expansion maps of different images to be processed, the distance between the two ears and the distance between the nose and the ear may be the same.

[0105] S203: Supplement the back texture of the bust in the first texture expansion image with the front texture of the bust in the first texture expansion image to obtain a second texture expansion image. The second texture expansion image is used to represent the surface texture of the bust.

[0106] The second texture development map is used to characterize the surface texture of the bust, that is, the second texture development map describes the omnidirectional texture of the surface of the bust. The surface texture can be the texture of the upper body surface of the three-dimensional target person. That is, the surface texture includes a texture around the upper body surface with the direction perpendicular to the ground plane as the axis.

[0107] The surface texture includes the front texture of the bust and the back texture of the bust. For example, the front of the bust includes the face, the front neck or the front shoulders, etc. The back of the bust may include the back of the head, the back neck or the back shoulders, etc.

[0108] S204, obtaining a three-dimensional model of the bust according to the second texture development graph.

[0109] In one possible implementation, see Figure 4 As shown, S202 obtaining the first texture expansion map according to the image to be processed can be implemented through the following S401-S404.

[0110] S401, removing the background in the image to be processed to obtain a front image of the bust.

[0111] Exemplarily, the first neural network model can be used to remove the background in the processed image to obtain a frontal image of the bust. The first neural network model is used to segment the foreground and background of the image and output a foreground image. In the implementation of the present application, the foreground image is the frontal image of the bust.

[0112] S402, performing semantic segmentation on the front image to obtain a head semantic mask of the front image.

[0113] Semantic segmentation is the process of grouping / segmenting pixels according to the different semantic meanings they express in the image.

[0114] Exemplarily, semantic segmentation of the front image can be performed using a fully convolutional network (FCN), such as a U-net network, a SegNet network, DeepLab, RefineNet, or PSPNet.

[0115] S403, obtaining a three-dimensional mesh model of the bust according to the head semantic mask and the front image.

[0116] Exemplarily, the semantic mask of the head and the frontal image can be input into a second neural network model, and the second neural network model is used for human body reconstruction. The second neural network model outputs a three-dimensional (3D) truncated signed distance function (TSDF) volume. Then, the surface mesh of the TSDF volume is extracted to obtain a three-dimensional mesh model of the bust.

[0117] For example, when extracting the TSDF volume surface mesh, a marching cube algorithm or other three-dimensional isosurface extraction algorithms may be used.

[0118] S404: Obtain a first texture expansion map according to the three-dimensional mesh model.

[0119] In a possible implementation, before obtaining the first texture unfolding map according to the three-dimensional mesh model, at least one of the following processing may be performed on the three-dimensional mesh model: hole filling processing, mesh homogenization processing, or mesh smoothing processing.

[0120] For example, when performing hole filling processing on a three-dimensional mesh model, a triangular mesh hole filling method based on radial basis function (RBF) or a hole filling algorithm based on Poisson's equation can be used. When performing mesh homogenization processing on a three-dimensional mesh model, mesh homogenization algorithms such as point clustering method, edge collapse and vertex addition and deletion method can be used. When performing mesh smoothing processing on a three-dimensional mesh model, a mesh smoothing method based on Poisson's equation or discrete Laplace equation can be used.

[0121] After filling holes in the 3D mesh model, the 3D mesh model becomes more complete and the accuracy of 3D model reconstruction is improved. Mesh homogenization can prevent the mesh in the obtained 3D mesh model from being too dense or too sparse, which affects the accuracy of 3D model reconstruction. Mesh smoothing can remove inaccurate meshes, i.e. noise points, in the 3D mesh model, thereby improving the accuracy and smoothness of 3D model reconstruction.

[0122] Exemplarily, when it is determined based on the head semantics that the bust includes ears, when obtaining the first texture unfolded map of the three-dimensional mesh model, the first texture unfolded map of the three-dimensional mesh model can be obtained based on the positions of at least two organs on the head in the head semantic mask, so that at least two organs in the first texture unfolded map are located at preset positions.

[0123] For example, after obtaining the bust 3D mesh model, the bust texture can be circularly expanded to obtain a first texture expansion map in combination with the front texture of the bust. Circular expansion is a non-Atlas texture expansion. Non-atlas expansion is conducive to improving texture continuity and reducing gaps between textures. In addition, inpainting can be avoided, and the position of the semantic block on the texture map can be relatively fixed, which is convenient for machine learning.

[0124] As an example, Figure 3 Take the bust as an example, after A1 processing, we get Figure 5 The frontal image of the bust is shown in (a). The semantic segmentation of the frontal image of the bust is performed to obtain the semantic mask of the head, that is, the semantic mask of the head obtained after A2 processing is as follows: Figure 5 The head semantic mask and the front image can be input into the TSDF output by the second neural network model as shown in (b). Figure 5 As shown in (c), the surface mesh of the TSDF body is extracted to obtain the three-dimensional mesh model of the bust. Figure 5 Then, after circular expansion, the first texture expansion image is obtained as shown in (d). Figure 5 As shown in (e) in the figure. Further, after being processed in S203, the following is obtained: Figure 5 The second texture expansion image shown in (f) is the texture image after texture completion. Finally, the texture image after texture completion and the surface mesh of the three-dimensional mesh model are combined to obtain the final three-dimensional model of the bust with texture, as shown in FIG. Figure 5 As shown in (g).

[0125] In a possible implementation, S404 obtains a first texture unfolded image of the three-dimensional mesh model according to the positions of at least two organs on the head in the head semantic mask, which can be implemented as follows:

[0126] A1, performing texture expansion on the texture corresponding to the three-dimensional mesh model based on the central axis to obtain a third texture expansion map, wherein the central axis is a connecting line from the top of the head to the bottom of the head in the three-dimensional mesh model. For example, the top of the head may be the highest point on the top of the head. The bottom of the head may be the geometric center of the lowest surface of the three-dimensional mesh model. Figure 6 As shown, this is a schematic diagram of the central axis of the three-dimensional grid model.

[0127] For example, the third texture expansion map can be obtained based on the central axis according to the spatial angle or surface circumference of a certain point on the Mesh relative to the central axis. For example, the circumferential expansion according to the angle can satisfy the first rule. The first rule can be:

[0128]

[0129] Where (x, y, z) is the spatial coordinate of a point in a three-dimensional mesh model, [u, v] represents the pixel coordinate of the point with spatial coordinates (x, y, z) in the texture expansion map, and the pixel coordinate takes the lower left corner of the image as the origin.

[0130] Where W is the width of the texture expansion map, H is the height of the texture expansion map, and cx and cz represent the axis coordinates on the section with equal Y value.

[0131] It should be understood that the third texture expansion map describes the mesh texture of the three-dimensional mesh model, or is a mesh texture expansion map without pixel values ​​of pixels.

[0132] A2, determining the positions of at least two organs in the three-dimensional mesh model according to the positions of at least two organs on the head in the head semantic mask.

[0133] The head semantic mask includes the locations of at least two organs, that is, the coordinates of at least two organs can be determined according to the head semantic mask. Take the two organs as ears and nose as an example. In addition, it should be understood that the three-dimensional mesh model is obtained according to the head semantic mask, so there is a one-to-one mapping relationship between the ears and nose in the head semantic mask and the ears and nose in the three-dimensional mesh model, so the positions of the ears and nose in the three-dimensional mesh model can be determined according to the positions of the ears and nose in the head semantic mask.

[0134] A3, adjusting the third texture expansion map according to the locations of at least two organs in the three-dimensional mesh model to obtain a first texture expansion map.

[0135] For example, see Figure 8 shown. Figure 8 (a) shows the locations of ears and nose in the head semantic mask. Figure 8 (b) shows the positions of the ears and nose in the 3D mesh model.

[0136] One way is to adjust the ear and / or nose positions in the third texture expansion map according to the ear and nose positions in the three-dimensional mesh model, that is, determine the ear and / or nose positions in the third texture expansion map according to the ear and nose positions in the three-dimensional mesh model, and adjust the ear and / or nose positions so that the adjusted ear and nose are located at the preset positions of the third texture expansion map, and obtain the first texture expansion map according to the adjusted third texture expansion map and the front image, and the adjusted ear and nose are located at the preset positions of the first texture expansion map. For example, the third texture expansion map can be subjected to a distortion mapping process so that the ear and nose are located at the preset positions in the obtained first texture expansion map.

[0137] Another way is to obtain a third texture expansion map filled with color according to the third texture expansion map and the front image. The positions of the ears and nose in the third texture expansion map filled with color are determined according to the positions of the ears and nose in the three-dimensional network model, and the positions of the ears and / or nose are adjusted to obtain the first texture expansion map. The ears and nose are located at the preset positions of the first texture expansion map.

[0138] There is a mapping relationship between the coordinates of each pixel point in the third texture expansion map and the coordinates of each point in the three-dimensional mesh model, as shown in the first rule. Since the three-dimensional mesh model is obtained based on the front image, there is a mapping relationship between the coordinates of each point in the three-dimensional mesh model and the coordinates of the pixel point in the front image. Furthermore, there is a mapping relationship between the coordinates of each pixel point in the third texture expansion map and the coordinates of the pixel point in the front image. Therefore, the pixel value of the pixel point in the front image can be mapped to the third texture expansion map (or mapped to the adjusted third texture expansion map).

[0139] At present, the commonly used half-body texture unfolding is to unfold the front and back separately, that is, unfold into the texture of the visible part on the front and the invisible part on the back, resulting in poor texture reconstruction of the invisible hair and ears, and poor geometric mesh reconstruction of the ears, which is difficult to meet user needs. Through the above scheme, by performing semantically aligned texture unfolding on the mesh model, a complete texture unfolding image is obtained, which can improve the effect of texture completion of the invisible part; and through mesh optimization and texture replacement of the ear part, high-precision local texture and geometry are achieved, which can produce better texture completion and mesh detail reconstruction effects.

[0140] It should be noted that the adjusted ears and nose are located at the preset positions of the first texture expansion map. This process provides convenient conditions for the subsequent optimization of the ear part and can improve the accuracy of the subsequent optimization of the ear. The optimization of the ear part will be described later and will not be repeated here.

[0141] As a possible implementation, for a bust with long hair, the hair may completely cover the ear area. In this case, the subsequent optimization of the ear part may not be performed. Based on this, when performing texture expansion on the texture corresponding to the three-dimensional mesh model, the texture expansion map can be obtained based on the central axis according to the spatial angle or the perimeter of the surface at a certain point, and A2 and A3 may not be performed. In this scenario, the third texture expansion map filled with color can be directly obtained according to the third texture expansion map and the front image, that is, the filled third texture expansion map is used as the first texture expansion map.

[0142] As an example, see Figure 7 As shown in (a), a certain cross section of the head can be considered as a circle. Figure 7 In (a), the black dots between -π and π can be considered as the points on the cross section of the unfolding line used for texture unfolding. For example, after the distortion, the nose needs to be at the position of 0π, and the two ears are at the positions of -π and π respectively, that is, the two ears are symmetrical with the mouth as the center. Figure 7 As can be seen in (a), before the distortion process, the nose and the two ears are asymmetrical. Figure 7 As shown in (b), after being processed by A3, in the first texture expansion, the nose is located at the position of 0π. In order to more intuitively show the location of the ear, you can refer to Figure 7 (c) is a schematic diagram of the second texture expansion image after being processed in S203. Figure 7 In (c), the nose is located at position 0π, and the two ears are located at positions -π and π respectively.

[0143] After the back texture of the bust in the first texture expansion image is supplemented with the back texture of the bust in the first texture expansion image according to the front texture of the bust in the first texture expansion image to obtain the second texture expansion image, if the back seam line area is processed when obtaining the three-dimensional model of the bust according to the second texture expansion image, there may be gaps in the seam line area of ​​the obtained three-dimensional model. As an example, when obtaining the three-dimensional model of the bust based on the second texture expansion image, the texture seam line area in the second texture expansion image may be smoothed to obtain a fourth texture expansion image, and the three-dimensional model of the bust is obtained based on the fourth texture expansion image, and the texture seam line area is determined according to the expansion line used when the three-dimensional mesh model is texture expanded.

[0144] As an example, when a 3D mesh model is textured, the unwrapping line is on the back. Fig. 9 The left figure in the middle is a schematic diagram of the gap-like existence in the suture area of ​​the 3D model. Fig. 9 As shown in the right figure in the middle. The gap smoothing operation is to merge the texture seam line area in the second texture expansion image.

[0145] When fusing the texture seam line area in the second texture unfolded image, the back texture image of the bust can be determined based on the front image of the bust first; then the second texture unfolded image and the back texture image of the bust are weighted fused to obtain a fourth texture unfolded image.

[0146] Exemplarily, the second texture expansion image and the back texture image of the bust may be subjected to weighted fusion processing according to a set rule to obtain a fourth texture expansion image;

[0147] The setting rules can be:

[0148] I3(i,j)=αI1(i,j)+(1-α)I2(map1(i),map2(j));

[0149] α=I alpha (i,j);

[0150] Wherein, I1 represents the second texture expansion map, I2 represents the back image of the bust, I3 represents the fourth texture expansion map, I alpha represents the weight, map1 represents the mapping function of the pixel points in the back image of the bust to the second texture expansion map in the X-axis direction, map2 represents the mapping function of the pixel points in the back image of the bust to the second texture expansion map in the Y-axis direction, i represents the coordinate value of the pixel point in the X-axis direction, and j represents the coordinate value of the pixel point in the X-axis direction.

[0151] For example, see Fig.10As shown in FIG. 1 , the back texture image of the bust is determined according to the front image of the bust. Fig.10 As shown in (a), the weight of each pixel can be obtained by Fig.10 The second texture expansion image is shown in (b) of FIG. 10. The second texture expansion image ( Fig.10 (c)) and the back texture image of the bust ( Fig.10 (a)) According to Fig.10 (b) is fused to get Fig.10 (d) is the fourth texture expansion image. The fusion operation is Fig.10 It is indicated by a plus sign “+”.

[0152] The following is a detailed description of the ear optimization process. The half-length face includes ears, that is, the ears are not blocked by hair. Whether the ears are included can be determined based on the head semantic mask. When optimizing the ear part, the following methods can be used:

[0153] B1, first fuse the preconfigured ear model into the ear region in the three-dimensional mesh model to obtain a fused three-dimensional mesh model.

[0154] Exemplarily, the ear may be fused into the ear region of the three-dimensional mesh model by using a Laplacian mesh fusion method.

[0155] B2, the texture of the ear area on the fused three-dimensional network model is fused to the ear area located at the preset position of the fourth texture expansion map to obtain a fused fourth texture expansion map; finally, the three-dimensional model of the bust is obtained according to the fused fourth texture expansion map.

[0156] Exemplarily, the fusion method used to fuse the texture of the ear area on the fused three-dimensional network model to the ear area located at a preset position of the fourth texture expansion map may be a fusion algorithm based on the image Laplace gradient.

[0157] See also Fig.11 (a) is a schematic diagram of a three-dimensional mesh model. The ear region in the determined three-dimensional mesh model can be found in Fig.11 As shown in (b) in the figure, the geometric scale of the ear can be determined according to the size of the 3D mesh model. The size of the 3D mesh model can be preset by the user or a default size can be used. The ear model fits the ear area in the 3D mesh model, as shown in Fig.11 As shown in (c), the three-dimensional mesh model that fits the ear model is then fused, and the obtained fused three-dimensional mesh model can be seen in Fig.11 Then, the texture of the ear region on the fused three-dimensional network model is obtained, and the texture of the ear region is fused to the ear region at a preset position in the fourth texture expansion map.

[0158] As an example, for example, the image to be processed is Figure 2 The image shown in FIG. 1 shows how the reconstructed three-dimensional model is obtained through the solution provided in the embodiment of the present application. Fig.12 shown. Fig.12 (a) is a front view of the three-dimensional model. Fig.12 (b) is a schematic diagram of the back side of the three-dimensional model.

[0159] For example, the solution provided in the embodiment of the present application is applied to a virtual three-dimensional conversation. When the local terminal device (hereinafter referred to as the terminal) receives a trigger from the user and starts a virtual three-dimensional video call, the video stream is obtained through the terminal's camera, and the three-dimensional model can be reconstructed according to the three-dimensional reconstruction method of the half-length portrait based on a single-frame image proposed in this patent. The terminal drives the three-dimensional model through the expression of the user in each frame of the acquired video stream, and sends it to the opposite terminal. The opposite terminal displays the expression of the local terminal user simulated by the three-dimensional model. As an example, see Fig.13 As shown, the creation of the three-dimensional model can be performed by a computing cloud, where the electronic device sends a single frame image to the computing cloud, the computing cloud creates the three-dimensional model, and then sends the created three-dimensional model to the terminal.

[0160] In order to better illustrate the above embodiment, the present application embodiment further provides a device 1400, see Fig.14 As shown, the device 1400 may specifically include a functional module in an electronic device (such as Figure 1 The device 1400 may be a component of the processor 110 in the processor 110 or a software module executed by the processor 110, or the device 1400 may be a chip or a chip system, or the device 1400 may be a module in an electronic device. In an exemplary embodiment, the device may include an acquisition unit 1401 and a reconstruction unit 1402. The acquisition unit 1401 and the reconstruction unit 1402 respectively execute Figure 2 and Figure 4 The different steps of the method shown in the corresponding embodiment. For example, the acquisition unit 1401 can be used to execute S201 to acquire the image to be processed, and the reconstruction unit 1402 is used to execute the process of S202-S204. The specific implementation is as described above and will not be repeated here.

[0161] Therefore, it can be considered that the acquisition unit 1401 or the reconstruction unit 1402 mentioned above can be implemented by software, hardware, or a combination of software and hardware. When the module is implemented in hardware, the hardware can be any one or any combination of a CPU, a microprocessor, a DSP, a microcontroller unit (MCU), an artificial intelligence processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a dedicated digital circuit, a hardware accelerator, or a non-integrated discrete device, which can run the necessary software or not rely on the software to execute the above method flow, and is located in the previous Figure 1 When the module is implemented in software, the software exists in the form of computer program instructions and is stored in a memory, such as Figure 1 In the memory 120, the processor, for example Figure 1 The processor 110 in the embodiment can be used to execute the program instructions to implement the above method flow. The processor may include but is not limited to at least one of the following: various computing devices that run software, such as a CPU, a microprocessor, a DSP, a microcontroller, or an artificial intelligence processor, each computing device may include one or more cores for executing software instructions to perform operations or processing. The processor may be a separate semiconductor chip, or it may be integrated into a semiconductor chip together with other circuits. For example, it may form a SoC (system on chip) with other circuits (such as a codec circuit, a hardware acceleration circuit, or various buses and interface circuits), or it may be integrated into the ASIC as a built-in processor of an ASIC. The ASIC with the integrated processor may be packaged separately or with other circuits. In addition to the core for executing software instructions for operations or processing, the processor may further include necessary hardware accelerators, such as FPGAs, PLDs (programmable logic devices), or logic circuits for implementing dedicated logic operations.

[0162] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0163] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0164] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0166] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for three-dimensional reconstruction of a bust, characterized in that: include: Acquire an image to be processed, wherein the image to be processed includes a bust of a target person, wherein the bust includes a frontal face; Obtaining a first texture expansion map according to the image to be processed, wherein the first texture expansion map is used to represent the front texture of the bust; Wherein, at least two organs among the facial features of the bust in the first texture expansion image are located at preset positions; According to the front texture of the bust in the first texture expansion map, the back texture of the bust in the first texture expansion map is supplemented to obtain a second texture expansion map, wherein the second texture expansion map is used to represent the surface texture of the bust; A three-dimensional model of the bust is obtained according to the second texture development map.

2. The method according to claim 1, characterized in that Obtaining a first texture expansion map according to the image to be processed includes: Removing the background from the image to be processed to obtain a front image of the bust; Performing semantic segmentation on the front image to obtain a head semantic mask of the front image; Obtaining a three-dimensional mesh model of the bust according to the head semantic mask and the frontal image; A first texture expansion map is obtained according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, wherein the at least two organs in the first texture expansion map are located at preset positions.

3. The method according to claim 2, characterized in that The step of obtaining a first texture unfolding map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model includes: Performing texture expansion on the texture corresponding to the three-dimensional mesh model based on a central axis to obtain a third texture expansion image, wherein the central axis is a connecting line from the top end of the head to the bottom end of the head in the three-dimensional mesh model; Determining the positions of the at least two organs in the three-dimensional mesh model according to the positions of the at least two organs on the head in the head semantic mask; The third texture expansion map is adjusted according to the locations of the at least two organs in the three-dimensional mesh model to obtain the first texture expansion map.

4. The method according to claim 2 or 3, characterized in that Obtaining a three-dimensional model of the bust according to the second texture development map includes: The texture stitching line area in the second texture unfolding map is smoothed to obtain a fourth texture unfolding map, and the three-dimensional model of the bust is obtained based on the fourth texture unfolding map, wherein the texture stitching line area is determined according to the unfolding line used when the texture is unfolded on the three-dimensional mesh model.

5. The method according to claim 4, characterized in that The at least two organs include an ear; and the method further comprises: Fusing the preconfigured ear model into the ear region in the three-dimensional mesh model to obtain a fused three-dimensional mesh model; Obtaining a three-dimensional model of the bust based on the processed fourth texture development image includes: The texture of the ear area on the fused three-dimensional network model is fused to the ear area located at a preset position of the fourth texture expansion map to obtain a fused fourth texture expansion map, and the three-dimensional model of the bust is obtained according to the fused fourth texture expansion map.

6. The method according to claim 5, characterized in that Smoothing the texture stitching line area in the second texture expansion image includes: Determine a back texture image of the bust according to the front image of the bust; The second texture expansion map and the back texture image of the bust are subjected to weighted fusion processing to obtain the fourth texture expansion map.

7. The method according to claim 6, characterized in that The second texture expansion map and the back texture image of the bust are subjected to weighted fusion processing to obtain the fourth texture expansion map, including: Performing weighted fusion processing on the second texture expansion image and the back image of the bust according to a set rule to obtain the fourth texture expansion image; The setting rules are: I3(i,j)=αI1(i,j)+(1-α)I2(map1(i),map2(j)); α=I alpha (i,j); Wherein, I1 represents the second texture expansion map, I2 represents the back image of the bust, I3 represents the fourth texture expansion map, I alpha represents the weight, map1 represents the mapping function of mapping the pixel points in the back image of the bust to the second texture expansion map in the X-axis direction, map2 represents the mapping function of mapping the pixel points in the back image of the bust to the second texture expansion map in the Y-axis direction, i represents the coordinate value of the pixel point in the X-axis direction, and j represents the coordinate value of the pixel point in the X-axis direction.

8. The method according to claim 2, characterized in that Before obtaining the first texture unfolding map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, the method further includes: Perform at least one of the following processing on the three-dimensional mesh model: Hole filling, mesh homogenization or mesh smoothing.

9. A three-dimensional reconstruction device for a bust, characterized in that: include: An acquisition unit, used for acquiring an image to be processed, wherein the image to be processed includes a bust of a target person, wherein the bust includes a frontal face; A reconstruction unit is used to obtain a first texture expansion map according to the image to be processed, wherein the first texture expansion map is used to characterize the front texture of the bust; wherein at least two organs among the facial features of the bust in the first texture expansion map are located at preset positions; a second texture expansion map is obtained by supplementing the back texture of the bust in the first texture expansion map according to the front texture of the bust in the first texture expansion map, wherein the second texture expansion map is used to characterize the surface texture of the bust; and a three-dimensional model of the bust is obtained according to the second texture expansion map.

10. The device according to claim 9, characterized in that When the reconstruction unit obtains the first texture expansion map according to the image to be processed, it is specifically used to: Removing the background from the image to be processed to obtain a front image of the bust; Performing semantic segmentation on the front image to obtain a head semantic mask of the front image; Obtaining a three-dimensional mesh model of the bust according to the head semantic mask and the frontal image; A first texture expansion map is obtained according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, wherein the at least two organs in the first texture expansion map are located at preset positions.

11. The device according to claim 10, characterized in that The reconstruction unit, when obtaining the first texture unfolding map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, is specifically used to: Performing texture expansion on the texture corresponding to the three-dimensional mesh model based on a central axis to obtain a third texture expansion image, wherein the central axis is a connecting line from the top end of the head to the bottom end of the head in the three-dimensional mesh model; Determining the positions of the at least two organs in the three-dimensional mesh model according to the positions of the at least two organs on the head in the head semantic mask; The third texture expansion map is adjusted according to the locations of the at least two organs in the three-dimensional mesh model to obtain the first texture expansion map.

12. The device according to claim 10 or 11, characterized in that The reconstruction unit, when obtaining the three-dimensional model of the bust according to the second texture development map, is specifically used to: The texture stitching line area in the second texture unfolding map is smoothed to obtain a fourth texture unfolding map, and the three-dimensional model of the bust is obtained based on the fourth texture unfolding map, wherein the texture stitching line area is determined according to the unfolding line used when the texture is unfolded on the three-dimensional mesh model.

13. The device according to claim 12, characterized in that The at least two organs include an ear; the reconstruction unit is further used to fuse the preconfigured ear model into the ear region in the three-dimensional mesh model to obtain a fused three-dimensional mesh model; The reconstruction unit, when obtaining the three-dimensional model of the bust based on the processed fourth texture development map, is specifically used to: The texture of the ear area on the fused three-dimensional network model is fused to the ear area located at a preset position of the fourth texture expansion map to obtain a fused fourth texture expansion map, and the three-dimensional model of the bust is obtained according to the fused fourth texture expansion map.

14. The device according to claim 13, characterized in that The reconstruction unit, when smoothing the texture stitching line area in the second texture expansion image, is specifically used to: Determine a back texture image of the bust according to the front image of the bust; The second texture expansion map and the back texture image of the bust are subjected to weighted fusion processing to obtain the fourth texture expansion map.

15. The device according to claim 14, characterized in that The reconstruction unit, when performing weighted fusion processing on the second texture expansion map and the back texture image of the bust to obtain the fourth texture expansion map, is specifically used to: Performing weighted fusion processing on the second texture expansion image and the back image of the bust according to a set rule to obtain the fourth texture expansion image; The setting rules are: I3(i,j)=αI1(i,j)+(1-α)I2(map1(i),map2(j)); α=I alpha (i,j); Wherein, I1 represents the second texture expansion map, I2 represents the back image of the bust, I3 represents the fourth texture expansion map, I alpha represents the weight, map1 represents the mapping function of mapping the pixel points in the back image of the bust to the second texture expansion map in the X-axis direction, map2 represents the mapping function of mapping the pixel points in the back image of the bust to the second texture expansion map in the Y-axis direction, i represents the coordinate value of the pixel point in the X-axis direction, and j represents the coordinate value of the pixel point in the X-axis direction.

16. The device according to claim 15, characterized in that The reconstruction unit, before obtaining the first texture unfolding map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, is further used to: Perform at least one of the following processing on the three-dimensional mesh model: Hole filling, mesh homogenization or mesh smoothing.

17. An electronic device, characterized in that: It includes a processor and a memory; wherein the processor is coupled to the memory; The memory is used to store program instructions; The processor is used to read the program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed on an electronic device or a processor, the electronic device executes the method according to any one of claims 1 to 8.

19. A computer program product, characterized in that When the computer program product runs on an electronic device, the electronic device or processor executes the method according to any one of claims 1 to 8.

20. A chip, characterized in that: The chip is coupled to a memory in an electronic device, so that the electronic device executes the method according to any one of claims 1 to 8.

Images