A method for reconstructing a three-dimensional model of a person and an electronic device

Abstract

The embodiment of the application provides a kind of reconstruction method of character three-dimensional model and electronic equipment, it is related to three-dimensional model reconstruction field, can improve the efficiency of generating the three-dimensional model of target character.The method comprises: in the case where the target character wears first clothes, first global feature and first detail feature are extracted from the feature information of the target character;First global feature is used to reflect the global state of first clothes when the target character wears first clothes;First detail feature is used to reflect the detail state of first clothes when the target character wears first clothes;According to the first global feature, the preset garment feature map is modulated, and the first garment feature map is obtained;According to the first detail feature, the first garment feature map is modulated, and the feature map of the target character wearing target clothes is obtained;According to the feature map of target clothes, the three-dimensional model of the target character wearing target clothes is generated.

Description

Technical Field

[0001] This application relates to the field of three-dimensional model reconstruction, and more particularly to a method and electronic device for reconstructing a three-dimensional human figure model. Background Technology

[0002] With the rapid development of computer technology, the reconstruction of three-dimensional human models has become one of the important and hot topics in the field of computer science.

[0003] Traditional human body 3D model reconstruction involves: acquiring multiple images of the target person from different angles using multiple cameras or a single camera; generating multiple 3D point clouds related to the target person based on these multiple images; fusing these multiple 3D point clouds to obtain the target 3D point cloud of the target person; and finally generating a 3D model of the target person based on the target 3D point cloud.

[0004] However, the above-mentioned method for reconstructing a 3D human body model involves a large amount of computation in generating multiple 3D point clouds and fusing them, resulting in low efficiency in generating a 3D model of the target person. Summary of the Invention

[0005] This application provides a method and electronic device for reconstructing a 3D model of a person, which can improve the efficiency of generating a 3D model of a target person. To achieve the above objective, this application adopts the following technical solution:

[0006] In a first aspect, embodiments of this application provide a method for reconstructing a three-dimensional model of a person. The method includes: extracting a first global feature and a first detail feature from the feature information of the target person when the target person is wearing a first garment; the first global feature is used to reflect the global state of the first garment when the target person is wearing the first garment; the first detail feature is used to reflect the detailed state of the first garment when the target person is wearing the first garment; modulating a preset garment feature map according to the first global feature to obtain a first garment feature map; modulating the first garment feature map according to the first detail feature to obtain a feature map of the target person wearing the target garment; and generating a three-dimensional model of the target person wearing the target garment based on the feature map of the target garment.

[0007] The method for reconstructing a 3D model of a person provided in this application involves the following steps: First, when the target person is wearing a first garment, an electronic device extracts a first global feature and a first detailed feature from the target person's feature information. Second, the electronic device modulates a preset garment feature map based on the first global feature to obtain a first garment feature map. Third, the electronic device modulates the first garment feature map based on the first detailed feature to obtain a feature map of the target person wearing the target garment. Finally, the electronic device generates a 3D model of the target person wearing the target garment based on the feature map of the target garment. Thus, compared to existing methods for generating 3D models, the method for reconstructing a 3D model of a person provided in this application does not require generating a 3D point cloud map of the target person based on images of the target person from multiple angles, thereby reducing the computational load in the process of generating the 3D model of the target person and improving the efficiency of generating the target 3D model.

[0008] In one possible implementation, the method for reconstructing the aforementioned 3D character model further includes: the target clothing being a first garment.

[0009] In one possible implementation, the aforementioned first clothing feature map may include only the feature information of the first clothing or include the feature information of both the target person and the first clothing.

[0010] In one possible implementation, any of the above modulation processes includes at least one convolutional layer.

[0011] In one possible implementation, the aforementioned preset clothing feature map is generated through at least one convolutional layer.

[0012] In one possible implementation, the method for reconstructing the aforementioned 3D character model further includes: the target clothing being a second garment, which is the clothing the target character is to be dressed in; the modulation of the preset clothing feature map based on the first global feature to obtain the first clothing feature map includes: determining a second global feature in the feature library of the clothing to be dressed based on the first global feature; the second global feature being the global feature in the feature library of the clothing to be dressed that has the smallest error with the first global feature; and using the second global feature to modulate the preset clothing feature map to obtain the first clothing feature map.

[0013] The method for reconstructing a 3D model of a person provided in this application involves an electronic device determining a second global feature from a feature library of clothing to be changed based on a first global feature. Then, the electronic device modulates a preset clothing feature map using the second global feature to obtain a first clothing feature map. Subsequently, the electronic device modulates the detail features of the first clothing feature map based on a first detail feature to obtain a feature map of the target person wearing the clothing to be changed. Finally, the electronic device generates a 3D model of the target person wearing the target clothing based on the feature map of the clothing to be changed. In this way, the problem of changing clothes for the target person in the 3D model is solved.

[0014] In one possible implementation, the above-mentioned modulation of the preset clothing feature map based on the first global feature to obtain the first clothing feature map includes: adjusting the global features of the preset clothing feature map based on the first global feature to obtain the adjusted global features; and fusing the adjusted global features with the preset clothing feature map to obtain the first clothing feature map. The above-mentioned modulation of the first clothing feature map based on the first detail feature to obtain the feature map of the target person wearing the target clothing includes: adjusting the detail features of the first clothing feature map based on the first detail feature to obtain the adjusted detail features; and fusing the adjusted detail features with the first clothing feature map to obtain the feature map of the target person wearing the target clothing.

[0015] In one possible implementation, adjusting the global features of the preset clothing feature map based on the first global features to obtain the adjusted global features includes: adjusting the global features of the preset clothing feature map based on the first global features using the first global features to obtain the adjusted global features; adjusting the detail features of the first clothing feature map based on the first detail features to obtain the adjusted detail features includes: adjusting the detail features of the first clothing feature map based on the first detail features using the first detail features using the second style modulation network to obtain the adjusted detail features.

[0016] In one possible implementation, fusing the adjusted global features and the preset clothing feature map to obtain a first clothing feature map includes: fusing the adjusted global features and the preset clothing feature map based on a first fusion network to obtain the first clothing feature map; fusing the adjusted detail features and the first clothing feature map to obtain a feature map of the target person wearing the target clothing includes: fusing the adjusted detail features and the first clothing feature map based on a second fusion network to obtain a feature map of the target person wearing the target clothing.

[0017] In one possible implementation, the method for reconstructing the aforementioned 3D model of a person further includes: extracting first global features and first detailed features from the feature information of the target person based on a feature extraction network.

[0018] In one possible implementation, the method for reconstructing the 3D model of the person further includes: the feature extraction network, the first style modulation network, the second style modulation network, the first fusion network, and the second fusion network are trained based on a training sample set, which includes feature information of the target person and clothing feature maps of the person wearing clothing samples.

[0019] In one possible implementation, the method for reconstructing the 3D model of the person mentioned above further includes: determining the clothing consistency loss based on the feature map of the target person wearing the target clothing and the real feature map of the target person wearing the target clothing; and updating the first style modulation network and the second style modulation network based on the clothing consistency loss.

[0020] The method for reconstructing a 3D model of a person provided in this application embodiment involves an electronic device determining a second global feature from a feature library of clothing to be changed based on a first global feature. Then, the electronic device modulates a preset clothing feature map using the second global feature to obtain a first clothing feature map. Subsequently, the electronic device modulates the first clothing feature map based on a first detail feature to obtain a feature map of the target person wearing the clothing to be changed. Finally, the electronic device determines the clothing consistency loss based on the feature map of the target person wearing the target clothing and the actual feature map of the target person wearing the target clothing. Based on this clothing consistency loss, the electronic device updates a first style modulation network and a second style modulation network, so that the electronic device adjusts the global and detail features according to the updated first and second style modulation networks to generate a highly realistic 3D model of the target person after changing clothes.

[0021] In one possible implementation, the method for reconstructing the three-dimensional model of the person mentioned above further includes: extracting feature information of the target person from the target image, wherein the target image is an image of the target person wearing the first clothing mentioned above.

[0022] Secondly, the reconstruction apparatus for a three-dimensional model of a person provided in this application includes: an extraction module, a processing module, and a generation module; the extraction module is used to extract a first global feature and a first detail feature from the feature information of the target person when the target person is wearing a first garment; the first global feature is used to reflect the global state of the first garment when the target person is wearing the first garment; the first detail feature is used to reflect the detailed state of the first garment when the target person is wearing the first garment; the processing module is used to modulate a preset garment feature map according to the first global feature to obtain a first garment feature map; the processing module is also used to modulate the first garment feature map according to the first detail feature to obtain a feature map of the target person wearing the target garment; the generation module is also used to generate a three-dimensional model of the target person wearing the target garment according to the feature map of the target garment.

[0023] In one possible implementation, the reconstruction apparatus for a three-dimensional human figure provided in this application embodiment further includes: the target clothing being a first garment.

[0024] In one possible implementation, the aforementioned first clothing feature map may include only the feature information of the first clothing or include the feature information of both the target person and the first clothing.

[0025] In one possible implementation, any of the above modulation processes includes at least one convolutional layer.

[0026] In one possible implementation, the aforementioned preset clothing feature map is generated through at least one convolutional layer.

[0027] In one possible implementation, the reconstruction device for a three-dimensional human figure model provided in this application embodiment further includes: a determining module; the determining module is used to determine a second global feature in the feature library of the clothing to be changed based on a first global feature; the second global feature is the global feature in the feature library of the clothing to be changed that has the smallest error with the first global feature; the processing module is specifically used to modulate the preset clothing feature map using the second global feature to obtain a first clothing feature map.

[0028] In one possible implementation, the reconstructing apparatus for a 3D human figure provided in this application includes a processing module comprising: a modulation module and a fusion module; the modulation module is used to adjust the global features of a preset clothing feature map according to a first global feature to obtain the adjusted global features; the fusion module is used to fuse the adjusted global features and the preset clothing feature map to obtain the first clothing feature map. The modulation module is further used to adjust the detail features of the first clothing feature map according to the first detail features to obtain the adjusted detail features; the fusion module is further used to fuse the adjusted detail features and the first clothing feature map to obtain a feature map of the target person wearing the target clothing.

[0029] In one possible implementation, the reconstruction device for a 3D character model provided in this application embodiment further includes: a modulation module; the modulation module is specifically used to adjust the global features of a preset clothing feature map based on a first style modulation network using the aforementioned first global features to obtain adjusted global features; the modulation module is also used to adjust the detail features of the aforementioned first clothing feature map based on a second style modulation network using first detail features to obtain adjusted detail features.

[0030] In one possible implementation, the aforementioned fusion module is specifically used to fuse the adjusted global features and the aforementioned preset clothing feature map based on the first fusion network to obtain the first clothing feature map; the fusion module is also specifically used to fuse the adjusted detailed features and the aforementioned first clothing feature map based on the second fusion network to obtain the feature map of the target person wearing the aforementioned target clothing.

[0031] In one possible implementation, the extraction module is specifically used to extract the first global feature and the first detailed feature from the feature information of the target person based on the feature extraction network.

[0032] In one possible implementation, the reconstruction apparatus for a 3D model of a person provided in this application embodiment further includes: the feature extraction network, the first style modulation network, the second style modulation network, the first fusion network, and the second fusion network are trained based on a training sample set, which includes feature information of the target person and clothing feature maps of the person sample when wearing clothing samples.

[0033] In one possible implementation, the reconstruction apparatus for a 3D model of a person provided in this application embodiment further includes: an update module; a determination module for determining clothing consistency loss based on the feature map of the target person wearing the target clothing and the real feature map of the target person wearing the target clothing; and an update module for updating the first style modulation network and the second style modulation network based on the clothing consistency loss.

[0034] In one possible implementation, the extraction module is used to extract feature information of the target person from the target image, wherein the target image is an image of the target person wearing the first garment.

[0035] Thirdly, the electronic device provided in the embodiments of this application includes a memory and a processor, the memory being coupled to the processor; the memory is used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed by the processor, the electronic device performs the method described in the first aspect and any one of its possible implementations.

[0036] Fourthly, embodiments of this application provide a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any of its possible implementations.

[0037] Fifthly, an embodiment of this application provides a computer program product that, when run on a computer, causes the computer to perform the method described in the first aspect and any of its possible implementations.

[0038] It should be understood that the beneficial effects achieved by the second to fifth aspects of the technical solutions and the corresponding possible implementations of the embodiments of this application can be referred to the above-described technical effects of the first aspect and its corresponding possible implementations. Attached Figure Description

[0039] Figure 1 A schematic diagram of an electronic device structure provided in an embodiment of this application;

[0040] Figure 2 A schematic diagram of a method for reconstructing a 3D human figure model provided in this application embodiment. Figure 1 ;

[0041] Figure 3 A schematic diagram of a method for reconstructing a 3D human figure model provided in this application embodiment. Figure 2 ;

[0042] Figure 4 A schematic diagram of image segmentation provided in an embodiment of this application;

[0043] Figure 5 A schematic diagram of a method for reconstructing a 3D human figure model provided in this application embodiment. Figure 3 ;

[0044] Figure 6 A schematic diagram of a method for reconstructing a 3D human figure model provided in this application embodiment. Figure 4 ;

[0045] Figure 7 A schematic diagram of a method for reconstructing a 3D human figure model provided in this application embodiment. Figure 5 ;

[0046] Figure 8 A schematic diagram of a method for reconstructing a 3D human figure model provided in this application embodiment. Figure 6 ;

[0047] Figure 9 A schematic diagram of an electronic device provided in an embodiment of this application. Figure 1 ;

[0048] Figure 10 A schematic diagram of an electronic device provided in an embodiment of this application. Figure 2 . Detailed Implementation

[0049] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0050] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first garment" and "second garment" are used to distinguish different garments, not to describe a specific order of garments.

[0051] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0052] First, we will explain the method for reconstructing a three-dimensional human figure model and some concepts involved in the electronic device provided in the embodiments of this application.

[0053] (1) Three-dimensional model reconstruction: refers to the process of reconstructing a three-dimensional model based on a single or multiple two-dimensional images.

[0054] (2) Adaptive Instance Normalization (AdaIN) formula:

[0055] (y)(x-μ(x) / (x))+μ(y)

[0056] Where x and y are the feature maps of the content image and style image, respectively. and μ are the mean and standard deviation, respectively. This formula brings the mean and standard deviation of the content image closer to the mean and standard deviation of the style image.

[0057] With the development of computer technology, the application of human body 3D reconstruction technology in fields such as 3D printing, entertainment, and remote augmented reality (AR) calls is becoming increasingly widespread.

[0058] Existing 3D human body reconstruction technology reconstructs a 3D model of a target person based on multiple images from different angles. One implementation involves: acquiring M images of the target person from different angles using multiple cameras or a single camera, where M is an integer greater than or equal to 2; generating N 3D point clouds of the target person based on these M images, where N is an integer greater than or equal to 1 and less than or equal to M; fusing these N 3D point clouds to obtain the target 3D point cloud of the target person; and obtaining a 3D mesh model of the target person based on this target 3D point cloud. Finally, obtaining a texture map of the target person based on this 3D mesh model, and combining this texture map with the surface mesh of the 3D mesh model to obtain the 3D model of the target person.

[0059] However, the above-mentioned method for reconstructing a 3D human body model involves a large amount of computation in generating multiple 3D point clouds and fusing them, resulting in low efficiency in generating a 3D model of the target person.

[0060] Addressing the issue of low efficiency in generating 3D models of target individuals using existing 3D reconstruction technologies, this application provides a method and electronic device for reconstructing 3D models of individuals. Specifically, the method includes: when the target individual is wearing a first garment, the electronic device first extracts a first global feature and a first detail feature from the target individual's feature information; the first global feature reflects the global state of the first garment when the target individual is wearing it; the first detail feature reflects the detailed state of the first garment when the target individual is wearing it; secondly, the electronic device modulates a preset garment feature map based on the first global feature to obtain a first garment feature map; then, the electronic device modulates the first garment feature map based on the first detail feature to obtain a feature map of the target individual wearing the target garment; finally, the electronic device generates a 3D model of the target individual wearing the target garment based on the feature map of the target garment. The technical solution provided by this application improves the efficiency of generating 3D models of target individuals.

[0061] It should be noted that the method for reconstructing a 3D human model provided in this application embodiment can be applied to scenarios such as virtual try-on, virtual 3D communication, 3D printing, and AR / VR (augmented reality and virtual reality).

[0062] For example, the electronic device in the embodiments of this application may be a tablet computer, mobile phone, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), etc. The embodiments of this application do not impose special restrictions on the specific form of the electronic device.

[0063] The execution subject of the method for reconstructing a 3D human figure model provided in this application embodiment can be a device that supports the reconstruction of a 3D human figure model. This execution device can be one of the following: Figure 1 The electronic device shown is an example of a device that can also be the central processing unit (CPU) of the electronic device. This application uses an electronic device to perform a method for reconstructing a 3D human figure model as an example to illustrate the method provided in this application for reconstructing a 3D human figure model.

[0064] Please refer to Figure 1 In this application embodiment, an electronic device is used as an example. Figure 1 Taking the mobile phone 100 shown as an example, the electronic device provided in this application embodiment will be described. Among them, Figure 1 The mobile phone 100 shown is merely an example of an electronic device, and the mobile phone 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. Figure 1 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0065] like Figure 1 As shown, the mobile phone 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0066] The aforementioned sensor module 180 may include sensors such as pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, and bone conduction sensors.

[0067] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the mobile phone 100. In other embodiments, the mobile phone 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0068] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0069] The controller can be the nerve center and command center of the mobile phone 100. The controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0070] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0071] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0072] It is understood that the interface connection relationships between the modules illustrated in this embodiment are merely illustrative and do not constitute a structural limitation on the mobile phone 100. In other embodiments, the mobile phone 100 may also adopt different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0073] The charging management module 140 receives charging input from a charger, which can be a wireless charger or a wired charger. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0074] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc. In some embodiments, the power management module 141 and the charging management module 140 may also be housed in the same device.

[0075] The wireless communication function of mobile phone 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. In some embodiments, antenna 1 of mobile phone 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling mobile phone 100 to communicate with networks and other devices through wireless communication technology.

[0076] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in mobile phone 100 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0077] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on the mobile phone 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation.

[0078] The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 can be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be housed in the same device.

[0079] The wireless communication module 160 can provide solutions for wireless communication applications on the mobile phone 100, including wireless local area networks (WLAN) (such as wireless fidelity, Wi-Fi), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. For example, in this embodiment, the mobile phone 100 can access a Wi-Fi network through the wireless communication module 160.

[0080] The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0081] The mobile phone 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0082] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. For example, in this embodiment, the display screen 194 can be used to display the application interface of the first application described above, such as a device sharing interface, a device search interface, and a QR code scanning interface.

[0083] Mobile phone 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194, and application processor. The ISP is used to process data fed back by camera 193. Camera 193 is used to capture still images or videos. In some embodiments, mobile phone 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0084] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0085] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of mobile phone 100 by running the instructions stored in internal memory 121. For example, in this embodiment, processor 110 can execute instructions stored in internal memory 121, which may include a program storage area and a data storage area.

[0086] The program storage area can store the operating system, at least one application required for a function (such as sound playback, image playback, etc.). The data storage area can store data created during the use of the mobile phone 100 (such as audio data, phonebook, etc.). In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0087] The mobile phone 100 can achieve audio functions such as music playback and recording through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 270D, and application processor.

[0088] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the mobile phone 100. The mobile phone 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0089] although Figure 1 As not shown, the mobile phone 100 may also have a flash, a miniature projection device, a near field communication (NFC) device, etc., which will not be described in detail here.

[0090] The methods described in the following embodiments can all be implemented in an electronic device having the above-described hardware structure. The electronic device described above is used as an example in the following embodiments. Figure 1 Taking the mobile phone 100 shown as an example, the method for reconstructing a three-dimensional human figure model according to an embodiment of this application will be described.

[0091] The method for reconstructing a 3D model of a person provided in this application is used to generate a 3D model of a target person wearing a certain garment. Specifically, it includes two application scenarios: the first scenario is to generate a 3D model of the target person wearing the first garment based on an image of the target person wearing the first garment; the second scenario is to generate a 3D model of the target person wearing the second garment based on an image of the target person wearing the first garment, that is, in the process of generating the 3D model of the target person wearing the second garment, the target person's clothing is changed.

[0092] This application provides a method for reconstructing a 3D human figure model, such as... Figure 2 As shown, the reconstruction method of the three-dimensional model of the figure may include S201-S204.

[0093] S201. When the target person is wearing the first type of clothing, the electronic device extracts the first global feature and the first detailed feature from the feature information of the target person.

[0094] The aforementioned feature information of the target person is extracted by the electronic device from a single target image. The extraction process of the target person's feature information specifically includes: the electronic device acquiring (through the camera of the electronic device or from other storage media) a target image, which is an image of the target person wearing the first garment; the electronic device inputting the target image into an image segmentation model, which outputs a target person image and a background image; wherein, the target person image includes the target person wearing the first garment, and the background image contains other content in the target image besides the target person; the electronic device inputting the target person image into a preset feature extraction model to obtain the feature information of the target person.

[0095] It should be noted that the above image segmentation model is trained based on training samples and their corresponding annotations. The training samples are images of humans and backgrounds taken naturally in outdoor and indoor scenes. The annotations for the training samples are that the values ​​of the regions containing the human body are set to 1, and the values ​​of the regions containing the background are set to 0. The image segmentation model segments the training samples into human body images and background images based on the different values ​​of different regions in the training samples.

[0096] For example, such as Figure 4 As shown in (a), the target image includes a target person (a woman) and a background (birds, mountains, and the sun). The electronic device inputs this target image into an image segmentation model to obtain... Figure 4 The target person image shown in (b) and Figure 4 The background image shown in (c) is used. Then, the electronic device inputs the target person image into the preset feature extraction model to extract all features in the target person image and obtain the feature information of the target person.

[0097] The aforementioned first global feature is the feature information of the target person that reflects the global state of the first garment when the target person wears it. The first global feature may include at least one of the following: type, style, material, and size of the first garment. For example, when the target person wears a jacket, the first global feature includes: the looseness of the jacket on the target person and the size of the jacket on the target person.

[0098] The aforementioned first detailed feature is a feature information of the target person that reflects the detailed state of the first garment when the target person wears it. The first detailed feature includes at least one of the following: wrinkle information of various parts of the first garment and deformation information of various parts of the first garment. For example, when the target person wears a jacket, the first detailed feature includes: wrinkle information at the cuffs of the jacket, deformation of the waist of the jacket, and scaling information at the collar of the jacket when the jacket is worn by the target person.

[0099] Optional, combined Figure 2 ,like Figure 3 As shown, in one implementation, the extraction of the first global feature and the first detailed feature from the feature information of the target person (i.e., S201) can be achieved through S301.

[0100] S301. The electronic device is based on a feature extraction network to extract the first global feature and the first detailed feature from the feature information of the target person.

[0101] The aforementioned feature extraction network can consist of a first fully connected layer and at least one second fully connected layer. The first fully connected layer extracts a first global feature from the feature information of the target person, and the second fully connected layer extracts a first detailed feature from the feature information of the target person. The feature information of the target person is input into the feature extraction network, and the feature extraction network outputs the first global feature and the first detailed feature.

[0102] S202. The electronic device modulates the preset clothing feature map according to the first global feature to obtain the first clothing feature map.

[0103] The aforementioned preset clothing feature map is a clothing feature map randomly generated by an electronic device. The preset clothing feature map includes the three-dimensional features of the preset clothing. That is, the preset clothing feature map includes the front, side, and back features of the preset clothing. The three-dimensional features include global features and detailed features. For example, the preset clothing feature map randomly generated by the electronic device is a jacket clothing feature map, which includes all the features of the front, back, and sides of the jacket.

[0104] It should be noted that the above-mentioned electronic device modulates the preset clothing feature map according to the first global feature, specifically including: adjusting the global features of the preset clothing feature map according to the first global feature to obtain the adjusted global features; and then replacing the global features in the preset clothing feature map with the adjusted global features.

[0105] Optional, combined Figure 2 or Figure 3 ,like Figure 5 As shown, in one implementation, the above S202 can be implemented by S501-S502.

[0106] S501. The electronic device adjusts the global features of the preset clothing feature map according to the first global feature to obtain the adjusted global features.

[0107] The above S501 includes: adjusting the global features of a preset clothing feature map using the first global features based on the first style modulation network to obtain the adjusted global features; specifically, the first global features and the preset clothing feature map are input into the first style modulation network, and the first style modulation network outputs the adjusted global features.

[0108] It should be noted that the above-mentioned first style modulation network is based on AdaIN. The first garment is used as the style feature map, the preset garment feature map is used as the content feature map, and a certain feature in the first global feature (such as the size of the garment) and the variable used to describe the preset garment feature map are used as parameters. The AdaIN formula is used to adjust the global features of the preset garment to be closer to the first global feature.

[0109] S502, The electronic device merges the adjusted global features and the preset clothing feature map to obtain the first clothing feature map.

[0110] The above-mentioned S502 includes: based on the first fusion network, fusing the adjusted global features and the preset clothing feature map to obtain the first clothing feature map.

[0111] The above steps include: based on the first fusion network, fusing the adjusted global features and the first clothing feature map to obtain a feature map of the target person wearing the target clothing, wherein the first fusion network may be composed of at least one convolutional layer, which replaces the corresponding global features in the preset clothing feature map with the adjusted global features.

[0112] It should be noted that the above-mentioned preset clothing feature map includes the three-dimensional features of the preset clothing. Therefore, after fusing the adjusted global features and the preset clothing feature map, the first clothing feature map obtained is also a three-dimensional clothing feature map.

[0113] S203. The electronic device modulates the first clothing feature map according to the first detailed feature to obtain the feature map of the target person wearing the target clothing.

[0114] The target garment mentioned above is the first garment mentioned above.

[0115] Optional, combined Figure 2 , Figure 3 or Figure 5 ,like Figure 6 As shown, in one implementation, the above S203 can be implemented by S601-S602.

[0116] S601. The electronic device adjusts the detailed features of the first garment feature map according to the first detailed features to obtain the adjusted detailed features.

[0117] The above steps include: adjusting the detail features of the first clothing feature map using the first detail features based on the second style modulation network to obtain the adjusted detail features. Specifically, the first detail features and the preset clothing feature map are input into the second style modulation network, and the second style modulation network outputs the adjusted detail features.

[0118] It should be noted that the implementation of the second style modulation network is similar to that of the first style modulation network. For details, please refer to the relevant description of the first style modulation network in S501, which will not be repeated here.

[0119] S602, The electronic device merges the adjusted detailed features with the first clothing feature map to obtain a feature map of the target person wearing the target clothing.

[0120] The above steps include: based on the second fusion network, fusing the adjusted detail features and the first clothing feature map to obtain a feature map of the target person wearing the target clothing. The second fusion network can be composed of multiple convolutional layers, which replace the corresponding detail features in the first clothing feature map with the adjusted detail features.

[0121] It should be noted that the feature map of the target person wearing the target clothing obtained above includes the target person's human body features. These human body features can be transmitted to the fusion network along with global features or along with detailed features. When the target person's human body features are transmitted to the fusion network along with global features, the first clothing feature map above includes the target person's human body features. When the target person's human body features are transmitted to the fusion network along with detailed features, the feature map of the target person wearing the target clothing above includes the target person's human body features.

[0122] S204. The electronic device generates a 3D model of the target person wearing the target clothing based on the feature map of the target clothing.

[0123] The electronic device described above can generate a 3D model of a target person wearing the target clothing based on the feature map of the target clothing. This model can be generated based on Poisson reconstruction or other methods. In this application, the specific method of generating the 3D model of a target person wearing the target clothing is not limited.

[0124] It should be noted that the aforementioned feature extraction network, first style modulation network, second style modulation network, first fusion network, and second fusion network are trained based on a training sample set, which includes feature information of the target person and clothing feature maps of the person wearing clothing samples.

[0125] The aforementioned feature extraction network, first style modulation network, second style modulation network, first fusion network, and second fusion network together constitute the feature map generation network. The specific training process of this feature map generation network is as follows:

[0126] The electronic device inputs the feature information of the target person into the feature map generation network, and the feature map generation network outputs a feature map. Next, the electronic device generates an adversarial network based on the feature map output by the feature map generation network. Then, the electronic device calculates the adversarial loss based on the feature map output by the feature map generation network and the feature map in the adversarial network. When the adversarial loss meets a preset condition, the feature map output by the feature map generation network is a feature map that meets the condition. When the adversarial loss does not meet the preset condition, the electronic device updates the feature map generation network until the feature map generation network outputs a feature map that meets the condition.

[0127] The method for reconstructing a 3D model of a person provided in this application involves the following steps: First, when the target person is wearing a first garment, an electronic device extracts a first global feature and a first detailed feature from the target person's feature information. Second, the electronic device modulates a preset garment feature map based on the first global feature to obtain a first garment feature map. Third, the electronic device modulates the first garment feature map based on the first detailed feature to obtain a feature map of the target person wearing the target garment. Finally, the electronic device generates a 3D model of the target person wearing the target garment based on the feature map of the target garment. Thus, compared to existing methods for generating 3D models, the method for reconstructing a 3D model of a person provided in this application does not require generating a 3D point cloud map of the target person based on images of the target person from multiple angles, thereby reducing the computational load in the process of generating the 3D model of the target person and improving the efficiency of generating the target 3D model.

[0128] It should be noted that the method and electronic device for reconstructing the three-dimensional model of a person in this application embodiment can be applied in a virtual fitting scene (i.e., the second application scenario).

[0129] Optionally, when the method for reconstructing the 3D model of a person and the electronic device of this application are applied in a virtual fitting scene, such as Figure 7 As shown, the method for reconstructing a three-dimensional human figure model according to an embodiment of this application includes: S701-S705.

[0130] S701. When the target person is wearing the first clothing, the electronic device extracts the first global feature and the first detailed feature from the feature information of the target person.

[0131] It should be noted that the implementation of S701 is the same as that of S201. For a detailed description of S701, please refer to the relevant description of S201 above. It will not be repeated here.

[0132] S702. The electronic device determines the second global feature from the feature library of the garment to be changed based on the first global feature.

[0133] The second global feature mentioned above is the global feature with the smallest error between it and the first global feature in the feature library of the garment to be changed. The feature library of the garment to be changed is pre-configured and includes a variety of different global features of the garment to be changed.

[0134] For example, suppose the first global feature is the size format of the first garment XL, and the garment to be changed is a jacket; at this time, the electronic device filters out the sweater with the size format XL from the feature library of the garment to be changed based on the size format XL of the first garment.

[0135] S703. The electronic device modulates the preset clothing feature map using the second global feature to obtain the first clothing feature map.

[0136] The above steps specifically include S1-S2:

[0137] S1: The electronic device adjusts the global features of the preset clothing feature map according to the second global feature to obtain the adjusted global features.

[0138] It should be noted that the implementation of S1 is similar to that of S501. For a detailed description of S1, please refer to the relevant description of S501 above. It will not be repeated here.

[0139] S2: The electronic device fuses the adjusted global features and the preset clothing feature map to obtain the first clothing feature map.

[0140] It should be noted that the implementation of S2 is similar to that of S502. For a detailed description of S2, please refer to the relevant description of S502 above. It will not be repeated here.

[0141] S704. The electronic device modulates the first clothing feature map according to the first detailed feature to obtain a feature map of the target person wearing the target clothing.

[0142] It should be noted that the target clothing mentioned above is the clothing to be changed (i.e., the second clothing).

[0143] It should be noted that the implementation of S704 is similar to that of S203. For a detailed description of S704, please refer to the relevant description of S203 above. It will not be repeated here.

[0144] S705: The electronic device generates a 3D model of the target person wearing the target clothing based on the feature map of the target clothing.

[0145] It should be noted that the implementation of S705 is similar to that of S204. For a detailed description of S7054, please refer to the relevant description of S204 above. It will not be repeated here.

[0146] The method for reconstructing a 3D model of a person provided in this application involves an electronic device determining a second global feature from a feature library of clothing to be changed based on a first global feature. Then, the electronic device modulates the global features of a preset clothing feature map using the second global feature to obtain a first clothing feature map. Subsequently, the electronic device modulates the detail features of the first clothing feature map based on a first detail feature to obtain a feature map of the target person wearing the clothing to be changed. Finally, the electronic device generates a 3D model of the target person wearing the target clothing based on the feature map of the clothing to be changed. Thus, by using the second global feature and the first detail feature to modulate the preset clothing feature map, a feature map of the target person wearing the clothing to be changed is obtained, and then a 3D model of the target person wearing the target clothing is generated based on the feature map of the target person wearing the target clothing, thereby addressing the issue of changing the target person's clothes in the 3D model.

[0147] It should be noted that, based on, Figure 7 The method shown re-dresses the target character in the 3D model, but the resulting 3D model has a lot of noise, leading to poor realism. Therefore, as... Figure 8 As shown, the method for reconstructing a three-dimensional human figure model provided in this application embodiment further includes S801-S802 after S704.

[0148] S801, the electronic device determines the clothing consistency loss based on the feature map of the target person wearing the target clothing and the actual feature map of the target person wearing the target clothing.

[0149] The target clothing mentioned above is the second clothing mentioned above (i.e., the clothing that the target person is about to change into).

[0150] It should be noted that the aforementioned electronic device calculates the clothing consistency loss by comparing the feature map of the target person wearing the target clothing with the actual feature map of the target person wearing the target clothing. This clothing consistency loss is used to indicate the differences in global and detailed features between the feature map of the target person wearing the target clothing and the actual feature map of the target person wearing the target clothing.

[0151] Optionally, the above-mentioned clothing consistency loss can be: the electronic device inputs the feature map of the target person wearing the target clothing into the discrimination model, and the discrimination model outputs the discrimination result, which is used to indicate the consistency of clothing. Here, the clothing consistency loss is the negative value of clothing consistency; the consistency of clothing is represented by any number between 0 and 1; when the consistency of clothing is closer to 1, it indicates that the realism of the feature map of the target clothing is higher, and when the consistency of clothing is closer to 0, it indicates that the realism of the feature map of the target clothing is lower.

[0152] It should be noted that the above discrimination model is a pre-trained model. In the discrimination model, the feature map of the input target person wearing the target clothing is compared with the real feature map of the target person wearing the target clothing in the database to calculate the clothing consistency.

[0153] When the clothing consistency loss is less than or equal to a preset threshold, the electronic device generates a 3D model of the target person wearing the target clothing based on the feature map of the target clothing; when the clothing consistency loss is greater than the preset threshold, the electronic device executes the following S802.

[0154] S802, the electronic device updates the first style modulation network and the second style modulation network according to the clothing consistency loss.

[0155] The aforementioned updates to the first and second style modulation networks refer to adjusting the weights of the mean and variance of the style feature maps in AdaIn.

[0156] It should be noted that after executing S802, the electronic device re-executes S701-S704 so that the electronic device modulates the global and detailed features according to the updated first style modulation network and second style modulation network, thereby generating a highly realistic 3D model of the target character after the costume change.

[0157] The method for reconstructing a 3D model of a person provided in this application embodiment involves an electronic device determining a second global feature from a feature library of clothing to be changed based on a first global feature. Then, the electronic device modulates the global features of a preset clothing feature map using the second global feature to obtain a first clothing feature map. Subsequently, the electronic device modulates the detail features of the first clothing feature map based on a first detail feature to obtain a feature map of the target person wearing the clothing to be changed. Finally, the electronic device determines the clothing consistency loss based on the feature map of the target person wearing the target clothing and the actual feature map of the target person wearing the target clothing. Based on this clothing consistency loss, the electronic device updates a first style modulation network and a second style modulation network, so that the electronic device adjusts the global and detail features according to the updated first and second style modulation networks to generate a highly realistic 3D model of the target person after changing clothes.

[0158] Accordingly, this application provides a device for reconstructing a 3D human figure model. This device is used to execute the various steps in the above-described method for reconstructing a 3D human figure model. Based on the above method examples, this application can divide the device into functional modules. For example, each function can be divided into its own module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. The module division in this application is illustrative and represents only a logical functional division; other division methods may be used in actual implementation.

[0159] When dividing each function into modules according to its corresponding function. Figure 9 This diagram illustrates a possible structure of an electronic device for reconstructing a 3D human figure model as described in the above embodiments. Figure 9 As shown, the electronic device includes: an extraction module 901, a processing module 902, and a generation module 903.

[0160] The extraction module 901 is used to extract first global features and first detailed features from the feature information of the target person when the target person is wearing the first clothing, for example, by performing step S201 in the above method embodiment.

[0161] The processing module 902 is used to modulate the preset clothing feature map according to the first global feature to obtain the first clothing feature map; and is used to modulate the first clothing feature map according to the first detail feature to obtain the feature map of the target person wearing the target clothing, for example, by executing steps S202-S203 in the above method embodiment.

[0162] The generation module 903 is used to generate a three-dimensional model of a target person wearing the target clothing based on the feature map of the target clothing, for example, by performing step S204 in the above method embodiment.

[0163] Optionally, this application embodiment provides an electronic device, which further includes: a determination module 904.

[0164] The determining module 904 is used to determine the second global feature in the feature library of the garment to be changed based on the first global feature, for example, by performing step S702 in the above method embodiment.

[0165] The aforementioned processing module 902 is used to modulate the preset clothing feature map using the second global feature to obtain the first clothing feature map, for example, by executing step S703 in the above method embodiment.

[0166] Optionally, this application embodiment provides an electronic device, wherein the above-mentioned processing module 902 includes a modulation module 905 and a fusion module 906.

[0167] The modulation module 905 is used to adjust the global features of the preset clothing feature map according to the first global features to obtain the adjusted global features, and is used to adjust the detailed features of the first clothing feature map according to the first detailed features to obtain the adjusted detailed features, for example, by executing steps S501 and S601 in the above method embodiment.

[0168] The fusion module 906 is used to fuse the adjusted global features and the preset clothing feature map to obtain the first clothing feature map, and to fuse the adjusted detail features and the first clothing feature map to obtain the feature map of the target person wearing the target clothing, for example, by executing steps S502 and S602 in the above method embodiment.

[0169] Optionally, this application embodiment provides an electronic device, which further includes: an update module 907.

[0170] The aforementioned determining module 904 is used to determine the clothing consistency loss based on the feature map of the target person wearing the target clothing and the actual feature map of the target person wearing the target clothing, for example, by executing step S801 in the above method embodiment.

[0171] The update module 907 is used to update the first style modulation network and the second style modulation network according to the clothing consistency loss, for example, by performing step S802 in the above method embodiment.

[0172] The various modules of the above-mentioned electronic device can also be used to perform other actions in the above-mentioned method embodiments. All relevant content of each step involved in the above-mentioned method embodiments can be referred to in the functional description of the corresponding functional module, and will not be repeated here.

[0173] When using integrated units, the structural schematic diagram of the electronic device provided in the embodiments of this application is as follows: Figure 10 As shown. In Figure 10 The electronic device includes a processing module 1001 and a communication module 1002. The processing module 1001 controls and manages the operation of the electronic device, for example, executing the steps performed by the extraction module 901, processing module 902, generation module 903, determination module 904, modulation module 905, fusion module 906, and update module 907, and / or other processes for performing the techniques described herein. The communication module 1002 supports interaction between the electronic device and other devices, etc. Figure 10 As shown, the electronic device may also include a storage module 1003, which is used to store the characteristic information of the program code character of the electronic device.

[0174] The processing module 1001 can be a processor or a controller, for example... Figure 1 The processor 110 is located in the middle. The communication module 1002 can be a transceiver, RF circuit, or communication interface, for example... Figure 1 The mobile communication module 150 and / or wireless communication module 160 are included. The storage module 1003 may be a memory, such as... Figure 1 Internal memory 121.

[0175] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0176] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0177] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0178] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0180] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0181] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for reconstructing a 3D human figure model, characterized in that, include: When the target person is wearing the first type of clothing, extract the first global feature and the first detailed feature from the feature information of the target person; The first global feature is used to reflect the global state of the first garment when the target person wears the first garment; the first detailed feature is used to reflect the detailed state of the first garment when the target person wears the first garment. The preset clothing feature map is modulated based on the first global feature to obtain the first clothing feature map; Modulate the first clothing feature map based on the first detailed feature to obtain a feature map of the target person wearing the second clothing; wherein, the second clothing is the clothing that the target person is about to change into; A 3D model of the target person wearing the second garment is generated based on the feature map of the second garment; The step of modulating the preset clothing feature map based on the first global feature to obtain the first clothing feature map includes: Based on the first global feature, a second global feature is determined in the feature library of the garment to be changed; the second global feature is the global feature in the feature library of the garment to be changed that has the smallest error with the first global feature. The preset clothing feature map is modulated using the second global feature to obtain the first clothing feature map.

2. The method according to claim 1, characterized in that, The step of modulating the preset clothing feature map based on the first global feature to obtain the first clothing feature map includes: The global features of the preset clothing feature map are adjusted based on the first global feature to obtain the adjusted global features; The adjusted global features and the preset clothing feature map are fused to obtain the first clothing feature map; The step of modulating the first clothing feature map based on the first detailed features to obtain a feature map of the target person wearing the second clothing includes: Adjust the detailed features of the first garment feature map according to the first detailed features to obtain the adjusted detailed features; The adjusted detailed features and the first clothing feature map are fused together to obtain the feature map of the target person wearing the second clothing.

3. The method according to claim 2, characterized in that, The step of adjusting the global features of the preset clothing feature map based on the first global feature to obtain the adjusted global features includes: Based on the first style modulation network, the global features of the preset clothing feature map are adjusted using the first global features to obtain the adjusted global features; The step of adjusting the detailed features of the first garment feature map based on the first detailed features to obtain the adjusted detailed features includes: Based on the second style modulation network, the first detail features are used to adjust the detail features of the first clothing feature map to obtain the adjusted detail features.

4. The method according to claim 3, characterized in that, The step of fusing the adjusted global features and the preset clothing feature map to obtain the first clothing feature map includes: Based on the first fusion network, the adjusted global features and the preset clothing feature map are fused to obtain the first clothing feature map; The step of fusing the adjusted detailed features with the first clothing feature map to obtain a feature map of the target person wearing the second clothing includes: Based on the second fusion network, the adjusted detailed features and the first clothing feature map are fused to obtain the feature map of the target person wearing the second clothing.

5. The method according to claim 4, characterized in that, The extraction of the first global feature and the first detailed feature from the feature information of the target person includes: Based on the feature extraction network, the first global feature and the first detailed feature are extracted from the feature information of the target person.

6. The method according to claim 5, characterized in that, The method further includes: The feature extraction network, the first style modulation network, the second style modulation network, the first fusion network, and the second fusion network are trained based on a training sample set, which includes feature information of the target person and clothing feature maps of the person wearing clothing samples.

7. The method according to any one of claims 3 to 6, characterized in that, The method further includes: Based on the feature map of the target person wearing the second clothing and the actual feature map of the target person wearing the second clothing, determine the clothing consistency loss; The first style modulation network and the second style modulation network are updated based on the clothing consistency loss.

8. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Extract feature information of the target person from the target image, wherein the target image is an image of the target person wearing the first garment.

9. A device for reconstructing a three-dimensional human figure model, characterized in that, include: Extraction module, processing module, generation module, and determination module; The extraction module is used to extract a first global feature and a first detailed feature from the feature information of the target person when the target person is wearing the first garment; the first global feature is used to reflect the global state of the first garment when the target person is wearing the first garment; the first detailed feature is used to reflect the detailed state of the first garment when the target person is wearing the first garment. The processing module is configured to modulate a preset clothing feature map based on the first global feature to obtain a first clothing feature map; and to modulate the first clothing feature map based on the first detail feature to obtain a feature map of the target person wearing a second garment; wherein the second garment is the garment to be changed into by the target person. The generation module is used to generate a three-dimensional model of the target person wearing the second garment based on the feature map of the second garment; The determining module is used to determine a second global feature in the feature library of the garment to be changed based on the first global feature; the second global feature is the global feature in the feature library of the garment to be changed that has the smallest error with the first global feature; The processing module is specifically used to modulate the preset clothing feature map using the second global feature to obtain the first clothing feature map.

10. The apparatus according to claim 9, characterized in that, The processing module includes: a modulation module and a fusion module; The modulation module is used to adjust the global features of the preset clothing feature map according to the first global features to obtain the adjusted global features; The fusion module is used to fuse the adjusted global features and the preset clothing feature map to obtain the first clothing feature map; The modulation module is used to adjust the detailed features of the first garment feature map according to the first detailed features to obtain the adjusted detailed features. The fusion module is further configured to fuse the adjusted detail features with the first clothing feature map to obtain a feature map of the target person wearing the second clothing.

11. The apparatus according to claim 10, characterized in that, The modulation module is specifically used to adjust the global features of a preset clothing feature map based on the first style modulation network using the first global features to obtain the adjusted global features; and to adjust the detail features of the first clothing feature map based on the second style modulation network using the first detail features to obtain the adjusted detail features.

12. The apparatus according to claim 11, characterized in that, The fusion module is specifically used to fuse the adjusted global features and the preset clothing feature map based on the first fusion network to obtain the first clothing feature map; and specifically used to fuse the adjusted detailed features and the first clothing feature map based on the second fusion network to obtain the feature map of the target person wearing the second clothing.

13. The apparatus according to claim 12, characterized in that, The extraction module is specifically used to extract the first global feature and the first detailed feature from the feature information of the target person based on the feature extraction network.

14. The apparatus according to claim 13, characterized in that, The feature extraction network, the first style modulation network, the second style modulation network, the first fusion network, and the second fusion network are trained based on a training sample set, which includes feature information of the target person and clothing feature maps of the person wearing clothing samples.

15. The apparatus according to any one of claims 11 to 14, characterized in that, The reconstruction device for the three-dimensional model of the person also includes: an update module; The determining module is further configured to determine the clothing consistency loss based on the feature map of the target person wearing the second clothing and the real feature map of the target person wearing the second clothing; The update module is used to update the first style modulation network and the second style modulation network according to the clothing consistency loss.

16. The apparatus according to any one of claims 9 to 14, characterized in that, The extraction module is also used to extract feature information of the target person from the target image, wherein the target image is an image of the target person wearing the first garment.

17. An electronic device, characterized in that, The device includes a memory and a processor, the memory being coupled to the processor; the memory is used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed by the processor, the processor causes the processor to perform the method as described in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on a computing device, cause the computing device to perform the method as described in any one of claims 1 to 8.

Images