IMAGE PROCESSING METHOD, APPARATUS, AND ELECTRONIC DEVICE

The image processing method adapts clothing to human bodies using target appearance flow features, eliminating the need for human body analysis, thus achieving efficient and real-time high-quality virtual dress-up.

JP7596015B2Active Publication Date: 2024-12-09TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023522881
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-01-27
Filing Date
2022-01-20
Publication Date
2024-12-09
Estimated Expiration
2042-01-20

AI Technical Summary

Technical Problem

Current virtual dress-up technologies rely heavily on human body analysis, leading to inaccurate results and long processing times, making real-time high-quality virtual dress-up challenging.

Method used

An image processing method that generates a virtual dress-up image by adapting clothing to a human body without human body analysis, using target appearance flow features to represent deformation, and fusing transformed clothing images with human body images.

Benefits of technology

Enables high-quality virtual dress-up without human body analysis, improving efficiency and enabling real-time performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007596015000001
    Figure 0007596015000001
  • Figure 0007596015000002
    Figure 0007596015000002
  • Figure 0007596015000003
    Figure 0007596015000003
Patent Text Reader

Abstract

[0009] An embodiment of the present application discloses an image processing method and apparatus, the method including the steps of: acquiring a first image including a target person and a second image including a target outfit; generating a target appearance flow feature based on image features of the first image and the second image, the target appearance flow feature being used to represent a transformation caused by the target outfit adapting to the target person's body; generating a transformed image in which the target outfit adapts to the body based on the target appearance flow feature; and generating a virtual dress-up image based on the transformed image and the first image, in which the target person is wearing the target outfit adapted to the body. The technical solution of the embodiment of the present application does not need to rely on human body analysis results, and can achieve high-quality virtual dress-up.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method, apparatus, electronic device, and computer-readable storage medium.

[0002] This application claims priority to a Chinese patent application filed with the China Patent Office on January 27, 2021, bearing application number 202110141360.3 and entitled "Image Processing Method, Apparatus, Electronic Device, and Storage Medium," the entire contents of which are incorporated herein by reference. [Background technology]

[0003] The virtual dress-up technology refers to the merging of a human body image and a clothing image by technical means to obtain an image of the user wearing the clothing, which can facilitate the user to understand the effect of wearing the clothing, without the user having to actually wear the clothing. The virtual dress-up technology is widely applied in scenes such as online shopping, clothing exhibition, clothing design, or virtual try-on in offline shopping.

[0004] In the current virtual dress-up technology, it is necessary to rely on the results of human body analysis of human body images. An ideal virtual dress-up dataset should include images of a designated person wearing an arbitrary outfit, images including the target outfit, and images of the designated person wearing the target outfit, but it is difficult to obtain images of the same person wearing two different outfits while maintaining exactly the same movements. Therefore, the currently adopted virtual dress-up dataset only includes images of the designated person wearing the target outfit, and it is necessary to clear the target outfit area of ​​the designated person using the results of human body analysis, and then reconstruct the human body image using the images including the target outfit.

[0005] As can be seen from this, the realization of such a technology depends heavily on the results of human body analysis, and when the results of human body analysis are inaccurate, a virtual dress-up image in which the designated person and the target outfit do not match may be generated. In addition, in practical application scenarios, the process of human body analysis requires a long time, which causes the virtual dress-up result to be unable to be obtained in real time. Summary of the Invention [Problem to be solved by the invention]

[0006] In order to solve the above technical problems, the embodiments of the present application provide an image processing method, an apparatus, an electronic device, and a computer-readable storage medium, which eliminate the need to perform virtual dress-up depending on the results of human body analysis, and further avoid various problems caused by performing virtual dress-up depending on the results of human body analysis, thereby realizing high-quality virtual dress-up, while improving the efficiency of virtual dress-up, and realizing real-time virtual dress-up. [Means for solving the problem]

[0007] According to one aspect of an embodiment of the present application, there is provided an image processing method, the method being executed by a computing device and including the steps of: acquiring a first image including a target person and a second image including a target outfit; generating a target appearance flow feature based on image features of the first image and image features of the second image, the target appearance flow feature being used to represent the deformation caused by the target outfit adapting to the target person's body, and generating a transformed image in which the target outfit adapts to the human body based on the target appearance flow feature; and generating a virtual dress-up image based on a fusion of the transformed image and the first image, wherein in the virtual dress-up image, the target person is wearing a target outfit adapting to the human body.

[0008] According to one aspect of an embodiment of the present application, an image processing device is provided, comprising: an image acquisition module configured to acquire a first image including a target person and a second image including a target outfit; an information generation module configured to generate target appearance flow features used to represent the deformation caused by the target outfit adapting to the target person's body based on image features of the first image and image features of the second image, and to generate a transformed image in which the target outfit adapts to the human body based on the target appearance flow features; and a virtual dress-up module configured to generate a virtual dress-up image based on a fusion of the transformed image and the first image, wherein in the virtual dress-up image, the target person is wearing the target outfit adapting to the human body.

[0009] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising a processor and a memory, wherein computer-readable instructions are stored in the memory, and the computer-readable instructions, when executed by the processor, realize the image processing method.

[0010] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided having computer-readable instructions stored thereon that, when executed by a processor of a computer, cause the computer to perform the image processing method.

[0011] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer readable storage medium, a processor of a computing device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computing device to perform the image processing method provided in the various alternative embodiments. Effect of the Invention

[0012] In the technical solution provided by the embodiments of the present application, there is no need to perform virtual dress-up depending on the human body analysis result, and a deformation adapted to the human body is generated for the target dress by obtaining the target appearance flow feature of the deformation caused by the target dress adapting to the target person's body, and finally, the image of the target dress after deformation (for example, the post-deformation image) is merged with the first image including the target person to obtain a virtual dress-up image, thereby solving various problems caused by performing virtual dress-up depending on the human body analysis result in the realization of related technologies, and realizing high-quality virtual dress-up, and at the same time improving the efficiency of virtual dress-up and realizing real-time virtual dress-up.

[0013] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and are not intended to limit the present application.

[0014] The drawings herein are incorporated in and constitute a part of the specification, showing embodiments that are applicable to the present application, and are used together with the specification to interpret the principles of the present application. It is apparent that the drawings in the following description are merely some embodiments of the present application, and those skilled in the art can further obtain other drawings based on these drawings without requiring creative labor. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram of one embodiment of the present application; [Diagram 2] FIG. 2 is a structural schematic diagram of a virtual dress-up student model shown in an embodiment of the present application. [Diagram 3] FIG. 3 is a structural schematic diagram of a first clothing transformation sub-model 11 shown in FIG. 2 according to an embodiment. [Figure 4] 4 is a flowchart of appearance flow feature prediction performed by the “FN-2” module shown in FIG. 3 on the second image feature layer. [Diagram 5] 11 is a flowchart of an image processing method according to another embodiment of the present application. [Figure 6]1 is a flowchart illustrating the training of a virtual dress-up student model according to an embodiment of the present application. [Figure 7] FIG. 1 is a block diagram of an image processing device according to an embodiment of the present application. [Figure 8] 1 shows a structural schematic diagram of a computer system of an electronic device suitable for implementing an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Now, a detailed description of the exemplary embodiments will be given, examples of which are illustrated in the drawings. When the following description refers to the drawings, the same numerals in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0017] The block diagrams shown in the drawings are merely functional entities that do not necessarily correspond to physically separate entities, i.e. they may be implemented by employing software forms, or in one or more hardware modules or integrated circuits, or in different networks and / or processor and / or microcontroller devices.

[0018] The flowcharts shown in the drawings are merely exemplary illustrations, and do not necessarily include all contents and operations / steps, nor do they necessarily have to be performed according to the order described. For example, some operations / steps can be further decomposed, while some operations / steps can be merged or partially merged, and therefore the actual execution order may change based on the actual situation.

[0019] It is necessary to further explain that "plurality" as referred to in this application refers to two or more than two. Terms such as "first" and "second" as used in this application can be used to describe various concepts in this specification, but unless otherwise specified, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, a first image may be called a second image, and similarly, a second image may be called a first image, without departing from the scope of this application.

[0020] Artificial Intelligence (AI) is the theory, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, sense the environment, acquire knowledge, and use the knowledge to obtain optimal results. In other words, AI is a comprehensive technology of computer science that seeks to understand the nature of intelligence and create new intelligent machines that can respond in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, giving them the functions of sensing, reasoning, and decision-making.

[0021] Artificial intelligence technology is a comprehensive discipline with a wide range of related fields, including hardware and software technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. Artificial intelligence software technology includes natural language processing technology and machine learning.

[0022] Machine learning (ML) is an interdisciplinary field that involves multiple disciplines, such as probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specializes in the study of how computers can mimic or realize human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continually improve their own performance. Machine learning is the core of artificial intelligence and is the fundamental path to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0023] Computer Vision (CV) is a science that studies how to make machines "see", and more specifically, refers to machine vision that uses cameras and computers instead of human eyes to recognize, track, measure, etc., targets, and then processes the images by the computer to make them more suitable for observation by the human eye or for transmission to an instrument for detection. As a scientific discipline, computer vision studies related theories and techniques to create artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology typically includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, virtual reality, augmented reality, synchronous positioning, and map construction, and further includes general face recognition and biometric feature recognition technologies such as fingerprint recognition.

[0024] The image processing method provided in the embodiment of the present application will be described below based on artificial intelligence technology and computer vision technology.

[0025] The embodiment of the present application provides an image processing method, the execution subject is a computer device, and can fuse a human body image and a clothing image. In one possible manner, the computer device is a terminal, and the terminal can be a smart phone, a tablet PC, a laptop, a desktop computer, a smart speaker, a smart watch, an in-vehicle computer, etc. In another possible manner, the computer device is a server, and the server can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system, where multiple servers can constitute a blockchain. The server can be a node in the blockchain, and the server can further be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms.

[0026] The image processing method provided by the embodiment of the present application can be applied to any scene where a human body image and a clothing image are integrated. For example, in the scene of virtual dressing during online shopping, if a user wants to understand the effect of wearing a certain garment, only the human body image of the user and the clothing image of the garment are provided, and the method provided by the embodiment of the present application is used to process the human body image and the clothing image, so that the user can obtain the human body image wearing the garment, and realize online virtual dressing, and the user does not need to wear the actual garment.

[0027] In addition, the image processing method provided by the embodiments of the present application can also be applied in scenes such as clothing design, clothing exhibition, or virtual try-on for offline shopping, thereby providing real-time virtual dress-up functions, which will not be listed here.

[0028] As shown in Fig. 1, Fig. 1 is a flow chart of an image processing method according to an embodiment of the present application. The image processing method includes at least S110 to S150, and S110 to S150 can be specifically realized as a virtual dress-up student model. Here, the virtual dress-up student model is a kind of artificial intelligence model, and can realize the virtual dress-up of a target person without relying on the human body analysis result, and can not only generate a high-quality virtual dress-up image, but also improve the real-time performance of virtual dress-up.

[0029] The image processing method shown in FIG. 1 will now be described in detail.

[0030] S110: A first image including a target person and a second image including a target outfit are obtained.

[0031] In this embodiment, the target person refers to a person who is the subject of virtual dressing, and the target outfit refers to the outfit that the target person is going to wear.

[0032] For example, in a scene of virtual dressing during online shopping, the target person may be a user currently shopping online, the first image may be a body image of the user provided by the user, and the second image may be a picture of a target outfit loaded in a shopping platform. It is necessary to explain that the target person contained in the first image and the target outfit contained in the second image may be determined according to an actual application scene, but this is not limited thereto.

[0033] S130: Based on the image features of the first image and the image features of the second image, generate a target appearance flow feature used to represent the deformation caused by the target outfit adapting to the target person's body, and generate a deformed image in which the target outfit adapts to the human body based on the target appearance flow feature.

[0034] First, the image features of the first image are obtained by performing image feature extraction on the first image, and the image features of the second image are obtained by performing image feature extraction on the second image. For example, in some embodiments, the first image can be input to the first image feature extraction model, and the second image can be input to the second image feature extraction model (i.e., the first image is the input signal of the first image feature extraction model, and the second image is the input signal of the second image feature extraction model), and image feature extraction algorithms are arranged in both the first image feature extraction model and the second image feature extraction model, thereby obtaining the image features output by the first image feature extraction model for the first image, and obtaining the image features output by the second image feature extraction model for the second image.

[0035] The image features of the first image output by the first image feature extraction model and the image features of the second image output by the second image feature extraction model may be multi-layer image features, which refer to multiple feature maps obtained in sequence in the process of performing image feature extraction on the first image and the second image.

[0036] Exemplarily, the first image feature extraction model and the second image feature extraction model may be pyramid feature extraction models. A feature pyramid network (FPN) is arranged in the pyramid feature extraction model, and the feature map pyramid output from the feature pyramid network is a multi-layer image feature corresponding to the image. For example, in some embodiments, the bottom-up part of the pyramid feature extraction model can be adopted to perform image feature extraction on the first image and the second image, and the bottom-up part can be understood to use a convolution network to perform image feature extraction. As the convolution becomes deeper, the spatial resolution of the image becomes relatively less and spatial information is lost, but by enriching high-level semantic information, a multi-layer image feature with a feature map size order from large to small is obtained.

[0037] Appearance flow feature refers to a two-dimensional coordinate vector, and is usually used to indicate which pixel in the source image can be used to reconstruct a specified pixel in the target image. In this embodiment, in order to realize high-quality virtual dress-up, it is necessary to establish an accurate and dense correspondence between the target person's body and the target outfit, so that the target outfit is deformed to fit the body. Therefore, in this embodiment, the source image refers to the second image, and can specifically refer to the target outfit area in the second image, and the target image to be reconstructed refers to the deformed image generated by the target outfit being adapted to the target person's body in the first image.

[0038] As is apparent from this, the target appearance flow features can represent the deformation that occurs when the target outfit is adapted to the target person's body in the first image, and based on the acquired target appearance flow features, a deformed image in which the target outfit is adapted to the human body can be generated.

[0039] When the image features of a first image are multi-layer image features output via a first image feature extraction model, and the image features of a second image are multi-layer image features output via a second image feature extraction model, appearance flow features are extracted layer by layer based on the first image feature extraction model and the multi-layer image features output by the second image feature extraction, and the appearance flow features obtained by performing extraction on the last image feature layer can be used as the target appearance flow features to be finally generated.

[0040] For example, in the first image feature layer, an appearance flow feature can be extracted based on the image features output by the first image feature extraction model and the second image feature extraction model, which is used to represent the deformation caused by the target clothing adapting to the target person's body. In each image feature layer after the first image feature layer, an optimization process can be performed on the appearance flow feature corresponding to the previous image feature layer based on the image features output by the first image feature extraction model and the second image feature extraction model, to obtain an appearance flow feature corresponding to the current image feature layer.

[0041] In the process of extracting the appearance flow features layer by layer based on the multi-layer image features output by the first image feature extraction model and the second image feature extraction model, the appearance flow features can be further extracted based on a preset secondary smoothness constraint condition, where the secondary smoothness constraint condition is a constraint condition set for the linear correspondence relationship between adjacent appearance flows, thereby improving the image quality of the transformed image in which the generated target clothing is adapted to the target person's body by further retaining features such as the pattern and stripe pattern of the target clothing.

[0042] S150: A virtual dress-up image is generated based on the fusion of the transformed image in which the target outfit is adapted to the target person's body and the first image, and in the virtual dress-up image, the target person is wearing the target outfit adapted to the human body.

[0043] Generating a virtual dress-up image based on fusing a transformed image in which the target clothing fits the target person's body with the first image can be specifically realized by an image fusion algorithm applied to the virtual dress-up. For example, the Res-UNet algorithm can be adopted, and this embodiment is not limited thereto.

[0044] As can be seen from the above, the technical solution provided by the embodiment does not need to perform virtual dress-up depending on the human body analysis result, and generates a deformation for the target dress that is adapted to the human body by obtaining the target appearance flow feature of the deformation caused by the target dress adapting to the target person's body, and finally obtains a virtual dress-up image by fusing the image of the target dress after deformation (for example, the post-deformation image) with the first image including the target person. This avoids the occurrence of problems such as the virtual dress-up image being low quality and the relatively poor real-time performance of virtual dress-up caused by performing virtual dress-up depending on the human body analysis result, and realizes high-quality virtual dress-up. At the same time, it improves the efficiency of virtual dress-up and realizes real-time virtual dress-up.

[0045] Referring to Fig. 2, Fig. 2 is a structural schematic diagram of a virtual dress-up student model shown in one embodiment of the present application. The exemplary virtual dress-up student model 10 includes a first clothing transformation sub-model 11 and a first clothing generation sub-model 12, where the first clothing transformation sub-model 11 can execute S130 in the embodiment shown in Fig. 1, and the first clothing generation sub-model 12 can execute S150 in the embodiment shown in Fig. 1.

[0046] As shown in FIG. 2, by inputting a first image including a target person and a second image including a target outfit into the virtual dress-up student model 10, the virtual dress-up student model 10 can output a corresponding virtual dress-up image, in which the target person is wearing a target outfit that fits the human body.

[0047] Except for the first image and the second image, the virtual dress-up student model 10 does not require any other input signals, and there is no need to input the body analysis results of the target person contained in the first image to the virtual dress-up student model 10.

[0048] Fig. 3 is a structural schematic diagram of an embodiment of the first clothing transformation sub-model 11 shown in Fig. 2. As shown in Fig. 3, the first clothing transformation sub-model 11 includes a first image feature extraction model, a second image feature extraction model, and an appearance flow feature prediction model.

[0049] Here, the first image feature extraction model is used to extract image features of a first image, and the second image feature extraction model is used to extract image features of a second image. As shown in Fig. 3, the first image feature extraction model performs image feature extraction on the first image to obtain multi-layer image features shown in order as c1 to c3, and the second image feature extraction model performs image feature extraction on the second image to obtain multi-layer image features shown in order as p1 to p3.

[0050] It should be noted that the multi-layer image features shown in FIG. 3 are only examples, and the number of layers of image features of the input image extracted by the first image feature extraction model and the second image feature extraction model can be set according to actual needs, and this embodiment is not limited in this regard.

[0051] The appearance flow feature prediction model is used to extract appearance flow features layer by layer based on the multi-layer image features output by the first image feature extraction model and the second image feature extraction model, and to use the appearance flow feature obtained by performing extraction on the last image feature layer as the target appearance flow feature to be finally generated. For example, the "FN-1" module shown in Fig. 3 is used to perform appearance flow feature prediction on the first image feature layer, the "FN-2" module is used to perform appearance flow feature prediction on the second image feature layer, and the "FN-3" module is used to perform appearance flow feature prediction on the third image feature layer. That is, the appearance flow feature prediction model is a progressive appearance flow feature prediction model.

[0052] As shown in Fig. 3, in the first image feature layer, the appearance flow feature prediction model first extracts appearance flow features used to represent the deformation caused by the target clothing adapting to the target person's body based on the image features output by the first image feature extraction model and the second image feature extraction model. In each image feature layer after the first image feature layer, the appearance flow feature prediction model performs optimization processing on the appearance flow features output from the previous image feature layer based on the image features output by the first image feature extraction model and the second image feature extraction model to obtain appearance flow features corresponding to the current image feature layer.

[0053] By this gradual processing method, the multi-layer image feature is continuously deepened in convolution, the spatial resolution of the image is gradually reduced, and the spatial information is gradually lost, but the high-level semantic information is enriched, so that the feature information contained in the appearance flow feature obtained by the appearance flow feature prediction model layer by layer is also becoming richer and more accurate. For example, in the appearance flow feature f1 to f3 shown in Fig. 3, the feature information contained therein is gradually enriched and gradually adapted to the target person's human body.

[0054] It is clear from this that the appearance flow feature obtained by the appearance flow feature prediction model in the last image feature layer can very accurately reflect the deformation caused by the target clothing adapting to the target person's body. The post-deformation image corresponding to the target clothing generated based on the appearance flow feature obtained by the appearance flow feature prediction model in the last image feature layer can create an accurate and close correspondence with the target person's body, and can then be fused based on the accurate deformation caused in the target clothing and the target person's body to obtain a high-quality virtual dress-up image.

[0055] FIG. 4 is a flowchart of the appearance flow feature prediction performed by the “FN-2” module shown in FIG. 3 on the second image feature layer. As shown in FIG. 4, first, an upsampling process is performed on the appearance flow feature f1 corresponding to the previous image feature layer to obtain an upsampling feature f1′. Next, a first transformation process is performed on the image feature c2 of the second image corresponding to the current feature layer based on the upsampling feature f1′ to obtain a first transformed feature c2′. Next, a calibration process is performed on the first transformed feature c2′ based on the image feature p2 of the first image corresponding to the current image feature layer to obtain a revised feature r2, and a convolution calculation is performed on the revised feature r2 to obtain a first convolution feature f2′″. Next, a second transformation process is performed on the image feature c2 of the second image corresponding to the current image feature layer based on the feature f2″ obtained by connecting the first convolution feature f2′″ and the upsampling feature f1′ to obtain a second transformed feature p2 c2″. The second post-transformation feature is a combination of the image feature p2 of the first image output by the current image feature layer and another feature c2''. Finally, a second convolution calculation is performed on the second post-transformation feature p2 c2'', and the second convolution feature f2' obtained by the calculation is concatenated with the first convolution feature f2'' to obtain the appearance flow feature f2 corresponding to the current image feature layer.

[0056] As can be seen from the above, performing upsampling processing on the appearance flow feature output by the previous image feature layer is beneficial to improving the resolution of the appearance flow feature of the current image feature layer. After that, by performing two deformation processes and two convolution calculations, the feature information contained in the upsampling feature can be further subdivided, which is equivalent to adding new spatial information of the appearance flow feature based on the appearance flow feature output by the previous image feature layer. This realizes optimization of the appearance flow feature output by the previous image feature layer, and obtains an appearance flow feature that can further reflect the deformation of the target clothing to fit the target person's body.

[0057] It is also worth noting that in some embodiments, the appearance flow feature prediction model extracts appearance flow features layer by layer based on the multi-layer image features output by the first image feature extraction model and the second image feature extraction model, and further extracts appearance flow features based on a preset quadratic smoothness constraint condition for the linear correspondence relationship between adjacent appearance flows, thereby further retaining features such as the pattern and stripe pattern of the target clothing.

[0058] 5 is a flow chart of an image processing method according to another embodiment of the present application. As shown in FIG. 5, the method is based on the embodiment shown in FIG. 1 and further includes S210 to S250. A detailed introduction is given as follows:

[0059] S210: Call the virtual dress-up teaching assistant model, input the human body analysis result corresponding to the person image including the designated person and the first clothing image including the clothing to be changed into the virtual dress-up teaching assistant model, and obtain the teaching assistant image output by the virtual dress-up teaching assistant model. In the teaching assistant image, the designated person is wearing the clothing to be changed that is adapted to the human body of the designated person.

[0060] First, as a point to be explained, this embodiment discloses a process of training the virtual dress-up student model shown in FIG. 2. In the training stage of the virtual dress-up student model, it is necessary to call the virtual dress-up teaching assistant model to perform auxiliary training, specifically, the virtual dress-up teaching assistant model is an artificial intelligence model that depends on the human body analysis result. By inputting the human body analysis result corresponding to the person image including the designated person and the first clothing image including the clothing to be changed into the virtual dress-up teaching assistant model, the virtual dress-up teaching assistant model can output the corresponding teaching assistant image. In the teaching assistant image, the designated person is wearing the clothing to be changed that is adapted to the human body of the designated person.

[0061] In this embodiment, the virtual dress-up data set is an image data set composed of a person image including a designated person, a first clothing image including the clothing to be changed, and a second clothing image including the original clothing worn by the designated person. Here, the number of person images, the first clothing image, and the second clothing image may be multiple, and the designated person included in different person images may or may not be the same, and this embodiment is not limited thereto.

[0062] S230: A second clothing image including the original clothing and the teaching assistant image are input into a virtual dress-up student model to be trained, and a student image output by the virtual dress-up student model to be trained is obtained. In the student image, the designated person wears the original clothing that fits the designated person's body in the teaching assistant image, and the original clothing is the clothing worn by the designated person in the person image.

[0063] This does not mean that the virtual dress-up student model realizes virtual dress-up depending on the human body analysis results, but rather that the features extracted by the virtual dress-up teaching assistant model based on the human body analysis results may contain richer semantic information and feature expression, so this embodiment uses the virtual dress-up teaching assistant model to guide and train the virtual dress-up student model.

[0064] That is, in this embodiment, the virtual dress-up student model is trained by using the method of knowledge distillation.

[0065] Here, knowledge distillation refers to training a student network using the inherent information of a teacher network. In the embodiment, the teacher network is a virtual dress-up teaching assistant model, and the inherent information of the teacher network refers to the feature representation and semantic information extracted by the virtual dress-up teaching assistant model based on the results of human body analysis.

[0066] A well-trained virtual dress-up student model can fully learn accurate and dense correspondence between human body and clothing, so that in practical application, there is no need to obtain the human body analysis result of the target person, and the virtual dress-up student model can still output a high-quality virtual dress-up image based on the first image including the target person and the second image including the target clothing input thereto.

[0067] Specifically, in this embodiment, the teaching assistant image output by the virtual dress-up teaching assistant model is input as teaching assistant knowledge to the virtual dress-up student model to be trained, and a second clothing image including the original clothing is input to the virtual dress-up student model to be trained, so that the virtual dress-up student model to be trained outputs a student image. In the student image, the designated person wears the original clothing that fits the designated person's body in the teaching assistant image.

[0068] S250: Using a person image as a teacher image, parameters are updated for the virtual dress-up student model to be trained based on image loss information between the student image and the teacher image.

[0069] In this embodiment, a person image is used as a teacher image to supervise the training process of the virtual dress-up student model. That is, the virtual dress-up student model can be directly supervised by the teacher image during the training process, which is beneficial to improve the performance of the virtual dress-up student model. Therefore, the virtual dress-up student model finally obtained by training can be freed from the dependence on the human body analysis result in practical application, and can output a high-quality virtual dress-up image according to the input first image and second image.

[0070] The image loss information between the student image and the teacher image may be obtained by performing a loss function value calculation on the student image and the teacher image. For example, an image loss value of the student image for the teacher image may be obtained, and the image loss value may include at least one of a pixel distance loss function value, a sense loss function value, and an opposition loss function value. Then, an addition operation is performed on the image loss value to obtain an image loss sum value of the student image for the teacher image, and finally, the image loss sum value is used as the image loss information between the student image and the teacher image to perform parameter update on the virtual dress-up student model to be trained, thereby completing one training of the virtual dress-up student model.

[0071] By performing multiple trainings on the virtual dress-up student model to be trained, the model performance of the virtual dress-up student model is gradually improved. When the image loss information between the student image and the teacher image is smaller than or equal to a preset image loss threshold, it indicates that the virtual dress-up student model has already achieved a relatively good model performance, and the training process of the virtual dress-up student model can be terminated.

[0072] It should also be mentioned that the human body analysis result may include information such as human body key points, human body pose heat map, and dense pose estimation, and in many situations, the virtual dress-up teaching assistant model can extract richer semantic information based on the human body analysis result, and the appearance flow features obtained by prediction are also more accurate. Therefore, the image quality of the teaching assistant image output by the virtual dress-up teaching assistant model should be higher than that of the student image output by the virtual dress-up student model.

[0073] If the human body analysis results input to the virtual dress-up teaching assistant model are not accurate, it will cause the virtual dress-up teaching assistant model to provide completely wrong guidance to the virtual dress-up student model during the training process of the virtual dress-up student model. Therefore, it is necessary to set up an adjustable knowledge distillation mechanism, so as to ensure that only accurate teaching assistant images can be used to train the virtual dress-up student model.

[0074] Specifically, by obtaining the image quality difference between the teaching assistant image and the student image before S250, if it is determined that the image quality difference is a positive value, it indicates that the image quality of the teaching assistant image is greater than that of the student image, and then S250 is executed, thereby training the virtual dress-up student model based on the teaching assistant image. If it is determined that the image quality difference is a negative value or zero, it indicates that the image quality of the teaching assistant image is not greater than that of the student image, and the human body analysis result inputted into the virtual dress-up teaching assistant model may be completely wrong, so the execution of S250 is terminated and the next round of the virtual dress-up student model training process begins.

[0075] FIG. 6 is a training flowchart of the virtual dress-up student model shown in one embodiment of the present application. As shown in FIG. 6, the virtual dress-up teaching assistant model 20 is used as an auxiliary model for training the virtual dress-up student model 10, and the virtual dress-up teaching assistant model 20 outputs a corresponding teaching assistant image based on the first clothing image input thereto and the human body analysis result obtained by performing human body analysis on the person image (i.e., the teacher image). Next, the teaching assistant image output by the virtual dress-up teaching assistant model 20 and the second clothing image are input to the virtual dress-up student model 10 to obtain the student image output by the virtual dress-up student model 10. Based on the image loss information between the student image and the teacher image, the virtual dress-up student model 10 can be updated as a parameter.

[0076] The virtual dress-up teaching assistant model 20 includes a second clothing transformation sub-model 21 and a second clothing generation sub-model 22. By calling the second clothing transformation sub-model 21, a transformed image in which the clothing to be changed is adapted to the human body of a designated person can be generated based on the human body analysis result and the image features of the first clothing image. The detailed process can be referred to the description of the corresponding embodiment in FIG. 3 and FIG. 4, and will not be described in detail here. By calling the second clothing generation sub-model 22, a teaching assistant image can be generated based on the fusion of the transformed image corresponding to the clothing to be changed output by the second clothing transformation sub-model and other image areas in the person image except the area wearing the original clothing.

[0077] In another embodiment, by calling the second dress-up generation sub-model 22, it is further possible to clear the area in which the designated person contained in the person image is wearing the original clothes based on the human body analysis result, thereby obtaining other image areas in the person image excluding the area in which the original clothes are worn.

[0078] It is necessary to explain that the first clothing transformation sub-model contained in the virtual dress-up student model and the second clothing transformation sub-model contained in the virtual dress-up teaching assistant model may have the same network structure, for example, the network structure shown in Fig. 3. The first clothing generation sub-model contained in the virtual dress-up student model and the second clothing generation sub-model contained in the virtual dress-up teaching assistant model may also have the same network structure. For example, the first clothing generation sub-model and the second clothing generation sub-model may be composed of an encoder-decoder network and a residual network, and the residual network is used to perform normalization processing on the upper layer network to which it is connected, which makes it easy to perform parameter optimization processing in the model training process.

[0079] As can be seen from the above, the present application uses a novel "teacher-teaching assistant-student" knowledge distillation mechanism to train a virtual dress-up student model that does not need to rely on human body analysis results, and the virtual dress-up student model is supervised by a teacher image during the training process, so that the virtual dress-up student model finally obtained by training does not need to rely on human body analysis results, and can generate highly realistic virtual dress-up results, thereby realizing high-quality virtual dress-up in a situation where it does not need to rely on human body analysis results.

[0080] 7 is a block diagram of an image processing device according to an embodiment of the present application. As shown in FIG. 7, in one exemplary embodiment, the image processing device includes: The virtual dress-up module 350 includes an image acquisition module 310 configured to acquire a first image including a target person and a second image including a target outfit; an information generation module 330 configured to generate target appearance flow features used to represent deformation caused by the target outfit adapting to the target person's body based on image features of the first image and image features of the second image, and to generate a transformed image in which the target outfit adapts to the human body based on the target appearance flow features; and a virtual dress-up module 350 configured to generate a virtual dress-up image based on a fusion of the transformed image and the first image, in which the target person is wearing the target outfit adapting to the human body.

[0081] In another exemplary embodiment, the information generation module 330 is The system includes a multi-layer image feature acquisition unit configured to use the first image as an input signal of a first image feature extraction model and the second image as an input signal of a second image feature extraction model, and to extract multi-layer image features corresponding to the input signals using the first image feature extraction model and the second image feature extraction model, respectively; and an appearance flow feature extraction unit configured to extract appearance flow features for each layer based on the multi-layer image features output by the first image feature extraction model and the second image feature extraction model, and to use the appearance flow feature obtained by performing extraction for the last image feature layer as a target appearance flow feature.

[0082] In another exemplary embodiment, the appearance flow feature extraction unit comprises: The system includes a first feature extraction subunit configured in the first image feature layer to extract appearance flow features used to represent the deformation caused by the target clothing adapting to the target person's body based on the image features output by the first image feature extraction model and the second image feature extraction model, and a second feature extraction subunit configured in each image feature layer after the first image feature layer to perform an optimization process on the appearance flow features corresponding to the previous image feature layer based on the image features output by the first image feature extraction model and the second image feature extraction model, to obtain appearance flow features corresponding to the current image feature layer.

[0083] In another exemplary embodiment, the second feature extraction subunit comprises: The image processing apparatus includes a first transformation processing subunit configured to perform an upsampling process based on an appearance flow feature corresponding to a previous image feature layer to obtain an upsampling feature, and to perform a first transformation process on an image feature of a second image corresponding to a current image feature layer based on the upsampling feature, to obtain a first post-transformation feature; a calibration processing subunit configured to perform a calibration process on the first post-transformation feature based on the image feature of the first image corresponding to the current image feature layer, and to perform a first convolution calculation on the post-transformation feature obtained by the calibration process, to obtain a first convolution feature; a second transformation processing subunit configured to perform a second transformation process on an image feature of the second image corresponding to the current image feature layer based on a feature obtained by connecting the first convolution feature and the upsampling feature, to obtain a second post-transformation feature; and an appearance flow feature acquisition subunit configured to perform a second convolution calculation on the second post-transformation feature, and to connect the second convolution feature obtained by the calculation and the first convolution feature to obtain an appearance flow feature corresponding to the current image feature layer.

[0084] In another exemplary embodiment, the information generation module 330 is In the process of extracting appearance flow features for each layer based on the multi-layer image features output by the first image feature extraction model and the second image feature extraction model, the method further includes a secondary smoothness constraint unit configured to extract appearance flow features based on a secondary smoothness constraint condition, where the secondary smoothness constraint condition is a predetermined constraint condition for the linear correspondence relationship between adjacent appearance flows.

[0085] In another exemplary embodiment, the information generation module 330 is arranged as a first clothing transformation sub-model contained in the virtual dress-up student model, and the virtual dress-up module 350 is arranged as a first clothing generation sub-model contained in the virtual dress-up student model.

[0086] In another exemplary embodiment, the image processing device further comprises: The teaching assistant image acquisition module is configured to call a virtual dress-up teaching assistant model, input a human body analysis result corresponding to a person image including a designated person and a first clothing image including a clothing target to be dressed up into the virtual dress-up teaching assistant model, and acquire a teaching assistant image output by the virtual dress-up teaching assistant model, in which the designated person is wearing the clothing target to be dressed up that fits the designated person's body in the teaching assistant image; a student image acquisition module is configured to input a second clothing image including an original clothing and the teaching assistant image into a virtual dress-up student model to be trained, and acquire a student image output by the virtual dress-up student model to be trained, in which the designated person is wearing the original clothing that fits the designated person's body in the teaching assistant image, and the original clothing is the clothing worn by the designated person in the person image; and a parameter update module configured to use the person image as a teacher image and perform parameter update on the virtual dress-up student model to be trained based on image loss information between the student image and the teacher image.

[0087] In another exemplary embodiment, the image processing device further comprises: The system further includes an image quality difference acquisition module configured to acquire an image quality difference between the teaching assistant image and the student image, and if the image quality difference is a positive value, perform a step of taking the person image as a teacher image and performing parameter update on the virtual dress-up student model to be trained based on image loss information between the student image and the teacher image.

[0088] In another exemplary embodiment, the teaching assistant image acquisition module comprises: The virtual dress-up teaching assistant model includes a second clothing transformation sub-model calling unit configured to call a second clothing transformation sub-model in the virtual dress-up teaching assistant model and generate a transformed image in which the clothing to be changed is adapted to the body of a specified person based on the human body analysis result and the image features of the first clothing image, and a second clothing generation sub-model calling unit configured to call a second clothing generation sub-model in the virtual dress-up model and generate a teaching assistant image based on a fusion of the transformed image corresponding to the clothing to be changed output by the second clothing transformation sub-model with other image areas in the person image excluding the area where the original clothing is worn.

[0089] In another exemplary embodiment, the teaching assistant image acquisition module comprises: The virtual dress-up model further includes an image region information acquisition unit configured to call a second dress-up generation sub-model in the virtual dress-up model, and clear an area in which a designated person contained in the person image is wearing the original clothes based on a human body analysis result, thereby obtaining other image areas in the person image excluding the area in which the designated person is wearing the original clothes.

[0090] In another exemplary embodiment, the parameter update module further comprises: The method includes: an image loss value acquisition unit configured to acquire an image loss value for a teacher image of a student image, where the image loss value includes at least one of a pixel distance loss function value, a perception loss function value, and an opposition loss function value; a loss value addition unit configured to perform an addition operation on the image loss value to obtain an image loss sum value for the teacher image of the student image; and a model parameter update unit configured to use the image loss sum value as image loss information between the student image and the teacher image to perform parameter update on the virtual dress-up student model to be trained.

[0091] In another exemplary embodiment, the first dress-up generation sub-model is composed of an encoder-decoder network and a residual network, and the residual network is used to perform normalization processing on the connected upper layer network.

[0092] It is necessary to explain that the apparatus provided in the above embodiment belongs to the same concept as the method provided in the above embodiment, where the specific manner in which each module and unit performs operations has already been described in detail in the method embodiment, and will not be described in detail here.

[0093] An embodiment of the present application further provides an electronic device, comprising a processor and a memory, where computer readable instructions are stored in the memory, which, when executed by the processor, implement the image processing method described above.

[0094] FIG. 8 shows a structural schematic diagram of a computer system of an electronic device suitable for implementing an embodiment of the present application.

[0095] It should be explained that the electronic device computer system 1600 shown in FIG. 8 is just one example and should not pose any limitation on the functionality and scope of use of the embodiments of the present application.

[0096] As shown in Fig. 8, the computer system 1600 includes a central processing unit (CPU) 1601, which can execute various suitable operations and processes based on programs stored in a read-only memory (ROM) 1602 or programs loaded from a storage portion 1608 into a random access memory (RAM) 1603, for example, executing the methods described in the above embodiments. Various programs and data required for the system operation are stored in the RAM 1603. The CPU 1601, the ROM 1602, and the RAM 1603 are connected to each other by a bus 1604. An input / output (I / O) interface 1605 is also connected to the bus 1604.

[0097] An input section 1606 including a keyboard, a mouse, etc., an output section 1607 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc., a storage section 1608 including a hard disk, etc., and a communication section 1609 including, for example, a LAN (Local Area Network) card, a modem, etc., are connected to the I / O interface 1605. The communication section 1609 executes communication processing via a network such as the Internet. A driver 1610 is also connected to the I / O interface 1605 as necessary. For example, a removable medium 1611 such as a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory is attached to the driver 1610 as necessary, so that a computer program read from the driver 1610 is easily installed in the storage section 1608 as necessary.

[0098] In particular, according to an embodiment of the present application, the processes described above with reference to the flowcharts may be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, the computer program being used to execute the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network by the communication portion 1609 and / or installed from a removable medium 1611. When the computer program is executed by the central processing unit (CPU) 1601, it performs various functions specific to the system of the present application.

[0099] It is necessary to explain that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium, or a computer-readable storage medium, or any combination of the above. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to, an electrical connection having one or more conductors, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, optical fiber, a portable compact magnetic disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used or be used in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, on which a computer-readable computer program is carried. Such a propagated data signal may take a number of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may transmit, propagate, or transmit a program used by or in connection with an instruction execution system, apparatus, or device. A computer program contained in a computer-readable medium may be transmitted by any suitable medium, including, but not limited to, wireless, wired, etc., or any suitable combination of the above.

[0100] The flowcharts and block diagrams in the drawings illustrate possible system architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present application. Here, each block in the flowcharts or block diagrams may represent a module, program segment, or part of code, which includes one or more executable instructions used to realize a certain logic function. It should also be noted that in some alternative implementations, the functions marked in the blocks may be performed in a different order from the order marked in the drawings. For example, two blocks shown in succession may actually be performed substantially in parallel, or in some cases they may be performed in the reverse order, depending on the related functions. It should also be noted that each block in the block diagrams or flowcharts, and combinations of blocks in the block diagrams or flowcharts may be realized in a dedicated system based on hardware that performs a certain function or operation, or in a combination of dedicated hardware and computer instructions.

[0101] In the description of the embodiments of the present application, the relevant units may be implemented in a software manner or in a hardware manner, and the described units may be installed in a processor, whereby, in some circumstances, the names of the units do not constitute limitations on the units themselves.

[0102] Another aspect of the present application further provides a computer-readable storage medium having a computer program stored therein, the computer program implementing the image processing method when executed by a processor. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently of the electronic device.

[0103] Another aspect of the present application further provides a computer program product, or a computer program, comprising computer instructions stored in a computer readable storage medium, a processor of a computing device reading the computer instructions from the computer readable storage medium, and the processor executing the computer instructions to cause the computing device to perform the image processing method provided in each of the above embodiments.

[0104] The above content is merely a preferred illustrative embodiment of the present application, and is not used to limit the implementation means of the present application. Those skilled in the art can very easily make corresponding changes or modifications based on the main concept and spirit of the present application, and therefore the protection scope of the present application should be in accordance with the protection scope required by the claims. [Explanation of symbols]

[0105] 310 Image Acquisition Module 330 Information Generation Module 1600 Computer Systems 1601 Central Processing Unit 1602 Memory 1603 Random Access Memory 1604 Bus 1605 I / O Interface 1606 Input section 1607 Output part 1608 Memory part 1609 Communication section 1610 Driver 1611 Medium

Claims

1. 1. A method of image processing implemented by a computing device, the method comprising: acquiring a first image including a target person and a second image including a target outfit; generating a target appearance flow feature used to represent a deformation caused by the target outfit adapting to the target person's body based on the image features of the first image and the image features of the second image, and generating a post-deformation image in which the target outfit adapts to the human body based on the target appearance flow feature, using the first image as an input signal of a first image feature extraction model and the second image as an input signal of a second image feature extraction model, and extracting multi-layer image features corresponding to the input signals by the first image feature extraction model and the second image feature extraction model; Extracting appearance flow features for each layer based on the multi-layer image features output by the first image feature extraction model and the second image feature extraction model, and extracting the appearance flow features for the last image feature layer to obtain the target appearance flow features; and generating a virtual dress-up image based on the fusion of the transformed image and the first image, wherein in the virtual dress-up image, the target person is wearing a target outfit that fits the human body.

2. A method of image processing, implemented by a computing device, comprising: acquiring a first image including a target person and a second image including a target outfit; generating a target appearance flow feature used to represent the deformation caused by the target outfit adapting to the target person's body based on the image features of the first image and the image features of the second image using a first clothing deformation sub-model in a virtual dress-up student model, and generating a post-deformation image in which the target outfit adapts to the human body based on the target appearance flow feature; An image processing method including a step of generating a virtual dress-up image based on a fusion of the transformed image and the first image by a first dress-up generation sub-model in the virtual dress-up student model, wherein in the virtual dress-up image, the target person is wearing a target outfit that fits the human body.

3. The step of extracting appearance flow features for each layer based on the multi-layer image features output by the first image feature extraction model and the second image feature extraction model, and setting the appearance flow features obtained by extracting the last image feature layer as the target appearance flow features, Extracting an appearance flow feature into a first image feature layer according to the image features output by the first image feature extraction model and the second image feature extraction model, the appearance flow feature being used to represent the deformation caused by the target clothing being adapted to the target person's body; 2. The method of claim 1, further comprising: for each image feature layer after the first image feature layer, performing an optimization process on the appearance flow features corresponding to the previous image feature layer based on the image features output by the first image feature extraction model and the second image feature extraction model, to obtain appearance flow features corresponding to the current image feature layer.

4. the step of performing an optimization process on the appearance flow feature corresponding to the previous image feature layer based on the image features output by the first image feature extraction model and the second image feature extraction model for each image feature layer after the first image feature layer, and acquiring the appearance flow feature corresponding to the current image feature layer; performing an upsampling process based on the appearance flow feature corresponding to the previous image feature layer to obtain an upsampling feature; performing a first transformation process on an image feature of a second image corresponding to the current image feature layer according to the upsampling feature to obtain a first post-transformation feature; performing a calibration process on the first post-transformation feature according to the image feature of a first image corresponding to the current image feature layer, and performing a first convolution calculation on the post-transformation feature obtained by the calibration process to obtain a first convolution feature; performing a second transformation process on the image features of a second image corresponding to the current image feature layer based on the features obtained by concatenating the first convolution features and the upsampling features to obtain second transformed features; performing a second convolution calculation on the second transformed features, and stitching the second convolution features obtained by the calculation with the first convolution features to obtain an appearance flow feature corresponding to the current image feature layer.

5. 2. The method according to claim 1 , wherein in the process of extracting appearance flow features for each layer based on the multi-layer image features output by the first image feature extraction model and the second image feature extraction model, the appearance flow features are further extracted based on a quadratic smoothness constraint condition, the quadratic smoothness constraint condition being a preset constraint condition for a linear correspondence relationship between adjacent appearance flows.

6. The method comprises: A step of calling a virtual dress-up teaching assistant model, inputting a human body analysis result corresponding to a person image including a designated person and a first clothing image including a clothing to be changed into the virtual dress-up teaching assistant model, and acquiring a teaching assistant image output by the virtual dress-up teaching assistant model, in which the designated person in the teaching assistant image is wearing a clothing to be changed that is adapted to the human body of the designated person; inputting a second clothing image including an original clothing and the teaching assistant image into a virtual dress-up student model to be trained to obtain a student image output by the virtual dress-up student model to be trained, in which the designated person in the student image is wearing an original clothing that fits the designated person's body in the teaching assistant image, and the original clothing is the clothing worn by the designated person in the person image; The method according to claim 1 or 2, further comprising: using the person image as a teacher image, and performing parameter update on the virtual dress-up student model to be trained based on image loss information between the student image and the teacher image.

7. Before the step of updating parameters for the virtual dress-up student model to be trained based on image loss information between the student image and the teacher image, the method includes: obtaining an image quality difference between the teaching assistant image and the student image; The method of claim 6, further comprising: if the image quality difference is a positive value, performing the step of taking the person image as a teacher image and updating parameters for the virtual dress-up student model to be trained based on image loss information between the student image and the teacher image.

8. The step of calling a virtual dress-up teaching assistant model, inputting a human body analysis result corresponding to a person image including a specified person and a first clothing image including a clothing to be changed into the virtual dress-up teaching assistant model, and acquiring a teaching assistant image output by the virtual dress-up teaching assistant model, calling a second clothing transformation sub-model in the virtual dress-up teaching assistant model, and generating a transformed image in which the clothing to be transformed is adapted to the human body of the designated person based on the human body analysis result and the image features of the first clothing image; 7. The method according to claim 6, further comprising: calling a second dress-up generation sub-model in the virtual dress-up model; and generating the teaching assistant image based on a fusion of a transformed image corresponding to the dress-up target outfit output by the second clothing transformation sub-model with other image areas in the person image excluding an area in which the original outfit is worn.

9. The method comprises:

9. The method according to claim 8, further comprising the step of calling a second dress-up generation sub-model in the virtual dress-up model, and clearing an area in which the specified person contained in the person image is wearing the original clothes based on the human body analysis result, and obtaining other image areas in the person image excluding the area in which the specified person is wearing the original clothes.

10. The step of updating parameters of the virtual dress-up student model of the training subject based on image loss information between the student image and the teacher image by using the person image as a teacher image, Obtaining an image loss value of the student image for the teacher image, the image loss value including at least one of a pixel distance loss function value, a perception loss function value, and an opposition loss function value; performing an addition operation on the image loss value to obtain an image loss sum value of the student image for the teacher image; The method of claim 6 , further comprising: performing parameter updates on the virtual dress-up student model to be trained using the image loss sum value as image loss information between the student image and the teacher image.

11. The method according to claim 2, wherein the first dress-up generation sub-model is composed of an encoder-decoder network and a residual network, and the residual network is used to perform normalization processing on the connected upper layer network.

12. An image processing device, the device being mounted on a computer device, the device comprising: an image capture module configured to capture a first image including a target person and a second image including a target outfit; An information generating module configured to generate a target appearance flow feature used to represent a deformation caused by the target outfit adapting to the target person's body based on the image features of the first image and the image features of the second image, and to generate a post-deformation image in which the target outfit adapts to the human body based on the target appearance flow feature, a multi-layer image feature acquisition unit configured to use the first image as an input signal of a first image feature extraction model and the second image as an input signal of a second image feature extraction model, and to extract multi-layer image features corresponding to the input signals through the first image feature extraction model and the second image feature extraction model, respectively; an appearance flow feature extraction unit configured to extract appearance flow features for each layer based on the multi-layer image features output by the first image feature extraction model and the second image feature extraction model, and to extract the appearance flow features for the last image feature layer and set the obtained appearance flow features as the target appearance flow features; an information generating module, a virtual dress-up module configured to generate a virtual dress-up image based on a fusion of the transformed image and the first image, wherein in the virtual dress-up image, the target person is wearing a target outfit adapted to the human body; 23. An image processing device comprising:

13. An image processing device, the device being mounted on a computer device, the device comprising: an image capture module configured to capture a first image including a target person and a second image including a target outfit; an information generating module configured to generate target appearance flow features used to represent the deformation caused by the target outfit adapting to the target person's body based on the image features of the first image and the image features of the second image, and to generate a post-deformation image in which the target outfit adapts to the human body based on the target appearance flow features, the information generating module being a first outfit deformation sub-model in the virtual dress-up student model; An image processing device comprising: a virtual dress-up module configured to generate a virtual dress-up image based on the fusion of the transformed image and the first image, wherein in the virtual dress-up image, the target person is wearing a target outfit that fits the human body, and the virtual dress-up module is a first dress-up generation sub-model in the virtual dress-up student model.

14. An electronic device, A memory having computer readable instructions stored thereon; and a processor adapted to read computer readable instructions stored in a memory to perform the method of any one of claims 1 to 11.

15. A computer program arranged to carry out the method according to any one of claims 1 to 11.

Citation Information

Patent Citations

  • Trial fitting simulation method

    JP2005216094A

  • Image adjustment method, apparatus, electronic device, storage medium and program

    JP2021108206A