Try-on image generation method and generation system, electronic device, and storage medium

Abstract

Provided in the embodiments of the present application are a try-on image generation method and generation system, an electronic device, and a medium. The try-on image generation method comprises: acquiring the category of a target clothes and try-on image generation configuration information input by a user, and a user-uploaded image; in response to the user selecting a system dress form to generate a model try-on image, acquiring a target dummy model image, and model posture information of the target dummy model image; on the basis of the user-uploaded image and the target dummy model image, synthesizing a clothes canvas image; and on the basis of the clothes canvas image, the model posture information, the category and the try-on image generation configuration information, generating a first try-on image generation task, so that a preset server end generates a model try-on image of the target clothes on the basis of the first try-on image generation task. The method generates model try-on images on the basis of model postures selected by users, thereby improving the quality of model try-on images.

Description

Try-on image generation method, generation system, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 27, 2023, with application number 202311270584.X and application name “Trying-on Image Generation Method, Generation System, Electronic Device and Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a fitting image generation method, a fitting image generation system, an electronic device, and a storage medium. Background Art

[0003] E-commerce websites provide e-commerce services, allowing users from various countries and regions to purchase products from various countries on cross-border e-commerce websites. Product images provide an intuitive and important channel for presenting product information to users. For example, for clothing, images of clothing being worn on a mannequin can intuitively display information such as the color, style, and fit of the garment. Therefore, displaying these images on e-commerce websites is particularly important for clothing products. However, due to practical factors such as the high cost of photographing actual models, many merchants often resort to simple product flatlay images or highly similar online images when listing their products. Prior art solutions have emerged for generating clothing try-on images based on AIGC (Artificial Intelligence Generated Content) image generation technology. These solutions require merchants to upload images of the clothing to be tried on. The system then automatically generates clothing try-on images based on the uploaded images. However, the quality of the images uploaded by merchants varies greatly, with variations in shooting angle, image quality, and mannequin integrity, which directly impacts the quality of the generated try-on images.

[0004] In summary, the existing method for generating try-on images needs to be improved.

[0005] Summary of the Invention

[0006] The embodiment of the present application provides a fitting image generation method, which can be used to generate fitting images of clothing including clothing, accessories, bags, shoes and hats, etc., and improve the quality of the generated clothing fitting images.

[0007] Correspondingly, the embodiments of the present application also provide a fitting image generation system, an electronic device and a storage medium to ensure the implementation and application of the above method.

[0008] In order to solve the above problems, the present application discloses a method for generating a try-on image, which is applied to a client. The method includes:

[0009] Obtain the category of the target clothing entered by the user and the configuration information for generating a try-on image, as well as the image uploaded by the user;

[0010] In response to a user selecting a system mannequin to generate a model fitting image, a target mannequin image and model posture information of the target mannequin image are acquired;

[0011] synthesizing a clothing canvas image based on the user-uploaded image and the target mannequin image;

[0012] Based on the clothing canvas image, the model posture information, the category and the fitting image generation configuration information, a first fitting image generation task is generated, so that the preset server generates a model fitting image of the target clothing based on the first fitting image generation task.

[0013] The present application also discloses a method for generating a try-on image, which is applied to a server. The method includes:

[0014] Obtaining the target clothing category, model pose information, try-on image generation configuration information, and clothing canvas image corresponding to the first try-on image generation task;

[0015] Performing image segmentation processing on the clothing canvas image to obtain a clothing mask image;

[0016] Based on the model posture information, obtaining a preset real-life model image;

[0017] Based on the category, the fitting image generation configuration information, the clothing canvas image, the real model image and the clothing mask image, a preset image generation service is called to generate a model fitting image of the target clothing.

[0018] The present application discloses a method for generating a try-on image, which is applied to a server. The method includes:

[0019] Obtain the target clothing category, try-on image generation configuration information, and user-uploaded images corresponding to the second try-on image generation task;

[0020] Performing image segmentation processing on the user-uploaded image to obtain a clothing canvas image, and performing image segmentation processing on the user-uploaded image to obtain a model mask image;

[0021] Performing image segmentation processing on the clothing canvas image to obtain a clothing mask image;

[0022] Based on the category, the fitting image generation configuration information, the clothing canvas image, the model mask image and the clothing mask image, a preset image generation service is called to generate a model fitting image of the target clothing.

[0023] The present application also discloses a fitting image generation system, which includes:

[0024] A client, configured to execute the method for generating clothing fitting images applied to a client disclosed in the embodiments of the present application;

[0025] The server is used to execute the clothing fitting image generation method applied to the server disclosed in the embodiments of the present application.

[0026] An embodiment of the present application also discloses an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described in the embodiment of the present application.

[0027] The embodiment of the present application further discloses a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in the embodiment of the present application.

[0028] Compared with the prior art, the embodiments of the present application have the following advantages:

[0029] In an embodiment of the present application, when it is impossible to upload a user-uploaded image of a full-limb mannequin or a real-life model wearing the target clothing, the user uploads an image of the clothing to be tried on and configures the category and fitting picture generation configuration information. Afterwards, the user can choose to use the system mannequin to generate a model fitting picture. The client will display a preset mannequin image of at least one model posture for the user to select, and synthesize a clothing canvas image based on the mannequin image selected by the user and the user-uploaded image; afterward, based on the clothing canvas image, the model posture information, the category and the fitting picture generation configuration information, a first fitting picture generation task is generated, so that the preset server generates a model fitting picture of the target clothing based on the first fitting picture generation task. The model fitting picture generated by this method introduces information such as model posture on the basis of retaining the clothing information in the user-uploaded image, effectively improving the quality and display effect of the generated model fitting picture.

[0030] Furthermore, when synthesizing clothing canvas images, users can adjust the matching relationship between clothing and models, making the combination of clothing and models in the generated fitting images more natural and fitting, thereby improving the display effect of clothing in the model fitting images.

[0031] When generating the first fitting image generation task, the model posture information is retained. When the user server executes the first fitting image generation task to generate the model fitting image, the system built-in real-life model image corresponding to the model posture information is further obtained to generate the model fitting image, so that the model's face and bones in the generated fitting image are more natural and beautiful. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a flowchart of one embodiment of a method for generating a try-on image disclosed in the present application;

[0033] FIG2 is a schematic diagram of a user-uploaded image uploaded by a user;

[0034] FIG3 is a schematic diagram of a mannequin image for user selection;

[0035] FIG4 is a schematic diagram of a clothing canvas image generated based on the user-uploaded image shown in FIG2 ;

[0036] FIG5 is a schematic diagram of a try-on image generated based on the clothing canvas image shown in FIG4 ;

[0037] FIG6 is a flowchart of another embodiment of a method for generating a try-on image disclosed in the present application;

[0038] FIG7 is a flowchart of another embodiment of a method for generating a try-on image disclosed in the present application;

[0039] FIG8 is a flowchart of another embodiment of a method for generating a try-on image disclosed in the present application;

[0040] FIG9 is a schematic diagram of an interactive process of a fitting image generation system disclosed in the present application;

[0041] FIG10 is a schematic structural diagram of an exemplary device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0043] Cross-border e-commerce websites serve sellers and buyers in hundreds of countries, so they need to provide services that generate clothing try-on images based on a given garment, tailored to users in different countries. For example, sellers can display images of models trying on clothing in the corresponding national styles for buyers in different countries.

[0044] The fitting image generation method disclosed in the embodiments of the present application can be implemented as a clothing fitting application, which generates corresponding clothing fitting images based on the clothing images uploaded by merchant users and the configured product categories, model styles, model faces, background styles and other information, so as to publish the clothing fitting images on the e-commerce platform for buyer users to browse.

[0045] Among them, clothing includes but is not limited to: clothing, shoes and hats, bags, accessories, etc. In the embodiments of this application, in order to facilitate the reader's understanding, the specific technical solutions of each embodiment are mainly described by taking the scenario of generating clothing try-on pictures based on clothing images as an example. Those skilled in the art should understand that the technical solutions disclosed in the embodiments of this application are also applicable to generating hat try-on pictures based on hat images, generating back try-on pictures based on backpack images, generating try-on pictures based on glasses images, etc. In the embodiments of this application, images generated by models showing wearing effects based on clothing images uploaded by users are collectively referred to as "clothing try-on pictures".

[0046] 1 , in an optional embodiment, a fitting image generation method disclosed in the present application is applied to a client of a clothing fitting application or a clothing fitting system, and the method includes steps 100 to 106 .

[0047] The specific implementation of each step is introduced below.

[0048] Step 100: Obtain the category of target clothing input by the user and configuration information for generating a try-on image, and the user uploads an image.

[0049] Wherein, the category can be a secondary product category on the target clothing e-commerce platform. In some optional embodiments, the category includes but is not limited to one or more of the following: tops, pants, skirts, dresses, etc. The try-on picture generation configuration information is used to describe the image style, model style, etc. of the try-on picture to be generated. The try-on picture generation configuration information is used as a partial prompt word source for generating prompt words for calling a graph generation application or service. In some optional embodiments, the try-on picture generation configuration information includes but is not limited to one or more of the following information: model skin color, model face, background style, etc. Wherein, the background style is used to describe the background style in the generated clothing try-on picture. In some optional embodiments, the background style includes but is not limited to: random background picture, random background color, specified style background picture, etc. The model face is used to describe the model face used in the generated model try-on picture.

[0050] In an embodiment of the present application, the target garment is the garment for which a try-on image is to be generated. To ensure the quality of the generated try-on image, the image of the target garment uploaded by the user through the client's preset image upload portal must include a complete image of the target garment. In an embodiment of the present application, the user-uploaded image of the target garment is referred to as a "user-uploaded image." In some optional embodiments, the client may preset one or more image upload portals, and the user may upload an image of the target garment by triggering the corresponding image upload portal.

[0051] In some optional embodiments, the user-uploaded image may be a flat image of clothing, a stereoscopic image of clothing worn on a mannequin, a stereoscopic image of clothing worn on a dummy model, a stereoscopic image of clothing worn on a real model, and the like.

[0052] In some optional embodiments, the preset image upload entrance includes but is not limited to one or more of the following: an AI (Artificial Intelligence) generated image upload entrance, a local image upload entrance, a media center image selection entrance, and a product link image upload entrance.

[0053] Optionally, a fitting image generation method disclosed in the present application can be implemented as a network application including: a client and a server. The client is provided with an interactive interface for interacting with the user, and the server user performs information processing based on the information obtained through the client to obtain a clothing fitting image. Optionally, the server can be implemented as one module or multiple modules according to application development and deployment requirements, and the one or more modules can be deployed in one or more servers respectively. For example, in an optional embodiment, the user uploads an image of the target clothing for which a clothing fitting image is to be generated through the client, and sets the product category of the target clothing, and sets the fitting image generation configuration information, etc. Correspondingly, the server obtains the image uploaded by the user and the information set by the user through the client.

[0054] For example, in some optional embodiments, the try-on image generation application or system is pre-configured with one or more model faces trained using image generation technology, and each model face is assigned a facial name. Accordingly, the client can display a portal for selecting a model face. After the user triggers this portal, the user is presented with the model faces pre-configured in the application or system for selection. After the user selects a specific model face, the model face information of the selected model face is used as the try-on image generation configuration information. The model face information can be a model name or other identification information.

[0055] In some optional embodiments, the AI-generated image upload portal can be in the form of an icon, hyperlink, or other form. Specifically, for example, the AI-generated image upload portal can be an icon set on the client page. After the user triggers the icon, the client obtains the prompt word entered by the user for generating an image of the target clothing item. Based on the prompt word entered by the user, the client invokes a preset generative model to generate the clothing item image. The client then displays the clothing item image generated by the preset generative model and obtains the clothing item image selected by the user as the user-uploaded image of the target clothing item. The preset generative model is a conventional generative model capable of generating images based on text prompts.

[0056] In some optional embodiments, the local image upload entry can be an interface-based image upload entry, a command-line image upload entry, or other image upload entry methods known in the art. Specifically, for example, the local image upload entry can be an image upload control provided on a client page. A user can trigger the image upload control, select an image of a target garment for which a fitting image is to be generated, and perform an upload operation. In this way, the client can retrieve the user-uploaded image through the image upload control and use it as the user-uploaded image of the target garment.

[0057] In some optional embodiments, the media center image selection portal is implemented as an icon, hyperlink, or other form. Specifically, for example, the media center image selection portal can be a hyperlink set on a client page. After the user triggers the hyperlink, the client accesses the page of the preset media center corresponding to the hyperlink based on the user's triggering. The page of the preset media center displays public product images, and the user can select an image from the page. The preset media center can then feed the user-selected image back to the client. In this way, the client can use the image fed back by the preset media center as the user-uploaded image of the target apparel.

[0058] In some optional embodiments, the product link image transmission entry can be implemented in the form of an edit box or the like. Specifically, for example, the product link image transmission entry can be an edit box set up on the client page, into which the user can enter the product link of a product already published on the e-commerce platform. The client obtains the product link entered in the edit box, parses it to obtain the product identifier contained in the product link, and then queries the product details associated with the product identifier in the product center of the e-commerce platform, and displays a list of the retrieved product detail images to the user. The user can select a product detail image from the displayed list. In this way, the client uses the product detail image selected by the user as the user-uploaded image of the target clothing item.

[0059] In the embodiments of this application, by providing multiple clothing image upload methods as described above, user convenience is effectively improved. For example, merchants who have already taken product photos can directly upload the captured clothing images to generate clothing try-on images. Merchants who have not taken product photos can directly use photos of already released products. For unreleased products, clothing images can also be uploaded by selecting images from the media center. Furthermore, in certain image upload methods, the product category of the target clothing item can be automatically obtained, providing greater convenience for users.

[0060] In terms of technical support, in the embodiments of the present application, by connecting the various product modules of the e-commerce platform and making full use of the product data of the e-commerce platform, greater operational convenience is provided to users.

[0061] Step 102 : in response to the user selecting a system mannequin to generate a model fitting image, a target mannequin image and model posture information of the target mannequin image are acquired.

[0062] In actual applications, when user-uploaded images are flat-lay images of clothing, the model's pose information is missing from the uploaded images. Furthermore, when user-uploaded images are three-dimensional images of clothing worn by a mannequin, due to limited shooting conditions, the mannequin's limbs are often missing from the uploaded images, as shown in the user-uploaded image in Figure 2. These user-uploaded images often lack model pose information. In such situations, existing try-on image generation applications are unable to generate high-quality try-on images.

[0063] In an embodiment of the present application, in order to improve the quality of the generated try-on images, a mannequin selection portal is provided on the client side, allowing the user to choose to use the model pose and clothing wearing state in the user-uploaded image uploaded by the merchant to generate the clothing try-on image, or to generate the clothing try-on image by configuring the model pose and clothing wearing state through the client side. For example, in the system mannequin option and the merchant mannequin option provided by the client side, if the user selects the system mannequin option, the client side will start the model pose and clothing wearing state configuration process, and when the clothing try-on image is generated on the server side, the model pose and clothing wearing state finally configured by the user will be retained; if the user selects the merchant mannequin option, then when the clothing try-on image is generated on the server side, the model pose and clothing wearing state in the user-uploaded image will be retained.

[0064] In some optional embodiments, in response to a user selecting a system mannequin to generate a model fitting image, obtaining a target mannequin image and model pose information of the target mannequin image, including: in response to a user selecting a system mannequin to generate a model fitting image, displaying a mannequin image of at least one model pose; obtaining a target mannequin image selected by the user, and model pose information of the target mannequin image.

[0065] For example, when a user chooses to use a system mannequin to generate a model fitting diagram, the client reads the built-in mannequin images of different postures of the system (such as the fitting diagram generation system), as well as the model posture information corresponding to each mannequin image, and displays the mannequin images to the user, as shown in Figure 3, for the user to choose. In some optional embodiments, the system can pre-set mannequin images of multiple postures for display to the user for selection, and has built-in real-life model images with the same posture as each mannequin image. Wherein, the real-life model images are used as the image source for extracting postures when the server generates the fitting diagram. Each mannequin image corresponds to a model posture, and each model posture corresponds to a real-life model image. In the embodiment of the present application, the model posture corresponding to the mannequin image is represented by the model posture information.

[0066] Afterwards, the mannequin image selected by the user is used as the target mannequin image, and the mannequin posture information of the target mannequin image is obtained.

[0067] In some optional embodiments, the user can select a mannequin image or multiple mannequin images as target mannequin images for subsequent generation of fitting images. That is, the user can select to generate a fitting image of the mannequin in one posture or multiple postures.

[0068] The client records the mannequin posture information corresponding to the target mannequin image according to the mannequin image selected by the user.

[0069] Step 104 : synthesizing a clothing canvas image based on the user-uploaded image and the target mannequin image.

[0070] After the user selects a mannequin image built into the system, the client further performs image synthesis processing on the image uploaded by the user and the mannequin image selected by the user to obtain a clothing canvas image.

[0071] For each target mannequin image selected by the user, the client performs image synthesis processing on the target mannequin image and the image uploaded by the user to obtain a clothing canvas image corresponding to each target mannequin image.

[0072] In some optional embodiments, the client can call an image segmentation service or application in the prior art to perform image segmentation processing on the user-uploaded image according to the category of the target clothing, and segment out an image that only includes the target clothing, which is recorded as a "clothing image". The client can identify the body parts in the mannequin image through human body recognition technology. Then, using the mannequin image as a canvas and the clothing image as a clothing layer on the canvas, the corresponding body part of the target clothing is determined according to the category of the target clothing, and based on the determined wearing body part, the display position and size of the clothing layer on the canvas is determined, so that the clothing image in the clothing layer just covers the wearing body part, achieving the visual effect of the clothing image being worn on the wearing body part. Finally, based on the display position and size of the canvas and the determined clothing layer on the canvas, the canvas and the clothing layer are synthesized to obtain an image of the mannequin wearing the target clothing, which is recorded as a "clothing canvas image" in the embodiment of the present application.

[0073] In other optional embodiments, a clothing canvas image is synthesized based on the user-uploaded image and the target mannequin image, including: creating a canvas based on the target mannequin image; performing clothing segmentation processing on the user-uploaded image to obtain a clothing image of the target clothing; creating a clothing layer based on the clothing image; based on the user's adjustment operation on the display information of the clothing layer in the canvas, displaying the adaptation result of the target clothing in the clothing layer and the mannequin in the target mannequin image in real time; based on the user's confirmation operation on the adaptation result, synthesizing the clothing layer and the canvas according to the adjusted display information to obtain a clothing canvas image.

[0074] The display information of the clothing layer in the canvas includes but is not limited to one or more of the following information: size, position, and rotation angle.

[0075] Optionally, after the user selects a mannequin image, the client may create a window and use the mannequin image selected by the user (ie, the target mannequin image) as the canvas of the window.

[0076] Optionally, the client invokes a segmentation service interface preset by the server, causing the server to perform image segmentation processing on the user-uploaded image to obtain a clothing image that includes only the target item of clothing. The clothing image has a transparent background, with the target item of clothing being the foreground. For example, performing clothing segmentation processing on the user-uploaded image to obtain the clothing image of the target item of clothing may include: triggering the server to perform clothing segmentation processing on the user-uploaded image to obtain the clothing image of the target item of clothing.

[0077] Afterwards, the client uses the clothing image as the clothing layer and draws the clothing layer on the upper layer of the canvas in the window to obtain a composite image of the mannequin image and the target clothing.

[0078] Since the mannequin images built into the system are of uniform size, while the sizes of the target clothing in the user-uploaded images are not uniform, it is possible that the target clothing and the mannequin sizes do not match. In addition, due to the placement of the target clothing in the user-uploaded images, the shooting angle, and other factors, the wearing posture of the target clothing on the mannequin in the synthesized image may be stiff and unnatural. In order to improve the display effect of the generated fitting image on the clothing, in an embodiment of the present application, the client supports the user to adapt the target clothing and the mannequin by adjusting the clothing layer, and by dragging the clothing layer, the wearing status of the target clothing on the mannequin in the mannequin image can be adjusted.

[0079] The adjustment operation includes: dragging operation or inputting display parameters of the clothing layer.

[0080] In some optional embodiments, the client can obtain the user's adjustment information on the clothing layer by detecting the mouse drag event on the clothing layer, for example, obtaining the horizontal or vertical rotation angle of the clothing layer, obtaining the zoom ratio of the clothing layer and other information, and refresh the clothing layer in the canvas based on the above adjustment information obtained to show the adjustment effect in real time.

[0081] In some optional embodiments, the client can set a clothing layer display parameter configuration interface, and the user can configure the display information of the clothing layer on the canvas as adjustment information through the display parameter configuration interface, and refresh the display of the clothing layer in the canvas according to the actual adjustment information obtained to display the adjustment effect in real time.

[0082] Finally, the user can confirm the final adjustment results by submitting the confirmation. After the user confirms the adaptation results, the client synthesizes the clothing layer and the canvas according to the adjusted display information to obtain the clothing canvas image. For detailed implementation of synthesizing the clothing layer and the canvas to obtain the clothing canvas image, please refer to the prior art techniques for synthesizing two or more layers to generate an image, which will not be further described here. Taking the user-uploaded image shown in Figure 2 as an example, after synthesis processing, the clothing canvas image shown in Figure 4 can be obtained.

[0083] In an embodiment of the present application, after the user chooses to generate a fitting image based on a system mannequin, the clothing layer is obtained by segmenting the image uploaded by the user, and the built-in mannequin model image selected by the user is used as the canvas. The user is supported to freely adjust the position, size, and angle of the clothing layer in the canvas to make the clothing fit the model's body, thereby improving the effect of the generated fitting image.

[0084] In some embodiments of the present application, the server performs clothing segmentation processing on the user-uploaded image to obtain a clothing image of the target clothing, including: calling a preset target detector based on a first preset semantic prompt word to perform target detection on the user-uploaded image to obtain position information of the target clothing in the user-uploaded image; performing image segmentation processing on the user-uploaded image based on the position information to obtain a clothing mask image that masks the target clothing; performing cutout processing on the user-uploaded image based on the clothing mask image to obtain a cutout image; and using a pixel mapping method and a Gaussian blur processing method to optimize the edge area of ​​the cutout image to obtain a clothing image.

[0085] Grounding DINO (an open-source object detector) is an open-set object detection algorithm. By specifying the semantics of the object to be segmented, Grounding DINO can identify the location of the target element in the image. For example, to segment clothing in an image, you can use "clothes" as the first predefined semantic cue word and a user-uploaded image as input. Based on this predefined semantic cue word, Grounding DINO will identify the location of the clothing element in the user-uploaded image.

[0086] In some optional embodiments, a Segment Anything (an image segmentation algorithm) algorithm may be used to further segment a mask image corresponding to a position in the user-uploaded image based on the position information output by Grounding DINO, such as a clothing mask image.

[0087] Next, an algorithm can be used to perform pixel mapping on the user-uploaded image based on the clothing mask image to obtain a cutout image. Existing techniques for cutouts based on mask images have the following drawbacks: first, white edges on the object; second, severe jagged edges on the object, requiring further refinement.

[0088] In some optional embodiments, a pixel mapping method and a Gaussian blur processing method are used to optimize the edge region of the cutout image to obtain a clothing image, including: determining the edge region of the target clothing based on the pixel values ​​of the alpha channel of the cutout image; performing N-th power pixel mapping in the edge region to reduce the edge region to obtain a white-edge-removed image; and using a Gaussian blur algorithm to blur the cutout boundary of the white-edge-removed image to obtain the clothing image. Where N is an integer greater than 2.

[0089] The alpha channel pixel values ​​of the extracted image are used to identify the main clothing area, clothing edge area, and background area. An N-th power mapping function is then applied to the edge area to remove pixel values. This reduces the white edge pixel range to a smaller size, making it imperceptible to the naked eye and thus reducing the edge area. A Gaussian blur algorithm is then used to reduce edge jaggedness, blurring the edge area to a certain degree. This results in an optimized cutout boundary, ultimately yielding the target clothing image.

[0090] By performing edge mapping on the alpha channel pixel values ​​to reduce the white edge area, and then using the Gaussian blur algorithm to blur the cutout boundary, the jagged edges of the cutout boundary can be reduced, further improving the quality of the clothing image obtained by cutout.

[0091] In other embodiments of the present application, other cutout methods in the prior art may be used to perform clothing segmentation processing on the user uploaded image to obtain a clothing image of the target clothing, which will not be described in detail in the embodiments of the present application.

[0092] Step 106: Generate a first fitting image generation task based on the clothing canvas image, the model posture information, the category, and the fitting image generation configuration information, so that the preset server generates a model fitting image of the target clothing based on the first fitting image generation task.

[0093] Next, the client can submit a first fitting image generation task based on the clothing canvas image, the target clothing category, and the fitting image generation configuration information input by the user. In this way, the server can perform the corresponding fitting image generation operation based on the first fitting image generation task.

[0094] In some optional embodiments, if the user selects multiple mannequin images, the client will submit a first fitting image generation task for each mannequin image, that is, generate a set of model fitting images based on the model posture in each mannequin image.

[0095] In some optional embodiments, a first fitting image generation task can be generated based on the clothing canvas image, the model posture information, the category, and the fitting image generation configuration information, so that the preset server generates a model fitting image of the target clothing based on the first fitting image generation task, including: using the image data of the clothing canvas image as the clothing canvas image information, and generating the first fitting image generation task based on the clothing canvas image information, the model posture information, the category, and the fitting image generation configuration information, so that the preset server generates a fitting image of the target clothing based on the first fitting image generation task. In this way, the data carried by the first task data will include: the image data of the clothing canvas image, the model posture information, the category and fitting image generation configuration information corresponding to the target clothing input by the user through the client, and the task identifier.

[0096] The configuration information for generating the try-on image includes but is not limited to information such as model skin color, model face, and background style.

[0097] For example, the user selects a mannequin image, and after confirming the generated clothing canvas image, the category and the fitting image generation configuration information, clicks the button set in the client interface to submit the generation task. The client detects the user's operation, uses the clothing canvas image, the category and the fitting image generation configuration information as task data, and submits it to the preset server to generate a first fitting image generation task.

[0098] Optionally, the client can submit the first fitting image generation task to the server by sending a task message to a task execution message queue, wherein the task message carries the task data. Optionally, the client can also submit the first fitting image generation task to the server by sending a task request to a preset server, wherein the task request carries the task data.

[0099] In some optional embodiments, based on the clothing canvas image, the model posture information, the category and the fitting picture generation configuration information, a first fitting picture generation task is generated, so that the preset server generates a model fitting picture of the target clothing based on the first fitting picture generation task, including: uploading the clothing canvas image to a preset storage end, obtaining the storage address of the clothing canvas image as the clothing canvas image information; based on the clothing canvas image information, the model posture information, the category and the fitting picture generation configuration information, a first fitting picture generation task is generated, so that the preset server generates a model fitting picture of the target clothing based on the first fitting picture generation task.

[0100] For example, after a user selects a mannequin image and confirms the generated clothing canvas image, the category, and the try-on image generation configuration information, they click a button on the client interface to submit the task. The client detects this user action and first uploads the clothing canvas image to a preset storage end, such as a preset storage server, and obtains the storage address of the clothing canvas image returned by the preset storage end. The client then submits a first try-on image generation task to the preset server, using the storage address of the clothing canvas image, the mannequin pose information, the category, and the try-on image generation configuration information as task data.

[0101] By uploading the clothing canvas image to the preset storage end and then downloading it for use when executing the fitting image generation task on the server end, the flexibility of using the clothing canvas image can be improved, the amount of task data can be reduced, and the system stability can be improved.

[0102] Among them, the preset server generates a model fitting image of the target clothing based on the first fitting image generation task, including: obtaining the category of the target clothing corresponding to the first fitting image generation task, model posture information, fitting image generation configuration information and clothing canvas image; performing image segmentation processing on the clothing canvas image to obtain a clothing mask image; based on the model posture information, obtaining a preset real-life model image; based on the category, the fitting image generation configuration information, the clothing canvas image, the real-life model image and the clothing mask image, calling the preset image generation service to generate a model fitting image of the target clothing.

[0103] The server executes the first fitting image generation task and generates a fitting image of the target garment based on the task data corresponding to the first fitting image generation task. The specific implementation of each step is described in the relevant embodiments below and is not repeated here. For the garment canvas image shown in Figure 4 , the server can generate a fitting image of a model, as shown in Figure 5 , where the model is an AI-generated, real-life model image.

[0104] In some optional embodiments, the server can upload the generated model fitting image to a pre-set storage device for storage and send a notification message to the server to inform it of the completion of the first fitting image generation task. In this way, the server can further push the generated model fitting image to the corresponding client based on the client's query request or proactively.

[0105] Optionally, the first fitting image generation task can be marked with a task identifier to store and query the model fitting image generated by the first fitting image generation task. For example, after the client submits the first fitting image generation task to the server, the server assigns a task identifier to the first fitting image generation task and binds the client's current user account to the task identifier. The execution result of the first fitting image generation task is stored by marking it with the task identifier. The notification message associated with the task is also marked with the task identifier of the task. In this way, the generated model fitting image can be queried and pushed through the task tag.

[0106] In other embodiments, the execution order of step 100 and step 102 can be interchanged. In the embodiment of the present application, there is no restriction on the order in which the user selects the mannequin type, uploads the user-uploaded image, configures the category, and generates the configuration information for the try-on image. Accordingly, there is no restriction on the order in which the client obtains the mannequin image, uploads the user-uploaded image, and generates the configuration information for the category and the try-on image.

[0107] 6 , in an optional embodiment, based on the aforementioned embodiment, after obtaining the category of the target clothing input by the user and the configuration information generated by the fitting image, and after the user uploads the image, the method further includes: step 101 and step 108 .

[0108] Step 101 , determine the type of avatar selected by the user. In response to the user selecting a merchant avatar, jump to step 109 . In response to the user selecting a system avatar, jump to step 102 .

[0109] In some optional embodiments, the avatar type selection entry set in the client interface includes: system avatar and merchant avatar, and the client and server will perform subsequent interactions based on the different avatar types selected by the user.

[0110] For example, when the user selects a system avatar, steps 102 to 106 are executed; when the user selects a merchant avatar, step 108 is executed.

[0111] Step 108, in response to the user selecting a merchant mannequin to generate a model fitting picture, a second fitting picture generation task is generated based on the user uploaded image, the category and the fitting picture generation configuration information, so that the preset server generates a model fitting picture of the target clothing based on the second fitting picture generation task.

[0112] If the user chooses to use a merchant avatar to generate a model fit image, the client directly generates a second fit image generation task based on the category of the user-uploaded image and the fit image generation configuration information. The specific implementation of generating the second fit image generation task based on the user-uploaded image, the category, and the fit image generation configuration information can be found in the specific implementation of generating the first fit image generation task based on the garment canvas image, the category, and the fit image generation configuration information, and will not be repeated here.

[0113] It should be noted that, unlike the task data corresponding to the first fitting image generation task, the source image used to generate the model fitting image in the task data corresponding to the second fitting image generation task is the image uploaded by the merchant, and the task data corresponding to the second fitting image generation task does not include model posture information.

[0114] In summary, when it is necessary to generate a clothing fitting picture, the user uploads an image of the clothing to be tried on and configures the category and fitting picture generation configuration information. Afterwards, the user can choose to use the system mannequin to generate a model fitting picture. The client will display a preset mannequin image of at least one model posture for the user to choose, and synthesize a clothing canvas image based on the mannequin image selected by the user and the user uploaded image; afterward, based on the clothing canvas image, the model posture information, the category and the fitting picture generation configuration information, a first fitting picture generation task is generated, so that the preset server generates a model fitting picture of the target clothing based on the first fitting picture generation task. The model fitting picture generated by this method introduces information such as model posture on the basis of retaining the clothing information in the user uploaded image, effectively improving the quality and display effect of the generated model fitting picture.

[0115] Furthermore, when synthesizing clothing canvas images, users can adjust the matching relationship between clothing and models, making the combination of clothing and models in the generated fitting images more natural and fitting, thereby improving the display effect of clothing in the model fitting images.

[0116] When generating the first fitting image generation task, the model posture information is retained. When the user server executes the first fitting image generation task to generate the model fitting image, the system built-in real-life model image corresponding to the model posture information is further obtained to generate the model fitting image, so that the model's face and bones in the generated fitting image are more natural and beautiful.

[0117] Based on the above embodiments, the present application also provides a clothing fitting image generation method for performing a first fitting image generation task to generate a model fitting image based on the clothing canvas image, model posture information, category, fitting image generation configuration information, etc. corresponding to the first fitting image generation task.

[0118] 7 , the fitting image generation method disclosed in this embodiment is applied to a fitting image generation system or a server of an application. The method includes steps 702 to 708 .

[0119] Step 702 : Obtain the target clothing category, model posture information, try-on image generation configuration information, and clothing canvas image corresponding to the first try-on image generation task.

[0120] In some optional embodiments, the target clothing category, model pose information, fitting image generation configuration information, and clothing canvas image corresponding to the first fitting image generation task can be obtained through task data. The task data can be stored in a preset task message queue using a task identification tag. In other optional embodiments, the task data can be transmitted using interface parameters of a preset service. In the embodiments of the present application, there is no limitation on the transmission method of the task data.

[0121] The category may be a secondary product category of the target clothing on an e-commerce platform. The fitting image generation configuration information is set by the user for the target clothing, and includes but is not limited to information such as model skin color, model face, and background style. The clothing canvas image is an image with a mannequin of the target clothing as the foreground and a transparent background. The clothing canvas image is a composite image based on a user-uploaded image and a mannequin image in a posture selected by the user from a mannequin image pre-set by the system or application. The model posture information is used to identify the mannequin image used to synthesize the clothing canvas image.

[0122] In some optional embodiments, if the fitting image generation configuration information included in the task data corresponding to the first fitting image generation task is a clothing canvas image, the server can directly obtain the image data of the clothing canvas image from the task data. In some optional embodiments, if the fitting image generation configuration information included in the task data corresponding to the first fitting image generation task is the storage address of the clothing canvas image, the server needs to download the clothing canvas image from the storage address.

[0123] The method for generating the first fitting image generation task and the specific method for generating task data are described in the previous embodiment and will not be repeated here.

[0124] Step 704 : performing image segmentation processing on the clothing canvas image to obtain a clothing mask image.

[0125] In some optional embodiments, performing image segmentation processing on the clothing canvas image to obtain a clothing mask image includes: invoking a preset object detector based on a first preset semantic cue word, performing object detection on the clothing canvas image, and obtaining position information of the target clothing item in the clothing canvas image; and performing image segmentation processing on the clothing canvas image based on the position information to obtain a clothing mask image that masks the target clothing item. For example, "clothes" (i.e., clothing) can be used as the first preset semantic cue word, and Grounding DINO can be invoked based on the first preset semantic cue word, with the clothing canvas image as the input image. Grounding DINO will then identify the position information of the clothing element in the clothing canvas image.

[0126] In some optional embodiments, a Segment Anything (an image segmentation algorithm) algorithm can be used to further segment the mask image corresponding to the position in the clothing canvas image based on the position information output by Grounding DINO, that is, to obtain a clothing mask image.

[0127] Step 706: Acquire a preset real-life model image based on the model posture information.

[0128] As previously mentioned, the model pose information corresponds to a live-action model image built into the try-on image generation application or system. Therefore, by retrieving the built-in live-action model image based on the model pose information, the live-action model image corresponding to the model pose information can be obtained. For example, if the built-in live-action model image and the mannequin image are named based on the model pose type, and the user selects a system mannequin to generate a model try-on image and further selects a mannequin image in pose A, the model pose information can be represented as pose A. In this step, the built-in live-action model image is retrieved based on pose A to obtain the live-action model image in pose A.

[0129] Step 708 : Based on the category, the fitting image generation configuration information, the clothing canvas image, the real model image, and the clothing mask image, a preset image generation service is called to generate a model fitting image of the target clothing.

[0130] Next, based on the category and the fitting image generation configuration information, a prompt word is generated, and the clothing canvas image, the real model image and the clothing mask image are used as input images. The preset image generation service is called to generate a model fitting image of the target clothing.

[0131] The Stable Diffusion large model has the ability to generate images based on prompt words. Stable Diffusion can be combined with other models to generate images with specified effects. ControlNet (control network) is a new neural network concept that controls pre-trained large models such as Stable Diffusion through additional inputs. ControlNet is used to control the generated image content and currently supports ControlNet models such as Inpaint (image restoration) and OpenPose (pose). For example, the Stable Diffusion large model combined with inpainting (image restoration model) implements Stable Diffusion inpainting (stable diffusion restoration), which is an efficient, robust and high-quality image restoration algorithm. It combines the algorithm based on partial differential equations and interpolates the missing parts through diffusion and gradient descent to achieve image restoration. Combined with the positive and negative prompt words of Stable Diffusion, the expected image effect can be achieved.

[0132] In some optional embodiments, images of real models are generated based on Stable Diffusion inpainting technology. Optionally, the preset image generation service integrates the following control models through a preset generative large model: a first model for controlling the details of the generated image, a second model for controlling the outline details of the generated image, and a third model for controlling the pose of the model in the generated image. The preset generative large model may be a Stable Diffusion large model. The first model may be an Inpainting model, the second model may be a SoftEdge preprocessing model, and the third model may be an OpenPose model. The first, second, and third models may be pre-trained models from the prior art. Subsequently, the first, second, and third models are integrated according to the model loading specifications of the Stable Diffusion large model to obtain the preset image generation service. The generation method of the preset image generation service refers to the prior art application technology combining the Stable Diffusion large model and the ControlNet model, and will not be further described here.

[0133] Furthermore, for the optimization of the generation effect in Stable Diffusion, the embodiment of the present application uses the ADetailer face_yolov8s.pt model (face optimization model) to optimize the model's face and the hand_yolov8n.pt model (a hand optimization model) to optimize the hands, so as to generate a model fitting picture with more delicate facial and hand details.

[0134] Correspondingly, the preset image generation service is called based on the category, the fitting image generation configuration information, the clothing canvas image, the real-life model image and the clothing mask image to generate the model fitting image of the target clothing, including: generating target prompt words based on the category, the fitting image generation configuration information and the preset prompt words; using the clothing canvas image and the clothing mask image as inputs of the first model, using the clothing canvas image and the real-life model image as inputs of the second model, and using the real-life model image as input of the third model, in combination with the target prompt words, calling the preset image generation service to generate the model fitting image of the target clothing.

[0135] Prompts are used to control the image style generated by the Stable Diffusion model. The user-entered configuration information for fitting images is used to control the style of the model fitting images. Therefore, a portion of prompts needs to be generated based on this configuration information. Furthermore, the fitting image generation system or application pre-sets trained universal positive prompts for all supported clothing categories. These can be used as part of the prompts to improve the quality and stability of the generated model fitting images.

[0136] In some optional embodiments, the generation of target prompt words based on the categories, the try-on pictures and the configuration information and preset prompt words includes: obtaining the first prompt word corresponding to the category based on the preset prompt word for each preset category; generating the second prompt word based on the try-on picture generation configuration information; and splicing the first prompt word and the second prompt word to generate the target prompt word. The preset prompt words for each preset category are obtained through repeated training, and the embodiments of the present application do not limit the preset prompt words for each category. As mentioned above, the try-on picture generation configuration information includes but is not limited to one or more of the following information: model skin color, model face, and background style. According to the try-on picture generation configuration information, the second prompt word that describes the model skin color, model face, and background style of the try-on picture in the generated model try-on picture can be respectively generated. Afterwards, the first prompt word and the second prompt word are spliced ​​in the specified order to generate the target prompt word.

[0137] In some embodiments of the present application, obtaining the first prompt word corresponding to each preset category based on the preset prompt word includes any of the following: when the category is tops, obtaining the first prompt word for generating bottoms based on the preset prompt word for each preset category; when the category is skirts or pants, obtaining the first prompt word for generating tops based on the preset prompt word for each preset category; when the category is dresses, obtaining the first prompt word for not generating matching clothing based on the preset prompt word for each preset category. Taking the preset categories including tops, skirts, dresses, and pants as an example, the preset prompt words can include the following prompt words: for tops, the preset prompt words include prompt words generated for bottoms; for skirts / pants, the preset prompt words include prompt words generated for tops; for dresses, the preset prompt words include negative prompt words for not generating inner wear or bottoms. By setting prompt words describing matching information for preset categories, clothing matching can be achieved in model try-on pictures, further improving the user experience.

[0138] In an embodiment of the present application, it is necessary to use a third model (such as ControlNet's OpenPose preprocessor) to generate a skeleton diagram from a real-life model image, so that the skeleton diagram can be used as part of the input data of the Stable Diffusion large model, so that the model pose in the generated model fitting picture is natural. In addition, it is necessary to use the clothing canvas image and the clothing mask image as input images of the first model, and use the first model (such as ControlNet's SoftEdge preprocessor) to limit the approximate area of ​​the model's body and clothing. At the same time, the clothing canvas image and the real-life model image are used as inputs of the second model (such as ControlNet's Inpainting preprocessor) to restore the photo to the greatest extent and finely control the clothing material details and edge areas.

[0139] Stable Diffusion's image-generating service can be called via HTTP (Hypertext Transfer Protocol) in JSON format. The mapping of parameters in JSON follows the image-generating application standard. For example, when a clothing canvas image and a clothing mask image are required as input to the Inpating model, the value of the init_images field in the JSON field can be set to the clothing canvas image converted to a base64-encoded string, and the value of the mask field in the JSON field can be set to the clothing mask image converted to a base64-encoded string. Similarly, convert other images into base64-encoded strings and assign values ​​to the corresponding fields to obtain JSON text.

[0140] Afterwards, the target prompt word and JSON text are used as input for Stable Diffusion's image generation service, which then begins generating images based on the input text. During the image generation phase, Stable Diffusion uses ControlNet's preprocessors, OpenPose and SoftEdge, to control the model's pose and presentation, and utilizes the ADetailer plugin to optimize the model's face and hands. This results in a high-quality model fitting image, with the target garment, the model's skin tone, face, and background style matching the user's configuration, and the model's expression and pose appearing natural.

[0141] In summary, the fitting image generation method disclosed in the embodiment of the present application, after obtaining the category, model posture information, fitting image generation configuration information and clothing canvas image of the target clothing corresponding to the first fitting image generation task, first performs image segmentation processing on the clothing canvas image to obtain a clothing mask image, and obtains a preset real-life model image based on the model posture information; then, based on the category, the fitting image generation configuration information, the clothing canvas image, the real-life model image and the clothing mask image, calls the preset image generation service to generate a model fitting image of the target clothing, which not only retains the original appearance of the clothing in the user-uploaded image, but also incorporates the model skin color, model face and background style customized in the fitting image generation configuration information set by the user, as well as the model posture selected by the user, thereby improving the quality of the generated model fitting image. In addition, by selecting the system-preset real-life model image instead of the mannequin model image, it can ensure that the hand and facial details are retained to the greatest extent, making the facial and hand details of the generated model fitting image more natural and the generated image quality more stable.

[0142] Furthermore, by pre-setting matching prompts for each category and outputting matching try-on images, the visual effects of the model try-on images are improved, enhancing the user experience. Furthermore, by pre-setting prompts for each category, the stability of the generated model try-on images can be improved.

[0143] Based on the above embodiments, the present application also provides a clothing fitting image generation method for performing a second fitting image generation task to generate a model fitting image based on user uploaded images, categories, fitting image generation configuration information, etc. corresponding to the second fitting image generation task.

[0144] 8 , the fitting image generation method disclosed in this embodiment is applied to a fitting image generation system or a server of an application. The method includes steps 802 to 808 .

[0145] Step 802: Obtain the target clothing category, try-on image generation configuration information, and user-uploaded images corresponding to the second try-on image generation task.

[0146] In some optional embodiments, the target clothing category, try-on image generation configuration information, and user-uploaded image corresponding to the second try-on image generation task can be obtained through task data. The task data can be stored in a preset task message queue using a task identification tag. In other optional embodiments, the task data can be transmitted using preset service interface parameters. In the embodiments of the present application, there is no limitation on the transmission method of the task data.

[0147] The category may be a secondary product category of the target garment on an e-commerce platform. The try-on image generation configuration information is user-configured for the target garment and includes, but is not limited to, information such as model skin color, model face, and background style. The user-uploaded image is the original image of the target garment uploaded by the user.

[0148] In some optional embodiments, if the try-on image generation configuration information included in the task data corresponding to the second try-on image generation task is a user-uploaded image, the server may directly obtain the image data of the user-uploaded image from the task data. In some optional embodiments, if the try-on image generation configuration information included in the task data corresponding to the second try-on image generation task is a storage address of the user-uploaded image, the server needs to download the user-uploaded image from the storage address.

[0149] The method for generating the second fitting image generation task and the specific method for generating task data are described in the previous embodiment and will not be repeated here.

[0150] Step 804 : performing image segmentation processing on the user-uploaded image to obtain a clothing canvas image, and performing image segmentation processing on the user-uploaded image to obtain a model mask image.

[0151] In some optional embodiments, image segmentation processing is performed on the user-uploaded image to obtain a clothing canvas image, including: calling a preset target detector based on a first preset semantic prompt word, performing target detection on the user-uploaded image, and obtaining first position information of the target clothing in the user-uploaded image; performing image segmentation processing on the user-uploaded image based on the first position information to obtain a mask map for masking the target clothing; performing cutout processing on the user-uploaded image based on the mask map to obtain a cutout image; and using a pixel mapping method and a Gaussian blur processing method to optimize the edge area of ​​the cutout image to obtain a clothing canvas image.

[0152] In embodiments of the present application, the Grounding DINO object detector can be used to detect objects in an image. For example, if "clothes" is used as the first preset semantic prompt word, Grounding DINO is invoked based on the first preset semantic prompt word, and the user-uploaded image is used as the input image, Grounding DINO will identify the location information of the clothing element in the user-uploaded image, which is recorded as "first location information" in the embodiments of the present application.

[0153] Afterwards, the Segment Anything algorithm can be used to further segment the mask image corresponding to the position in the user uploaded image based on the first position information output by Grounding DINO, that is, to obtain the mask image corresponding to the clothing body.

[0154] For a specific implementation method of performing cutout processing on the user-uploaded image based on the mask image to obtain the cutout image, please refer to the above-mentioned specific implementation method of performing cutout processing on the user-uploaded image based on the clothing mask image to obtain the cutout image, which will not be repeated here.

[0155] The pixel mapping method and Gaussian blur processing method are used to optimize the edge area of ​​the extracted image to obtain the clothing canvas image. The specific implementation method can be found in the relevant description in the above embodiment and will not be repeated here.

[0156] By performing edge mapping on the alpha channel pixel values ​​of the cutout image to reduce the white edge area, and then using the Gaussian blur algorithm to blur the cutout boundary, the jagged edges of the cutout boundary can be reduced, further improving the quality of the clothing canvas image obtained by cutout.

[0157] In some optional embodiments, the image segmentation processing is performed on the user-uploaded image to obtain a model mask image, including: calling a preset target detector based on a second preset semantic prompt word, performing target detection on the user-uploaded image, and obtaining second position information of the model in the user-uploaded image; performing image segmentation processing on the user-uploaded image based on the second position information to obtain a model mask image that masks the model.

[0158] The Grounding DINO object detector is pre-trained with multiple semantic cues, and can retrieve cues for detecting targets from these pre-trained cues. In some optional embodiments, "model" (i.e., a model) can be used as a second pre-set semantic cue. Grounding DINO is invoked based on this second pre-set semantic cue, with the user-uploaded image as the input image. Grounding DINO will then identify the location information of the "model" element in the user-uploaded image, which is referred to as "second location information" in this embodiment of the application.

[0159] In some optional embodiments, a Segment Anything (an image segmentation algorithm) algorithm can be used to further segment the mask image corresponding to the position in the user uploaded image based on the second position information output by Grounding DINO, that is, to obtain a model mask image.

[0160] Step 806 : performing image segmentation processing on the clothing canvas image to obtain a clothing mask image.

[0161] The specific implementation method of performing image segmentation processing on the clothing canvas image to obtain the clothing mask image is described above and will not be repeated here.

[0162] Step 808: Based on the category, the fitting image generation configuration information, the clothing canvas image, the model mask image, and the clothing mask image, a preset image generation service is called to generate a model fitting image of the target clothing.

[0163] Among them, the preset image generation service integrates the following control models through a preset generative large model: a first model for controlling the details of the generated image, a second model for controlling the contour details of the generated image, and a third model for controlling the posture of the model in the generated image.

[0164] The preset image generation service called in this embodiment is described above and will not be repeated here.

[0165] Correspondingly, the configuration information generated based on the category, the try-on image, the clothing canvas image, the model mask image and the clothing mask image is called to generate a preset image generation service to generate a model try-on image of the target clothing, including: generating a target prompt word based on the category, the try-on image configuration information and the preset prompt word; using the clothing canvas image and the clothing mask image as inputs of the first model, using the clothing canvas image and the model mask image as inputs of the second model, and using the model mask image as input of the third model, combining the target prompt word, calling the preset image generation service to generate a model try-on image of the target clothing.

[0166] In some optional embodiments, the generation of configuration information and preset prompt words based on the categories and the fitting pictures to generate target prompt words includes: obtaining a third prompt word corresponding to the category based on the preset prompt words for each preset category; generating configuration information based on the fitting pictures to generate a fourth prompt word; and splicing the third prompt word and the fourth prompt word to generate a target prompt word.

[0167] For a specific implementation method of obtaining the third prompt word corresponding to each preset category based on the preset prompt word for the category, please refer to the specific implementation method of obtaining the first prompt word corresponding to the category based on the preset prompt word for each preset category in the previous article, which will not be repeated here.

[0168] For a specific implementation method of generating the fourth prompt word based on the configuration information generated by the fitting image, please refer to the specific implementation method of obtaining the second prompt word corresponding to the category based on the preset prompt word for each preset category in the previous article, which will not be repeated here.

[0169] In some embodiments of the present application, the third prompt word corresponding to each preset category is obtained based on the preset prompt word for each preset category, including any of the following: when the category is tops, based on the preset prompt word for each preset category, a third prompt word for generating bottoms is obtained; when the category is skirts or pants, based on the preset prompt word for each preset category, a third prompt word for generating tops is obtained; when the category is dresses, based on the preset prompt word for each preset category, a third prompt word for not generating bottoms or inner wear is obtained. Taking the preset categories including tops, skirts, dresses, and pants as an example, the preset prompt words can include the following prompt words: for tops, the preset prompt words include prompt words generated for bottoms; for skirts / pants, the preset prompt words include prompt words generated for tops; for dresses, the preset prompt words include negative prompt words for not generating inner wear or bottoms. By setting prompt words describing matching information for preset categories, clothing matching can be achieved in model try-on pictures, further improving user experience.

[0170] When using a merchant mannequin to generate a model fitting image, in an embodiment of the present application, it is necessary to use a third model (such as ControlNet's OpenPose preprocessor) to generate a skeleton image from the model mask image, so that the skeleton image can be used as part of the input data of the Stable Diffusion large model to retain the model's posture in the image uploaded by the user. In addition, it is necessary to use the clothing canvas image and the clothing mask image as input images of the first model, and use the first model (such as ControlNet's SoftEdge preprocessor) to limit the approximate area of ​​the model's body and clothing. At the same time, the clothing canvas image and the model mask image are used as inputs of the second model (such as ControlNet's Inpainting preprocessor) to restore the photo to the greatest extent and finely control the clothing material details and edge areas.

[0171] The interface calling method of Stable Diffusion's graph generation service is described above and will not be repeated here.

[0172] During the image generation phase of Stable Diffusion, it uses ControlNet's preprocessors OpenPose and SoftEdge to control the model's pose and display effects, and enables the ADetailer plug-in to optimize the model's face and hands. This results in high-quality model fitting images where the clothing is the target clothing, the model's skin color, face, and background style match the user's configuration, and the model's expression and pose are natural.

[0173] In summary, the fitting image generation method disclosed in the embodiment of the present application, after obtaining the category of the target clothing corresponding to the second fitting image generation task, the fitting image generation configuration information and the user uploaded image, performs image segmentation processing on the user uploaded image to obtain a clothing canvas image and a model mask image, which are used to restore the clothing details and retain the model posture in the user uploaded image, and then performs image segmentation processing on the clothing canvas image to obtain a clothing mask image, which is used to depict the edge information of the clothing. Thereafter, based on the category, the fitting image generation configuration information, the clothing canvas image, the model mask image and the clothing mask image, the preset image generation service is called to generate a model fitting image of the target clothing, so that the generated model fitting image not only retains the model posture and the original appearance of the clothing in the user uploaded image, but also incorporates the model skin color, model face and background style customized in the fitting image generation configuration information set by the user, and the details of the model's hands, face, and clothing in the model fitting image are more natural.

[0174] Furthermore, by pre-setting the top and bottom matching prompts for each category and outputting matching try-on images, the visual effects of the model try-on images are improved, thus enhancing the user experience. Furthermore, by pre-setting the prompts for each category, the stability of the generated model try-on images can be improved.

[0175] 9 , based on the above embodiment, an embodiment of the present application further provides a fitting image generation system, the system comprising: a client 900 and a server 910, wherein the client 900 is used to execute the clothing fitting image generation method described in the aforementioned embodiments corresponding to the step flowcharts of FIG. 1 and FIG. 6 ; the server 910 is used to execute the clothing fitting image generation method described in the aforementioned embodiments corresponding to the step flowcharts in FIG. 7 and / or FIG. 8 .

[0176] In an optional embodiment, as shown in Figure 9 , the server 910 further includes a control module 911, a segmentation service module 912, and an image generation service module 913. The control module 911, the segmentation service module 912, and the image generation service module 913 can be deployed separately or partially independently, or all integrated. The following, combined with the interactive flow chart shown in Figure 9 , illustrates the steps of an optional embodiment of the execution process of the try-on image generation system disclosed in this application.

[0177] The client 900 is used to obtain the category of the target clothing input by the user and the configuration information for generating a try-on picture, and the user uploads an image.

[0178] The client 900 is also used to determine the type of mannequin selected by the user. If it is a system mannequin, the client 900 is also used to obtain a target mannequin image and the model posture information of the target mannequin image, and synthesize a clothing canvas image based on the user uploaded image and the target mannequin image, and generate a first fitting image generation task based on the clothing canvas image, the model posture information, the category and the fitting image generation configuration information, and send the first fitting image generation task to the server.

[0179] The client 900 is also used to generate a second fitting picture generation task based on the category of the user uploaded image and the fitting picture generation configuration information when determining that the user chooses to use the merchant mannequin to generate a model fitting picture, and send the second fitting picture generation task to the server.

[0180] The server 910 is configured to generate a model fitting image of the target garment based on the first fitting image generation task; and / or generate a model fitting image of the target garment based on the second fitting image generation task.

[0181] In some optional embodiments, during the process of the server 910 generating the model fitting image of the target garment based on the first fitting image generation task,

[0182] The control module 911 is configured to obtain the target clothing category, model posture information, fitting image generation configuration information, and clothing canvas image corresponding to the first fitting image generation task;

[0183] The control module 911 is further configured to call the segmentation service module 912 to perform image segmentation processing on the clothing canvas image to obtain a clothing mask image;

[0184] The control module 911 is further configured to obtain a preset real-life model image based on the model posture information;

[0185] The control module 911 is further configured to call the image generation service module 913 based on the category, the fitting image generation configuration information, the clothing canvas image, the real model image and the clothing mask image to generate a model fitting image of the target clothing.

[0186] At this point, the process of generating model fitting images based on the system mannequin is completed.

[0187] In some optional embodiments, during the process of the server 910 generating the model fitting image of the target garment based on the second fitting image generation task,

[0188] The control module 911 is further configured to obtain the target clothing category, try-on image generation configuration information, and the user-uploaded image corresponding to the second try-on image generation task;

[0189] The control module 911 is further configured to call the segmentation service module 912 to perform image segmentation processing on the user-uploaded image to obtain a clothing canvas image, and to call the segmentation service module 912 to perform image segmentation processing on the user-uploaded image to obtain a model mask image;

[0190] The control module 911 is further configured to call the segmentation service module 912 to perform image segmentation processing on the clothing canvas image to obtain a clothing mask image;

[0191] The control module 911 is further configured to call the image generation service module 913 based on the category, the fitting image generation configuration information, the clothing canvas image, the model mask image and the clothing mask image to generate a model fitting image of the target clothing.

[0192] At this point, the process of generating model fitting images based on the merchant mannequin is completed.

[0193] The specific implementation of the client 900 and the control module 911 can be found in the above embodiments and will not be repeated here. The specific implementation of the graph generation service module 913 can be found in the above Stable Diffusion graph generation service and will not be repeated here. In some optional embodiments, the segmentation service module 912 can be implemented based on the rounding DINO target detector and Segment Anything (an image segmentation algorithm). The segmentation service module 912 can also adopt other segmentation services in the prior art, and the specific implementation of the segmentation service module 912 is not limited in the embodiments of the present application.

[0194] The customer segment 900 is also used to query and display the model fitting pictures.

[0195] In summary, the fitting picture generation system disclosed in the embodiment of the present application, when the user uploads an image of a non-full-limb mannequin wearing clothing or a tiled image, the user uploads the image of the target clothing on the client, and configures the category, model skin color and / or model face and / or background style of the generated model fitting picture as needed, then only needs to select the system mannequin, and the system can display mannequins in various postures for the user, and generate a model fitting picture based on the model posture selected by the user and the user-configured information. In the generated model fitting picture, the model wears the target clothing in the model posture selected by the user, which effectively improves the quality of the generated model fitting picture, and the details such as the model face and hands in the model fitting picture are natural, and the generated model fitting picture is more stable. For situations where the model posture in the user-uploaded image needs to be adopted, the user can select the merchant mannequin mode, and the system can generate a model fitting picture based on the user-configured information. In the generated model fitting picture, the model wears the target clothing in the model posture in the user-uploaded image.

[0196] It should be noted that the model in the model fitting picture generated in the embodiment of the present application may be a model generated using the image-to-image technology, or may be a model whose portrait is authorized by the model himself to be used.

[0197] It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0198] It should be noted that for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the order of the actions described, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0199] Based on the above embodiment, this embodiment further provides a device for generating a try-on image, which is applied to an electronic device such as a client device. The device includes:

[0200] A user uploaded image and configuration information acquisition module is used to obtain the category of target clothing input by the user and configuration information for generating a try-on image, as well as the user uploaded image;

[0201] A full-body mannequin image and posture acquisition module is configured to generate a model fitting image in response to a user selecting a system mannequin, acquire a target mannequin image, and mannequin posture information of the target mannequin image;

[0202] a clothing canvas image generation module, configured to synthesize a clothing canvas image based on the user uploaded image and the target mannequin image;

[0203] The first model fitting image generation module is used to generate a first fitting image generation task based on the clothing canvas image, the model posture information, the category and the fitting image generation configuration information, so that the preset server generates a model fitting image of the target clothing based on the first fitting image generation task.

[0204] In some optional embodiments, the clothing canvas image generation module is further configured to:

[0205] Creating a canvas based on the target mannequin image;

[0206] Performing clothing segmentation processing on the user uploaded image to obtain a clothing image of the target clothing;

[0207] Creating a clothing layer based on the clothing image;

[0208] Based on the user's adjustment operation on the display information of the clothing layer in the canvas, the adaptation result of the target clothing in the clothing layer and the mannequin in the target mannequin image is displayed in real time;

[0209] Based on a user's confirmation operation on the adaptation result, the clothing layer and the canvas are synthesized according to the adjusted display information to obtain a clothing canvas image.

[0210] In some optional embodiments, after obtaining the target clothing category and fitting image generation configuration information input by the user, and after the user uploads the image, the device further includes:

[0211] The second model fitting picture generation module is used to generate a model fitting picture in response to the user selecting a merchant mannequin, and to generate a second fitting picture generation task based on the user uploaded image, the category and the fitting picture generation configuration information, so that the preset server generates a model fitting picture of the target clothing based on the second fitting picture generation task.

[0212] In some optional embodiments, the fitting image generation configuration information includes one or more of the following information: model skin color, model face, and background style.

[0213] In summary, when it is necessary to generate a clothing try-on picture, the user uploads an image of the clothing to be tried on and configures the category and try-on picture generation configuration information. Afterwards, the user can choose to use the system mannequin to generate a model try-on picture. The client will display a preset mannequin image of at least one model posture for the user to choose, and synthesize a clothing canvas image based on the mannequin image selected by the user and the user uploaded image; afterward, based on the clothing canvas image, the model posture information, the category and the try-on picture generation configuration information, a first try-on picture generation task is generated, so that the preset server generates a model try-on picture of the target clothing based on the first try-on picture generation task. The model try-on picture generated by this method introduces information such as model posture on the basis of retaining the clothing information in the user uploaded image, effectively improving the quality and display effect of the generated model try-on picture.

[0214] Furthermore, when synthesizing clothing canvas images, users can adjust the matching relationship between clothing and models, making the combination of clothing and models in the generated fitting images more natural and fitting, thereby improving the display effect of clothing in the model fitting images.

[0215] When generating the first fitting image generation task, the model posture information is retained. When the user server executes the first fitting image generation task to generate the model fitting image, the system built-in real-life model image corresponding to the model posture information is further obtained to generate the model fitting image, so that the model's face and bones in the generated fitting image are more natural and beautiful.

[0216] Based on the above embodiment, this embodiment further provides a device for generating a try-on image, which is applied to an electronic device such as a server device. The device includes:

[0217] A first task data acquisition module is used to obtain the target clothing category, model posture information, try-on image generation configuration information, and clothing canvas image corresponding to the first try-on image generation task;

[0218] a clothing mask image generation module, configured to perform image segmentation processing on the clothing canvas image to obtain a clothing mask image;

[0219] A real-life model image acquisition module, configured to acquire a preset real-life model image based on the model posture information;

[0220] The model fitting image generation module is used to call the preset image generation service based on the category, the fitting image generation configuration information, the clothing canvas image, the real model image and the clothing mask image to generate a model fitting image of the target clothing.

[0221] In some optional embodiments, the preset image generation service integrates the following control models through a preset generative model: a first model for controlling the details of the generated image, a second model for controlling the outline details of the generated image, and a third model for controlling the pose of the model in the generated image; the model fitting image generation module is further configured to:

[0222] Generate target prompt words based on the category, the try-on image, configuration information, and preset prompt words;

[0223] The clothing canvas image and the clothing mask image are used as inputs of the first model, the clothing canvas image and the real-life model image are used as inputs of the second model, and the real-life model image is used as input of the third model. In combination with the target prompt word, a preset image generation service is called to generate a model fitting image of the target clothing.

[0224] In some optional embodiments, generating the target prompt word based on the category, the fitting image, and the configuration information and preset prompt words includes:

[0225] Based on the preset prompt word for each preset category, obtaining a first prompt word corresponding to the category;

[0226] Generate configuration information based on the try-on image and generate a second prompt word;

[0227] The first prompt word and the second prompt word are concatenated to generate a target prompt word.

[0228] In some optional embodiments, obtaining the first prompt word corresponding to each preset category based on the preset prompt word includes any one of the following:

[0229] In a case where the category is upper clothing, obtaining a first prompt word for generating lower clothing based on a preset prompt word for each preset category;

[0230] In a case where the category is skirts or pants, obtaining a first prompt word for generating a top based on a preset prompt word for each preset category;

[0231] In a case where the category is dresses, a first prompt word that does not generate matching clothing is obtained based on the prompt words preset for each preset category.

[0232] In some optional embodiments, the clothing mask image generating module is further configured to:

[0233] calling a preset object detector based on the first preset semantic prompt word, performing object detection on the clothing canvas image, and obtaining position information of the target clothing in the clothing canvas image;

[0234] Image segmentation processing is performed on the clothing canvas image based on the position information to obtain a clothing mask image that masks the target clothing.

[0235] In summary, after obtaining the target clothing category, model pose information, try-on image generation configuration information, and clothing canvas image corresponding to the first try-on image generation task, the clothing canvas image is first segmented to obtain a clothing mask image. Based on the model pose information, a preset real-life model image is obtained. Then, based on the category, the try-on image generation configuration information, the clothing canvas image, the real-life model image, and the clothing mask image, a preset image generation service is called to generate a model try-on image of the target clothing. This not only retains the original appearance of the clothing in the user-uploaded image, but also incorporates the model skin color, model face, and background style customized in the try-on image generation configuration information set by the user, as well as the model pose selected by the user, thereby improving the quality of the generated model try-on image. Furthermore, by selecting the system-preset real-life model image instead of the mannequin image, it ensures that hand and facial details are preserved to the greatest extent possible, making the facial and hand details of the generated model try-on image more natural and the generated image quality more stable.

[0236] Furthermore, by pre-setting the top and bottom matching prompts for each category and outputting matching try-on images, the visual effects of the model try-on images are improved, thus enhancing the user experience. Furthermore, by pre-setting the prompts for each category, the stability of the generated model try-on images can be improved.

[0237] On the basis of the above embodiment, the present application also provides a clothing fitting picture generation method for performing a second fitting picture generation task to generate a model fitting picture based on the user uploaded image, category, fitting picture generation configuration information, etc. corresponding to the second fitting picture generation task.

[0238] Based on the above embodiment, this embodiment further provides a device for generating a try-on image, which is applied to an electronic device such as a server device. The device includes:

[0239] The second task data acquisition module is used to obtain the target clothing category, try-on image generation configuration information, and user-uploaded images corresponding to the second try-on image generation task;

[0240] A clothing canvas image and model mask image acquisition module, configured to perform image segmentation processing on the user-uploaded image to obtain a clothing canvas image, and to perform image segmentation processing on the user-uploaded image to obtain a model mask image;

[0241] a clothing mask image acquisition module, configured to perform image segmentation processing on the clothing canvas image to obtain a clothing mask image;

[0242] The model fitting image generation module is used to call the preset image generation service based on the category, the fitting image generation configuration information, the clothing canvas image, the model mask image and the clothing mask image to generate a model fitting image of the target clothing.

[0243] In some optional embodiments, the preset image generation service integrates the following control models through a preset generative macro model: a first model for controlling generated image details, a second model for controlling generated image outline details, and a third model for controlling the pose of the model in the generated image;

[0244] The model fitting image generation module is further configured to:

[0245] Generate target prompt words based on the category, the try-on image, configuration information, and preset prompt words;

[0246] The clothing canvas image and the clothing mask image are used as inputs of the first model, the clothing canvas image and the model mask image are used as inputs of the second model, and the model mask image is used as input of the third model. In combination with the target prompt word, a preset image generation service is called to generate a model fitting image of the target clothing.

[0247] In some optional embodiments, performing image segmentation processing on the user-uploaded image to obtain a clothing canvas image includes:

[0248] Based on the first preset semantic prompt word, a preset object detector is called to perform object detection on the user-uploaded image to obtain first position information of the target clothing in the user-uploaded image;

[0249] performing image segmentation processing on the user-uploaded image based on the first location information to obtain a mask image for the target clothing;

[0250] Performing a cutout process on the user-uploaded image based on the mask image to obtain a cutout image;

[0251] A pixel mapping method and a Gaussian blur processing method are adopted to optimize the edge area of ​​the extracted image to obtain a clothing canvas image.

[0252] In some optional embodiments, performing image segmentation processing on the user-uploaded image to obtain a model mask image includes:

[0253] Based on the second preset semantic prompt word, a preset object detector is called to perform object detection on the user-uploaded image to obtain second position information of the model in the user-uploaded image;

[0254] Image segmentation processing is performed on the user-uploaded image based on the second position information to obtain a model mask image for masking the model.

[0255] In summary, the fitting image generation device disclosed in the embodiment of the present application, after obtaining the category of the target clothing corresponding to the second fitting image generation task, the fitting image generation configuration information and the user uploaded image, performs image segmentation processing on the user uploaded image to obtain a clothing canvas image and a model mask image, which are used to restore the clothing details and retain the model posture in the user uploaded image. Then, the clothing canvas image is subjected to image segmentation processing to obtain a clothing mask image, which is used to depict the edge information of the clothing. Thereafter, based on the category, the fitting image generation configuration information, the clothing canvas image, the model mask image and the clothing mask image, the preset image generation service is called to generate a model fitting image of the target clothing, so that the generated model fitting image not only retains the model posture and the original appearance of the clothing in the user uploaded image, but also incorporates the model skin color, model face and background style customized in the fitting image generation configuration information set by the user, and the details of the model's hands, face, and clothing in the model fitting image are more natural.

[0256] Furthermore, by presetting the matching prompt words for each category, outputting the matching try-on pictures, improving the visual effect of the model try-on pictures, and improving the user experience. In addition, by presetting the prompt words for each category, the stability of the generated model try-on pictures can be improved.

[0257] An embodiment of the present application further provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0258] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the embodiment of the present application.

[0259] The present application also provides an electronic device comprising: a processor and a memory communicatively connected to the processor; the memory storing computer-executable instructions; and the processor executing the computer-executable instructions stored in the memory to implement the method described in the present application. In the present application, the electronic device includes a server, a terminal device, and other devices.

[0260] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as a server (cluster), a terminal, etc. FIG10 schematically illustrates an exemplary device 1000 that may be used to implement various embodiments described in this application.

[0261] For one embodiment, Figure 10 shows an exemplary apparatus 1000 having one or more processors 1002, a control module (chip set) 1004 coupled to at least one of the processor(s) 1002, a memory 1006 coupled to the control module 1004, a non-volatile memory (NVM) / storage device 1008 coupled to the control module 1004, one or more input / output devices 1010 coupled to the control module 1004, and a network interface 1012 coupled to the control module 1004.

[0262] The processor 1002 may include one or more single-core or multi-core processors, and the processor 1002 may include any combination of general-purpose processors or dedicated processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, the apparatus 1000 can serve as a server, terminal, or other device described in the embodiments of the present application.

[0263] In some embodiments, the apparatus 1000 may include one or more computer-readable media (e.g., memory 1006 or NVM / storage 1008) having instructions 1014 and one or more processors 1002 configured in conjunction with the one or more computer-readable media to execute the instructions 1014 to implement a module to perform the actions described in the present disclosure.

[0264] For one embodiment, the control module 1004 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1002 and / or any suitable device or component in communication with the control module 1004 .

[0265] The control module 1004 may include a memory controller module to provide an interface to the memory 1006. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0266] The memory 1006 can be used, for example, to load and store data and / or instructions 1014 for the device 1000. For one embodiment, the memory 1006 can include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1006 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0267] For one embodiment, the control module 1004 may include one or more input / output controllers to provide an interface to the NVM / storage device 1008 and the input / output device(s) 1010 .

[0268] For example, NVM / storage 1008 may be used to store data and / or instructions 1014. NVM / storage 1008 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0269] NVM / storage device 1008 may include storage resources that are part of the device on which apparatus 1000 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1008 may be accessible over a network via input / output device(s) 1010.

[0270] (One or more) input / output devices 1010 may provide an interface for apparatus 1000 to communicate with any other appropriate device. Input / output devices 1010 may include communication components, audio components, sensor components, etc. Network interface 1012 may provide an interface for apparatus 1000 to communicate via one or more networks. Apparatus 1000 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard such as Bluetooth, WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0271] For one embodiment, at least one of the processor(s) 1002 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1004. For one embodiment, at least one of the processor(s) 1002 may be packaged together with the logic of one or more controllers of the control module 1004 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1002 may be integrated on the same die with the logic of one or more controllers of the control module 1004. For one embodiment, at least one of the processor(s) 1002 may be integrated on the same die with the logic of one or more controllers of the control module 1004 to form a system-on-chip (SoC).

[0272] In various embodiments, apparatus 1000 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, apparatus 1000 may have more or fewer components and / or a different architecture. For example, in some embodiments, apparatus 1000 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0273] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in the memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0274] The present application also provides an electronic device comprising: a processor; and a memory storing executable code, wherein when the executable code is executed, the processor executes one or more methods described in the embodiments of the present application. The memory in the embodiments of the present application can store various data, such as target files, file-application association data, and other data, and can also include user behavior data, thereby providing a data foundation for various processing.

[0275] The embodiments of the present application further provide one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the embodiments of the present application.

[0276] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0277] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0278] The present application embodiment is described with reference to the flow chart and / or block diagram of the method, terminal device (system), and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flow chart and / or block diagram and the combination of the process and / or box in the flow chart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for realizing the function specified in one process or multiple processes and / or one box or multiple boxes of the flow chart.

[0279] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0280] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0281] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0282] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0283] The above is a detailed introduction to a fitting image generation method, a fitting image generation system, an electronic device and a storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for generating a fitting image, applied to a client, characterized in that: The method comprises: Obtain the category of the target clothing input by the user and the configuration information for generating the try-on picture, as well as the image uploaded by the user; In response to the user selecting the system mannequin to generate a model fitting image, a target mannequin image and model posture information of the target mannequin image are acquired; Synthesize a clothing canvas image based on the user uploaded image and the target mannequin image; Based on the clothing canvas image, the model posture information, the category and the fitting image generation configuration information, a first fitting image generation task is generated, so that the preset server generates a model fitting image of the target clothing based on the first fitting image generation task.

2. The method according to claim 1, characterized in that: The step of synthesizing a clothing canvas image based on the user uploaded image and the target mannequin image includes: Based on the target mannequin image, create a canvas; Performing clothing segmentation processing on the image uploaded by the user to obtain a clothing image of the target clothing; Based on the clothing image, create a clothing layer; Based on the user's adjustment operation on the display information of the clothing layer in the canvas, the adaptation result of the target clothing in the clothing layer and the mannequin in the target mannequin image is displayed in real time; Based on a user's confirmation operation on the adaptation result, the clothing layer and the canvas are synthesized according to the adjusted display information to obtain a clothing canvas image.

3. The method according to claim 1, characterized in that The step of obtaining the category of target clothing input by the user and the configuration information for generating the fitting picture, and after the user uploads the image, further includes: In response to the user selecting a merchant mannequin to generate a model fitting picture, a second fitting picture generation task is generated based on the user uploaded image, the category and the fitting picture generation configuration information, so that the preset server generates a model fitting picture of the target clothing based on the second fitting picture generation task.

4. The method according to claim 1, characterized in that The fitting image generation configuration information includes one or more of the following information: model skin color, model face, and background style.

5. A method for generating a fitting image, characterized in that: The method comprises: Obtaining the category of target clothing, model posture information, fitting image generation configuration information, and clothing canvas image corresponding to the first fitting image generation task; Performing image segmentation processing on the clothing canvas image to obtain a clothing mask image; Based on the model posture information, obtaining a preset real-life model image; Based on the category, the fitting image generation configuration information, the clothing canvas image, the real model image and the clothing mask image, a preset image generation service is called to generate a model fitting image of the target clothing.

6. The method according to claim 5, characterized in that The preset image generation service integrates the following control models through a preset generative large model: a first model for controlling the details of the generated image, a second model for controlling the contour details of the generated image, and a third model for controlling the pose of the model in the generated image; based on the category, the fitting image generation configuration information, the clothing canvas image, the real model image and the clothing mask image, the preset image generation service is called to generate the model fitting image of the target clothing, including: Generate target prompt words based on the category, the fitting picture generation configuration information and the preset prompt words; The clothing canvas image and the clothing mask image are used as inputs of the first model, the clothing canvas image and the real-life model image are used as inputs of the second model, and the real-life model image is used as input of the third model. In combination with the target prompt word, a preset image generation service is called to generate a model fitting image of the target clothing.

7. The method according to claim 6, characterized in that The generating of configuration information and preset prompt words based on the categories and the fitting pictures, and generating target prompt words, includes: Based on the preset prompt word for each preset category, obtaining a first prompt word corresponding to the category; Generate configuration information based on the try-on image and generate a second prompt word; The first prompt word and the second prompt word are concatenated to generate a target prompt word.

8. The method according to claim 7, characterized in that The acquiring, based on the preset prompt word for each preset category, the first prompt word corresponding to the category includes any one of the following: In the case where the category is upper clothing, based on the preset prompt words for each preset category, Get the first prompt word for generating the lower garment; In the case where the category is skirts or pants, based on the preset prompt words for each preset category, a first prompt word for generating a top is obtained; In the case where the category is dresses, based on the preset prompt words for each preset category, a first prompt word that does not generate matching clothing is obtained.

9. The method according to claim 5, characterized in that The performing image segmentation processing on the clothing canvas image to obtain a clothing mask image includes: Calling a preset target detector based on the first preset semantic prompt word, performing target detection on the clothing canvas image, and obtaining position information of the target clothing in the clothing canvas image; Image segmentation processing is performed on the clothing canvas image based on the position information to obtain a clothing mask image that masks the target clothing.

10. A method for generating a fitting image, characterized in that: The method comprises: Obtaining the category of target clothing, try-on image generation configuration information, and user-uploaded images corresponding to the second try-on image generation task; Performing image segmentation processing on the image uploaded by the user to obtain a clothing canvas image, and performing image segmentation processing on the image uploaded by the user to obtain a model mask image; Performing image segmentation processing on the clothing canvas image to obtain a clothing mask image; Based on the category, the fitting image generation configuration information, the clothing canvas image, the model mask image and the clothing mask image, a preset image generation service is called to generate a model fitting image of the target clothing.

11. The method according to claim 10, characterized in that The preset image generation service integrates the following control models through a preset generative large model: a first model for controlling the details of the generated image, a second model for controlling the contour details of the generated image, and a third model for controlling the pose of the model in the generated image; based on the category, the fitting image generation configuration information, the clothing canvas image, the model mask image and the clothing mask image, the preset image generation service is called to generate the model fitting image of the target clothing, including: Generate target prompt words based on the category, the fitting picture generation configuration information and the preset prompt words; The clothing canvas image and the clothing mask image are used as inputs of the first model, the clothing canvas image and the model mask image are used as inputs of the second model, and the model mask image is used as input of the third model, and the target prompt word is combined to call the prediction A picture generation service is provided to generate pictures of models trying on the target clothing.

12. The method according to claim 10, characterized in that The step of performing image segmentation processing on the image uploaded by the user to obtain a clothing canvas image includes: Based on the first preset semantic prompt word, a preset object detector is called to perform object detection on the image uploaded by the user to obtain first position information of the target clothing in the image uploaded by the user; Performing image segmentation processing on the user uploaded image based on the first position information to obtain a mask image for masking the target clothing; Performing a cutout process on the image uploaded by the user based on the mask image to obtain a cutout image; The pixel mapping method and the Gaussian blur processing method are used to optimize the edge area of ​​the extracted image to obtain a clothing canvas image.

13. The method according to claim 10, characterized in that The step of performing image segmentation processing on the image uploaded by the user to obtain a model mask image includes: Based on the second preset semantic prompt word, a preset object detector is called to perform object detection on the image uploaded by the user to obtain second position information of the model in the image uploaded by the user; Image segmentation processing is performed on the user uploaded image based on the second position information to obtain a model mask image for masking the model.

14. A fitting image generation system, characterized in that: The system comprises: A client, used to execute the method for generating a clothing fitting image according to any one of claims 1 to 4; The server is used to execute the clothing fitting image generating method as described in any one of claims 5 to 13.

15. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 13 when executed by a processor.

Images