Image processing method, model training method and electronic equipment

Abstract

The invention discloses an image processing method, a model training method and electronic equipment, and relates to the field of large model technology and image processing. The method comprises the steps that an image to be processed is acquired, and the image to be processed comprises a target object; performing feature extraction on the to-be-processed image by using a feature extraction model to obtain article features of the target article; and based on the article features and the edge information of the target article, guiding an image generation model to carry out conversion processing on the to-be-processed image to obtain a target image, other regions except the region where the target article is located in the target image being displayed through a preset texture, and the target article in the target image being in a preset state. The technical problem that the image processing effect is poor in the prior art is solved.

Description

Technical Field

[0001] The present application relates to large model technology and image processing fields, and specifically, to an image processing method, a model training method and an electronic device. Background Art

[0002] In e-commerce scenarios, merchants often need to convert clothing images containing models or real clothing images into standard white-background flat clothing images to meet product display needs. However, existing methods do not work well when dealing with clothing worn on models, folded clothing, or with messy backgrounds, resulting in high cost and high threshold for obtaining high-quality white-background flat clothing images, resulting in poor image processing results.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, a model training method, and an electronic device to at least solve the technical problem of poor image processing effect in related technologies.

[0005] According to one aspect of an embodiment of the present application, there is provided an image processing method, comprising: acquiring an image to be processed, wherein the image to be processed includes a target object; performing feature extraction on the image to be processed using a feature extraction model to obtain object features of the target object; and guiding an image generation model to perform transformation processing on the image to be processed based on the object features and edge information of the target object to obtain a target image, wherein areas of the target image other than an area where the target object is located are displayed using a preset texture, and the target object in the target image is in a preset state.

[0006] According to another aspect of an embodiment of the present application, a model training method is also provided, including: obtaining a sample image to be processed and a sample target image corresponding to the sample image to be processed, wherein the sample image to be processed includes a sample object; performing feature extraction on the sample image to be processed using a feature extraction model to obtain sample item features of the sample item; guiding an initial image generation model to perform transformation processing on the sample image to be processed based on the sample item features and sample edge information of the sample item features to obtain a sample transformed image; constructing a loss function based on the sample transformed image and the sample target image; and using the loss function to update model parameters of the initial image generation model to obtain an image generation model.

[0007] According to another aspect of an embodiment of the present application, there is also provided an image processing method, comprising: acquiring a clothing image, wherein the clothing image includes a target clothing; performing feature extraction on the clothing image using a feature extraction model to obtain clothing features of the target clothing; based on the clothing features and edge information of the target clothing, guiding an image generation model to perform transformation processing on the clothing image to obtain a target clothing image, wherein areas of the target clothing image other than an area where the target clothing is located are displayed using a preset texture, and the target clothing in the target clothing image is in a preset state.

[0008] According to another aspect of an embodiment of the present application, an image processing method is also provided, including: responding to an input instruction acting on an interactive interface, displaying an image to be processed on the interactive interface, wherein the image to be processed includes a target object; responding to a processing instruction acting on the interactive interface, displaying a target image corresponding to the image to be processed on the interactive interface, wherein other areas of the target image except the area where the target object is located are displayed using a preset texture, and the target object in the target image is in a preset state, the target image is obtained by guiding an image generation model to perform transformation processing on the image to be processed according to the object features of the target object and the edge information of the target object, and the object features are obtained by extracting features from the image to be processed using a feature extraction model.

[0009] According to another aspect of the embodiments of the present application, a computer terminal is further provided, including: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present application is executed when the program is running.

[0010] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in each embodiment of the present application.

[0011] According to another aspect of the embodiments of the present application, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0012] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0013] According to another aspect of the embodiments of the present application, a computer program is also provided, and when the computer program is executed by a processor, the methods in the various embodiments of the present application are implemented.

[0014] In an embodiment of the present application, an image to be processed is obtained, wherein the image to be processed includes a target object; a feature extraction model is used to extract features of the image to be processed to obtain object features of the target object; based on the object features and edge information of the target object, an image generation model is guided to transform the image to be processed to obtain a target image, wherein other areas in the target image except the area where the target object is located are displayed by preset textures, and the target object in the target image is in a preset state, thereby achieving the purpose of improving the image processing effect; it is easy to notice that the feature extraction module can be used to capture the details of the target object to form object features, and the edge information of the target object is used to provide conditional guidance for the process of transforming the image to be processed by the image generation model, thereby ensuring that the boundaries of the target object in the generated target image are clear, and the area where the non-target object is located can be filled by the preset texture, while ensuring that the target object is in a preset state, thereby improving the visual effect of the target image, thereby improving the display effect of the target object in the target image, and thus solving the technical problem of poor image processing effect in the related art.

[0015] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a schematic diagram of an application scenario of an image processing method according to an embodiment of the present application;

[0018] Figure 2 is a flowchart of an image processing method according to an embodiment of the present application;

[0019] Figure 3 is a flow chart of clothing image processing according to an embodiment of the present application;

[0020] Figure 4 is a schematic diagram of an image generation model structure according to an embodiment of the present application;

[0021] Figure 5 is a training flow chart of an image generation model according to an embodiment of the present application;

[0022] Figure 6 is a schematic diagram of the reasoning process of an image generation model according to an embodiment of the present application;

[0023] Figure 7is a flow chart of a model training method according to an embodiment of the present application;

[0024] Figure 8 is a flowchart of an image processing method according to an embodiment of the present application;

[0025] Fig. 9 is a flowchart of an image processing method according to an embodiment of the present application;

[0026] Fig.10 is a schematic diagram of an image processing device according to an embodiment of the present application;

[0027] Fig.11 is a schematic diagram of a model training device according to an embodiment of the present application;

[0028] Fig.12 is a schematic diagram of an image processing device according to an embodiment of the present application;

[0029] Fig.13 is a schematic diagram of an image processing device according to an embodiment of the present application;

[0030] Fig.14 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] The technical solution provided in this application is mainly implemented by large-scale model technology. The large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions or even more than 10 trillion model parameters. The large model can also be called a cornerstone model / foundation model (Foundation Model). The large model is pre-trained through large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as large-scale language model (Large Language Model, LLM), multi-modal pre-training model (multi-modal pre-training model), etc.

[0034] It should be noted that when the large model is actually applied, the pre-trained model can be fine-tuned through a small number of samples, so that the large model can be applied to different tasks. For example, the large model can be widely used in natural language processing (NLP), computer vision, speech processing and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, etc. It can also be widely used in natural language processing tasks such as text-based sentiment classification, text summary generation, machine translation, etc. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiment of the present application, data processing through feature extraction models and image generation models in image processing scenarios is taken as an example for explanation,

[0035] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0036] The diffusion model is a probability-based image generation model that is mainly used to generate high-quality images. In the field of machine learning, the diffusion model generates images by gradually adding and removing noise, which simulates the diffusion phenomenon of molecules in thermal motion.

[0037] The Stable Diffusion model (SD) is a specific implementation of the diffusion model, which aims to generate high-quality images while maintaining the stability and efficiency of the model.

[0038] The extended version of the stable diffusion model (Stable Diffusion XL, referred to as SDXL) is an extended version of StableDiffusion, which usually has higher image generation quality and can generate higher resolution images.

[0039] U-Net is a convolutional neural network architecture suitable for image segmentation and image generation tasks. It uses skip connections between the encoder and decoder to retain more details.

[0040] The control network (Control Net) is a neural network used to control the generation process of the diffusion model. Control Net guides the model to generate images of specific structures by introducing additional conditional inputs, such as edge maps or posture maps.

[0041] A mask is a binary image that is usually used to identify specific areas in an image, such as the location and shape of a foreground object. In image processing and computer vision, masks are often used to segment objects and remove backgrounds.

[0042] Edge detection (Canny Edge Detection) is an algorithm used to identify edges in an image. The Canny algorithm detects and refines edges through multi-stage processing (such as Gaussian filtering, intensity calculation, non-maximum suppression, and double threshold detection) to generate a clear edge map.

[0043] White-background flat-lay clothing images refer to images that display clothing in a flat manner on a white background. They are commonly seen in e-commerce scenarios and are used to clearly display the details and colors of clothing without being affected by the model or shooting environment.

[0044] According to an embodiment of the present application, an image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] Considering the huge amount of model parameters of the large model and the limited computing resources of the mobile terminal, the above-mentioned image processing method provided in the embodiment of the present application can be applied to the following Figure 1 The application scenarios shown are not limited to these. Figure 1 is a schematic diagram of an application scenario of an image processing method according to an embodiment of the present application. Figure 1In the application scenario shown, the large model is deployed in the server 10, and the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client device 20 here may include but is not limited to: a smart phone, a tablet computer, a laptop computer, a PDA, a personal computer, a smart home device, a vehicle-mounted device, etc. The client device 20 can interact with the user through a graphical user interface to implement the call of the large model, thereby implementing the method provided in the embodiment of the present application.

[0046] In the embodiment of the present application, the system composed of the client device and the server can perform the following steps: the client device performs the step of uploading the image to be processed, and the server performs feature extraction on the image to be processed using the feature extraction model to obtain the item features of the target item; based on the item features and the edge information of the target item, the image generation model is guided to transform the image to be processed to obtain the target image. It should be noted that the embodiment of the present application can be performed in the client device when the operating resources of the client device can meet the deployment and operation conditions of the large model.

[0047] Under the above operating environment, this application provides Figure 2 The image processing method shown. Figure 2 is a flow chart of an image processing method according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0048] Step S202, obtaining an image to be processed.

[0049] The image to be processed includes the target object.

[0050] The above-mentioned image to be processed may be an original image containing a specific target object, and may be a photograph, a scanned image, etc. Exemplarily, the image to be processed may be a photo of clothing worn by a model or folded at random, but is not limited thereto, and the image to be processed may be determined according to actual conditions.

[0051] The target object mentioned above may be an object to be identified and processed in the image to be processed. The target object may be an object to be displayed in the image to be processed, wherein the object to be displayed may include but is not limited to clothing, accessories, bags, etc. In the scene of clothing to white background image, the target object may be clothing to be displayed.

[0052] The above-mentioned images to be processed can be model pictures uploaded by merchants on e-commerce platforms, pictures of real objects, or images of clothing freely taken by users through mobile phones, cameras, etc. These images usually have complex backgrounds, and the clothing may be in different wearing states or in different postures and angles.

[0053] In an optional embodiment, after the image to be processed is acquired, some preprocessing may be performed on the image to be processed, such as cropping, scaling, color correction, etc., to ensure that the image size and format input into the algorithm model are consistent, which helps the model to better understand and process the image content.

[0054] Step S204: extract features from the image to be processed using a feature extraction model to obtain object features of the target object.

[0055] The above-mentioned feature extraction model (UNET-T2I) can be used to extract the features of the target object from the image, and these features can be shape, color, texture, etc. In the scene of clothing to white background image, the feature extraction model can be used to extract clothing features from the image containing clothing.

[0056] The above-mentioned item features may be data obtained by a feature extraction model that describes the characteristics of the target item. For example, for clothing, the item features may include information about the style, color, and material of the clothing.

[0057] The above feature extraction model can extract clothing features from any clothing image (it can be clothing worn by models or folded or tilted clothing in real photos). The "features" here can be understood as information such as color, texture, shape, size, position, etc. of clothing in the image. UNET-T2I can effectively learn and extract these clothing features from the input image by using the U-Net structure.

[0058] In an optional embodiment, the feature extraction model can accurately capture the detailed features of clothing, such as the material and pattern of the clothing. This information can be retained when the white-bottomed flat clothing image is generated later, ensuring that the generated image is as consistent as possible with the visual effect of the original clothing image. The feature extraction model not only focuses on the surface details of the clothing, but also understands the shape and structure of the clothing. Even if the clothing is worn on the model or folded or twisted in the original image, the original shape and flat state of the clothing can be restored through the extracted object features, which is the basis for image conversion. In the process of extracting object features, the model can automatically remove noise in the image, such as uneven lighting, shadows, cluttered background and other interference factors, and only retain features directly related to the clothing, which helps to improve the clarity and quality of the generated white-bottomed flat clothing image.

[0059] The feature extraction model in this application can adapt to various types of input images, including but not limited to pictures of models wearing clothes, real pictures of folding, etc., which greatly improves the scope of application and practicality of the algorithm and lowers the threshold for generating high-quality white-background flat clothing pictures. The analysis and processing of the image to be processed by the feature extraction model is a key step to ensure that the generated white-background flat clothing picture retains the details and shape of the clothing and removes background interference, which has a significant beneficial effect on improving the performance of the algorithm and the quality of the generated image.

[0060] Step S206: Based on the object features and the edge information of the target object, guide the image generation model to transform the image to be processed to obtain the target image.

[0061] Among them, other areas in the target image except the area where the target object is located are displayed through a preset texture, and the target object in the target image is in a preset state.

[0062] The above-mentioned edge information may be edge details of the target object in the image to be processed, and is used to define the outline of the target object in the image to be processed. The edge information may be extracted by an edge detection algorithm (Canny algorithm) to define the outer outline of the clothing. The above-mentioned edge information may be edge information of the target object in the target image when it is in a preset state, and the edge information may be determined by a template corresponding to a preset image, or the image to be processed may be predicted by an edge information prediction model to predict the outline of the target object when it is in a preset state. Exemplarily, the edge information may be the outline of the target object when it is in a flat state.

[0063] The above-mentioned image generation model can generate a model of a new image according to the extracted features. Here, the image generation model can be a model in an image-to-image restoration network (UNET-INP).

[0064] The target image may be an image obtained after conversion processing, wherein the target object is in a preset state and other areas are displayed by preset textures. The target image is a clothing image with a white background.

[0065] The above-mentioned preset texture may be a texture used to fill the area outside the target object in the target image, such as a white background, a light background, etc. The preset texture may make the target object more prominent in the target image.

[0066] The above-mentioned preset state may be a state in which the target object is expected to be presented in the target image, such as flattened, unfolded, etc. By setting the target object to the preset state, the target object may be displayed more clearly in the target image, thereby improving the image generation effect of the target image.

[0067] In an optional embodiment, any image containing a target object can be efficiently converted into a new image with a specific background and object state, which is suitable for multiple fields such as e-commerce, product catalog production, virtual reality, etc. First, the system receives any image as input, extracts item features using a feature extraction model, and obtains item edge information at the same time to accurately retain the shape of the object. Then, a guided image generation model is used to transform the image to be processed to obtain a target image. During the training process, the image generation model uses the features, edge information and preset background of the input image as conditions to generate a new image with a uniform background and preset object state. This application not only significantly reduces the cost and threshold of high-quality image production without the need for professional shooting equipment, but also realizes the automation and efficiency of image processing, improves the consistency and professionalism of image quality, and has the flexibility to process various object shapes, sizes and states.

[0068] The object features and the edge information of the target object can be used to guide the image generation model to convert the original image to be processed into the target image. The target image is characterized in that except for the area where the target object (such as clothing) is located, the rest of the background area is displayed with a preset texture (such as a pure white background), and the target object is adjusted to a preset state (such as a tiled state) in the target image. It should be noted that the preset texture and preset state can be set according to the needs of the user. For example, the preset texture can be set to a light-colored background, and the preset state can be set to a three-dimensional state. The specific setting effects of the preset texture and preset state are not limited here, and can be set according to actual conditions.

[0069] In the fields of e-commerce, virtual fitting, product catalog production, etc., merchants do not need to specifically shoot images of clothing in a flat state. They only need to provide any image containing clothing, and the algorithm can automatically convert it into the required white-background flat clothing image. This greatly reduces the cost and difficulty of producing professional product images and improves efficiency, especially in large-scale product catalog management, which can save a lot of time and resources. Through precise edge information and object feature control, the model can generate more natural and visually aesthetic white-background flat clothing images, avoiding problems such as distortion and deformation that may occur during manual processing, and improving the consistency and quality of the image. The algorithm has fewer restrictions on input images, whether it is a picture of clothing worn by a model, a folded physical picture, or an image with a complex background, it can be effectively processed, which means that merchants can use existing resources more flexibly without worrying about the shooting conditions or state of the original image.

[0070] The entire conversion process can be automated, reducing the reliance on professional image processing personnel, making image processing more convenient and faster. This automated processing has a particularly significant advantage in scenarios where a large number of images need to be processed in batches. The output white-bottomed flat clothing images can be presented with a preset pure white background and preset state (such as flat). This standardized output is conducive to the uniformity and professionalism of product display and improves the consumer browsing experience.

[0071] Using object features and edge information to guide the transformation processing of the image generation model can not only significantly improve the efficiency and quality of image processing and reduce production costs, but also improve the flexibility of merchants' operations, realize automated and standardized image processing, and bring significant beneficial effects to related industries such as e-commerce.

[0072] In the scenario of converting clothing to a white background image, first, obtain an image to be processed that contains clothing. Then, use a feature extraction model to extract clothing features in the image to be processed. Next, based on clothing features and clothing edge information, use an image generation model to convert the image to be processed into a clothing image with a white background. This application can generate high-quality white background flat clothing images from clothing images worn by models or folded at random, and can also significantly reduce the time and resources required for merchants to produce such images. The entire process is automated, reducing dependence on professional photography and post-processing.

[0073] Through the above steps, an image to be processed is obtained, wherein the image to be processed includes a target object; a feature extraction model is used to extract features of the image to be processed to obtain object features of the target object; based on the object features and the edge information of the target object, the image generation model is guided to transform the image to be processed to obtain the target image, wherein other areas in the target image except the area where the target object is located are displayed by preset textures, and the target object in the target image is in a preset state, thereby achieving the purpose of improving the image processing effect; it is easy to notice that the feature extraction module can be used to capture the details of the target object to form object features, and the edge information of the target object provides conditional guidance for the process of the image generation model transforming the image to be processed, ensuring that the boundaries of the target object in the generated target image are clear, and the area where the non-target object is located can be filled by the preset texture, while ensuring that the target object is in a preset state, thereby improving the visual effect of the target image, thereby improving the display effect of the target object in the target image, and thus solving the technical problem of poor image processing effect in the related technology.

[0074] In the above embodiments of the present application, based on the object features and the edge information of the target object, the image generation model is guided to perform transformation processing on the image to be processed to obtain the target image, including: determining the position information of the target object in the image to be processed based on the object features; performing mask processing on the image to be processed using the position information to obtain the mask image; based on the mask image, the object features and the edge information, the image generation model is guided to perform transformation processing on the image to be processed to obtain the target image.

[0075] The above-mentioned position information may be the geometric position and size of the target object in the image. The bounding box, center point coordinates, size, shape, direction and relationship with other elements in the image of the target object may be determined according to the object features, thereby determining the position information of the target object in the image to be processed.

[0076] In an optional embodiment, a binary segmentation mask can be generated, in which 1 represents the clothing area and 0 represents the background. The position bounding box of the clothing in the image can be located based on the position information, and then a mask covering the clothing is created based on the bounding box. For the clothing worn by the model, the key points of the human body can be detected first, and then the outline of the clothing can be estimated based on the connection relationship between the key points to create a mask. A binary image with the same size as the original image can be created, usually called a mask image. In this binary image, the clothing area is marked as 1 (or true), and the rest of the image is marked as 0 (or false). In this way, a mask is formed that can be used for subsequent processing to distinguish between the clothing area and the background.

[0077] After obtaining the mask image, the mask image can be applied to the image to be processed to highlight or process the clothing area while ignoring the background. After masking, various image processing or algorithm applications can be performed on the clothing area, such as color adjustment, texture analysis, defect detection, etc. The above mask image is used to guide the subsequent white background tile conversion, that is, when generating the white background tile map, ensure that the clothing area is converted while the background area remains white.

[0078] Through the above steps, a specific area in the image to be processed (such as the area where the target object is located) can be effectively operated, while other areas can be ignored or protected.

[0079] In an optional embodiment, the clothing features and position information in the image to be processed can be identified, and the clothing part can be separated by mask processing. Then, the edge information is used to define the outline of the clothing to ensure the accuracy of the shape of the clothing in the generated image. Finally, combined with this information, the trained image generation model is used for transformation processing to generate a clothing target image with a white background and flat surface. The object features (such as the color and pattern of the clothing) and edge information (i.e. the outline of the clothing) can be used to effectively extract the main part of the clothing through the intermediate mask processing stage, while ensuring the integrity of its shape and features. This process guides the image generation model to perform transformation, and finally obtains a clothing image with a white background and flat surface, which overcomes the limitations of the traditional method on clothing images, such as the model's wearing state, clothing folding or background clutter, significantly reduces the threshold and cost of obtaining a standard white background and flat clothing image, improves the standardization of the image, and facilitates the display and processing of goods in e-commerce scenarios.

[0080] For example, suppose there is an image of a model wearing a dress with complex patterns, which contains the model, a complex background, and part of the dress's folds. Using the algorithm in this solution, the item features of the dress are first identified, including its pattern, color, etc., as well as the edge information of the dress extracted by the edge algorithm. Next, mask processing is applied to retain only the area of ​​the dress, and an image with a white background is generated, in which the position of the dress is transparent or black (indicating the area to be generated). Then, the dress features with noise, the mask map, and the background features are input into the image generation model. After the model's transformation processing, a dress image with a white background is finally obtained, in which the dress's pattern, color, and other features are completely retained, and it is presented in a flat state, with a uniform white background. This process not only solves the problem of standardization of clothing images, but also greatly reduces the difficulty of obtaining white-background flat clothing images, providing e-commerce platform merchants with an efficient and low-cost image processing solution.

[0081] In the above embodiments of the present application, based on the mask image, object features and edge information, the image generation model is guided to transform the image to be processed to obtain a target image, including: using the feature extraction model to extract features of the mask image to obtain first region features and second region features, and extracting features of the edge information to obtain edge features, wherein the first region features are region features of the region where the target object is located in the image to be processed, and the second region features are region features of other regions in the image to be processed except the region where the target object is located; using the feature extraction model to extract features of the edge information to obtain edge features; transforming the image to be processed based on the first region features, the second region features, the edge features and the object features to obtain the target image.

[0082] The mask image is a special binary image used to mark the target area and the background area. Usually, the target area is marked as white (or 1) and the background area is marked as black (or 0). For example, in the present application, the mask image can be a binary image with the outline of the clothing marked, where the white part represents the location of the clothing and the black part represents the area outside the clothing.

[0083] The above conversion process refers to the process of converting an input image into an output image with specific attributes. In the present application, the conversion process is to convert a clothing image containing a model or folds into a clothing image with a white background.

[0084] In an optional embodiment, a feature extraction model can be used to extract the features of the target object in the image to be processed to obtain a first region feature and a second region feature, wherein the first region feature corresponds to the feature of the target object, and the second region feature represents the background feature other than the target object. Edge information can be feature extracted to obtain edge features. Finally, these features are input into an image generation model, which performs transformation processing based on these features to generate a target image, i.e., a white background flat clothing image.

[0085] A two-stage feature extraction and conversion process can be used to effectively convert any image containing clothing into a standard white-background flat-lay clothing image. Through the collaborative work between the models, not only the detailed visual features of the clothing are extracted, but also the precise contour information of the clothing is retained. Based on these rich and precise features, the image generation model can generate accurate white-background flat-lay clothing images even if the clothing in the input image is worn on the model or folded, greatly reducing the cost and complexity of producing such images, and improving efficiency and image quality.

[0086] In the above embodiment of the present application, the processed image is transformed based on the first region feature, the second region feature, the edge feature and the object feature to obtain a target image, including: inputting the first region feature, the second region feature, the edge feature and the object feature into the input layer of the image generation model, inputting the first intermediate feature of the feature extraction model into the intermediate layer of the image generation model, and outputting the target image through the output layer of the image generation model, wherein the first intermediate feature is the feature output by the intermediate layer of the feature extraction model.

[0087] In the above steps, we mentioned "intermediate layers" and "features of intermediate layer inputs". These terms are based on the neural network architecture in deep learning, especially image-to-image conversion models like U-Net. Let's explain these concepts and the meaning of the entire steps step by step.

[0088] The above-mentioned middle layer refers to the layer between the input layer and the output layer in the neural network. In the U-Net architecture, the middle layer involves the process of downsampling and upsampling of features, so that the local and global features of the image can be captured. The uniqueness of the U-Net design is that it has an encoder (downsampling path) and a decoder (upsampling path), and the middle layer is located at the intersection of these two parts. The middle layer can be any layer in the model except the input layer and the output layer. Among them, the middle layer can be understood as the stage where the model extracts and compresses deep features of the input image, which is a key part of the model architecture.

[0089] The first intermediate feature mentioned above can be the feature output by the intermediate layer of the feature extraction model, which contains the depth information and high-level representation of the image to be processed. These features are generated through a series of operations such as convolution, regularization and activation functions, and can capture key features such as shape, texture and color of the image.

[0090] In an optional embodiment, a feature extraction model (such as UNET-T2I) is used to extract features of the input image to be processed (i.e., any clothing image) to obtain two region features (first region features and second region features) as well as edge features (Canny edges) and object features (i.e., clothing features).

[0091] The above-extracted features (first region features, second region features, edge features, and object features) are input into the input layer of the image generation model. Here, edge features and object features can provide additional guidance information to help the model understand the boundaries and internal details of clothing in the image. At the same time, the first intermediate features output by the intermediate layer of the feature extraction model are injected into the intermediate layer of the image generation model. This process can be understood as passing high-level feature information from the feature extraction model to the image generation model to enhance the reconstruction ability of the generation model and ensure that the generated image has rich details and accurate shapes. Finally, the target image, that is, the converted white-bottom tiled clothing image, is output through the output layer of the image generation model. At this stage, the model reconstructs the image based on the input features and the injected intermediate layer features to generate a white-bottom clothing image with a pure background, while keeping the tiled state and clear details of the clothing.

[0092] Through the above steps, the processing power of the deep learning model can be used to convert complex clothing images containing background noise or models into clean, standard white-bottom flat clothing images, thereby reducing the cost and difficulty of producing such professional images and improving efficiency and automation.

[0093] In the above embodiment of the present application, the image generation model also includes: a control network, and the method also includes: using the control network to perform edge detection on the target object in the image to be processed to obtain edge information.

[0094] The control network (Control Net) mentioned above is a neural network that can be used to expand the function of the image generation model by adding additional conditional information to control certain properties of the generated image. In this application, Control Net is used to extract and process the edge information of the target object.

[0095] In an optional embodiment, after the image to be processed is input, the edge information of the target object in the image can be extracted through the control network; then, the edge information is sent to the image generation model together with the original image features, and the image generation model will generate a clothing image with a white background based on the input features and edge information. The extraction and use of the above edge information ensure that the generated clothing image can maintain the shape and details of the original clothing. The use of the control network to extract edge information can accurately capture the shape and details of the target object, even if the shape of the target object in the input image is irregular or there is a complex background. The control network significantly improves the accuracy and stability of image conversion, reduces the restrictions on the input image, thereby reducing the cost and threshold of image production, and improving the quality and efficiency of image generation.

[0096] For example, suppose a merchant has a photo of a model wearing a dress with complex patterns, with a busy urban street scene as the background. First, edge detection is performed on the photo to obtain edge features, and the outline information of the dress can be obtained based on the edge features. Then, the image features, outline information, and features of the white background of the dress are fed into the image generation model. The model generates a dress image with a white background based on the input information, in which the pattern and outline details of the dress are accurately retained, while the urban background is completely removed. The generated image conforms to the standard format of e-commerce display, greatly improving the commercial value and efficiency of clothing images. Edge features can be predicted by a separate outline prediction module, that is, predicting the outline shape of the clothes after they are laid flat. You can also search for matching clothing outline images in the template library based on any clothing image. The template library contains a large number of outline images of existing clothing after they are laid flat.

[0097] In the above embodiment of the present application, the processed image is transformed based on the first region feature, the second region feature, the edge feature and the object feature to obtain a target image, including: inputting the first region feature, the second region feature, the edge feature and the object feature into the input layer of the image generation model, respectively inputting the first intermediate feature of the feature extraction model and the second intermediate feature of the control network into the intermediate layer of the image generation model, and outputting the target image through the output layer of the image generation model, wherein the second intermediate feature is the feature output by the intermediate layer of the image generation model.

[0098] The first intermediate feature mentioned above refers to the feature generated during the processing of the feature extraction model. The feature extraction model can extract the features of clothing from any clothing image. These features contain information about the shape, texture, color and texture of the clothing. The first intermediate feature is output at the intermediate layer of the feature extraction model. It has not yet been completely abstracted into the final high-level feature, but it retains enough details for subsequent processing.

[0099] The second intermediate features mentioned above are the steps of injecting Control Net features during training and inference, and refer to the intermediate layer features generated by Control Net after processing the outer contour of the clothing. These features contain detailed information about the shape and contour of the clothing, and can be used to guide the image generation model to maintain the accurate shape of the clothing when generating a flat clothing image with a white background.

[0100] Control Net can perform a series of convolution and downsampling operations on the processed image in its network structure to generate multi-scale edge features. These features are then injected into the corresponding layer of the image generation model, fused with the clothing features extracted by the image generation model, and jointly guide the generation process. In this way, when the image generation model generates a white-bottomed flat clothing image, it can use the edge information provided by Control Net to ensure that the generated clothing image not only has the correct color, texture and other details, but also can accurately reflect the original shape and outline of the clothing, avoiding the distortion or distortion of the clothing shape in the generated image. Through this process, the present application can effectively process clothing images in complex situations including models, folding, tilting, etc., generate high-quality white-bottomed flat clothing images, and reduce the threshold and cost of making such images.

[0101] In a neural network, the intermediate layer is located between the input layer and the output layer, and usually includes multiple convolutional layers, regularization layers, activation layers, etc. Its function is to extract and transform features based on the original input data. The intermediate layer can capture features at different levels of the data. For example, in image processing, the shallow intermediate layer (close to the input layer) may capture low-level features such as edges and textures, while the deep intermediate layer (close to the output layer) may learn more abstract and advanced features, such as the shape and structure of objects. In the above image generation model, the intermediate layer not only receives the noisy clothing features (as direct input), but also receives the first intermediate features and the second intermediate features from the feature extraction model and the control network. These features are used to provide additional information and guidance in the image generation process.

[0102] In an optional embodiment, the first region features, the second region features, the edge features, and the item features can be input into the input layer of the image generation model. These features are derived from the original clothing image, the background image, and the contour information of the clothing, respectively. In the middle layer of the image generation model, the first intermediate features of the feature extraction model (UNET-T2I) and the second intermediate features of the control network (Control Net) are added to the processing process. These intermediate features provide deeper information about the clothing, helping the model to understand the details and contours of the clothing, so as to generate the target image more accurately. Finally, the target image, i.e., the white background tiled clothing image, is generated through the output layer of the image generation model. This process combines the input features and the processing results of the middle layer, and uses the skills learned by the model to transform the noise features, background features, contour information, and item features into a clear, standardized image with tiled clothing and a white background.

[0103] In the scenario of converting a white background flat clothing image, first, the features of the original clothing image, the features of the white background flat image after noise processing, the contour edge features, and the specific features of the object can be input into the input layer of the model. Then, in the middle layer of the model, the first intermediate features of the feature extraction model and the second intermediate features of the control network are injected to provide deeper information to guide the generation process. Finally, the output layer of the model generates the target image, i.e., the white background flat clothing image. When processing images in e-commerce scenarios, the method proposed in this application can significantly improve the efficiency and quality of generating white background flat clothing images and reduce manual intervention. Multiple features can be fused, including regional features, edge features, and item features. Through the synergy of two feature extraction models and the control network, even if the clothing is worn on the model, partially folded, or tilted in the input image, the white background flat clothing image can be accurately generated. This not only reduces the cost of making white background flat clothing images, but also improves the level of automation, providing merchants with a more convenient image creation space.

[0104] For example, suppose a merchant has a photo of a model wearing a shirt (image1), but needs a high-definition image (image2) of a shirt with a white background and flat surface for product display. Traditional methods may require specialized photographers and post-processing, which is costly and time-consuming. In this application, first, UNET-T2I extracts the features of the shirt in image1, including its color, style, etc.; Control Net obtains the edge contour of the shirt through the Canny algorithm to form edge features; at the same time, UNET-INP receives the image2 features after noise addition, the mask of the shirt's location, and the white background features. During the training process, the intermediate features generated by UNET-T2I and Control Net are also injected into the middle layer of UNET-INP to more accurately understand the shape, position, and texture of the shirt. Finally, UNET-INP outputs the target image, i.e., a shirt image with a white background, which does not require additional flat images as input, reducing the cost and threshold of obtaining a white background flat clothing image.

[0105] In the above embodiment of the present application, the method further includes: extracting contour information of the target object in the image to be processed based on the mask image; and performing edge detection on the image to be processed based on the contour information to obtain edge information.

[0106] The mask image is used to identify the position and shape of the region of interest in the image to be processed. The mask image is used to accurately identify and separate the clothing parts in the clothing image, even if the clothing is worn on the model or folded. For example, there is an image of a model wearing a skirt. Through mask image processing, a black and white image that highlights the outline of the skirt and completely blocks the model and other backgrounds can be obtained, where the skirt part is white (or 1) and the rest is black (or 0).

[0107] The above-mentioned contour information refers to the edge and shape features of the object, which can be used for object recognition and positioning, and the contour information can be extracted from the mask image through the edge detection method. For example, taking the image to be processed containing a skirt as an example, the contour information may include the waistline, hem edge and pleat edge of the skirt, which is used to keep the shape of the clothing accurate when subsequently generating a white background flat clothing image.

[0108] The above-mentioned edge detection is used to find the boundary of an object or the area where the surface properties change from the image. Among them, edge detection is used to extract the edge features of the target object from the mask image. For example, after edge detection is performed on the mask image of a skirt, the contour lines of the skirt can be clearly outlined, providing a basis for subsequent image processing.

[0109] The above-mentioned edge information is the result of edge detection, which contains the position information of the boundary of the object in the image. The edge information can be used to guide the accurate reconstruction of the flat shape of the clothing when generating the flat clothing image with a white background. For example, the edge information of the skirt may include the outermost edge of the skirt, the turning point of the fold, etc. This information helps to maintain the authenticity and accuracy of the clothing shape when generating the flat clothing image with a white background.

[0110] In the process of converting any clothing image into a white-background tiled clothing image, in addition to using Control Net to extract the Canny edge image, the shape of the generated clothing can also be directly controlled by the shape of the mask. By processing the mask image, the contour information of the target clothing is extracted, and then edge detection is performed to obtain more accurate edge information. This information can be used as a control signal in the subsequent image generation process to guide the model to generate a white-background tiled clothing image consistent with the original clothing shape.

[0111] In this application, through mask image extraction and edge detection, the original shape of clothing can be accurately identified and retained, even when the clothing is worn on a model or folded. On the basis of identifying clothing, the edge details of clothing can be accurately detected and retained, so that when generating a white-background tiled clothing image, the shape and details of the clothing can be maintained to the maximum extent, thereby improving the quality of the generated image. This application reduces the threshold and cost of generating white-background tiled images and improves the flexibility and efficiency of image processing.

[0112] For example, if there is a photo of a model wearing a dress with a complex pattern and a cluttered background. First, a separate contour prediction model can be used to predict the contour shape of the dress after it is flattened. Alternatively, a matching clothing contour image in the template library can be searched based on the photo. The contour information in the clothing contour image and the shape of the mask image can be used as input to directly guide the model to generate a dress image with a white background, in which the shape, pattern, and details of the dress are highly consistent with the original image, and the background is unified as white. This process can not only automatically process complex clothing shapes, but also efficiently generate high-quality white-background flat clothing images that meet e-commerce needs without losing details.

[0113] In the above embodiment of the present application, the method further includes: responding to a drawing instruction acting on the image to be processed, and generating edge information of the target object according to the drawing instruction.

[0114] It can provide users with an interactive way to define or adjust the edge information of the target object, and provide more precise guidance conditions for subsequent image generation models.

[0115] Users can directly draw or mark the boundaries of the target object on the image to be processed. These drawing operations are regarded as drawing instructions. For example, if the image to be processed is a photo of a model wearing a dress with a complex pattern, the user can use the drawing tool to outline the outline of the dress as the input of edge information.

[0116] The edge information generated according to the drawing instructions can be regarded as a user-defined control signal. By setting the user interaction method, the flexibility and customization of the system are increased. Users can manually adjust or correct the errors that may be caused by automatic edge detection according to actual needs or preferences to ensure the accuracy of edge information, thereby generating a white-background flat clothing image that meets the requirements better.

[0117] In the scenario of this application, by applying the user interaction method to the image generation task based on the diffusion model, and taking the edge information drawn by the user as one of the conditional inputs, together with the object features, the first area features and the second area features, it is used to guide the transformation processing process of the image generation model, thereby obtaining a higher quality target image.

[0118] This application can not only deal with the limitations brought by automatic edge detection, but also adapt to a wider range of application scenarios, where specific user input or fine-tuning is required. For example, when processing certain clothing with very complex or irregular shapes, automatic edge detection may not be able to fully meet the requirements. At this time, the addition of user drawing instructions can provide more accurate edge information, ensuring that the generated white-bottomed flat clothing image has accurate shape, rich details, and better visual effects, further improving the efficiency and quality of image processing.

[0119] In the above embodiment of the present application, the method also includes: obtaining a sample image to be processed and a sample target image corresponding to the sample image to be processed, wherein the sample image to be processed includes a sample object; using a feature extraction model to extract features of the sample image to be processed to obtain sample item features of the sample item; based on the sample item features and sample edge information of the sample item features, guiding the initial image generation model to transform the sample image to be processed to obtain a sample transformed image; constructing a loss function based on the sample transformed image and the sample target image; and using the loss function to update the model parameters of the initial image generation model to obtain an image generation model.

[0120] The above steps are the image generation model training process, which generates more accurate target images by updating the model parameters.

[0121] The above loss function is used to measure the difference between the sample transformation image generated by the model and the sample target image. Loss functions include but are not limited to mean square error, cross entropy error, etc. By calculating the value of the loss function, the quality of the image generated by the model and the degree of closeness between the sample transformation image and the real sample target image can be intuitively evaluated.

[0122] At the beginning of model training, a large amount of sample data needs to be prepared, namely, sample images to be processed and their corresponding sample target images. The sample images to be processed can be any images containing target objects, and the sample target images are the ideal results after processing. For example, for the scene of clothing to white background image conversion, the sample images to be processed are clothing images worn by models or real photos, and the sample target images are standard white background flat clothing images. The feature extraction model is used to extract features from the sample images to obtain the sample item features of the sample items.

[0123] Use a feature extraction model (such as UNET-T2I) to analyze the sample images to be processed and extract the features of the target objects in the images, including color, texture, shape, pattern, etc. These features will serve as important inputs for subsequent model training to help the model learn the key visual attributes of objects.

[0124] Based on the sample item features and the sample edge information of the sample item features, the initial image generation model is guided to transform the sample to-be-processed image to obtain the sample transformed image. The initial image generation model (such as UNET-INP) receives the sample item features and sample edge information as input during the training phase. These information guide the model to try to transform the sample to-be-processed image into the style of the sample target image. The output sample transformed image is the image generated by the model under the current parameter settings, and its goal is to be as close to the sample target image as possible.

[0125] A loss function is constructed based on the sample conversion image and the sample target image, and the model parameters of the initial image generation model are updated using the loss function to obtain the image generation model. Through gradient descent or other optimization algorithms, the model will adjust its internal parameters based on the feedback of the loss function so as to generate a sample conversion image that is closer to the sample target image in the next round of training. This process is repeated until the performance of the model on the training data set reaches the predetermined standard. At this time, the initial image generation model has been updated with parameters and has become a trained image generation model.

[0126] In the process of machine learning, through iterative optimization of a large amount of sample data, the image generation model can learn how to convert the image to be processed into the target image based on the object features and edge information. This is not only applicable to the scene of clothing to white background image, but can also be applied to a wider range of image processing tasks. Through training, the model can gradually improve the quality and accuracy of its generated images, reduce dependence on manual processing, and improve the automation level and efficiency of image processing.

[0127] Figure 3 is a flow chart of clothing image processing according to an embodiment of the present application, such as Figure 3 As shown, the method includes:

[0128] Step S301, in response to receiving a clothing image upload instruction, displaying the clothing image uploaded by the user on the interactive interface;

[0129] Step S302 , in response to receiving a clothing image generation instruction, generating a white background tile image corresponding to the clothing image.

[0130] Two U-Nets are used in this application. One is UNET-T2I, which is used as an information extraction module (that is, the feature extraction model mentioned above) to extract clothing features in any clothing image. The other is UNET-INP. In addition to inputting clothing features after adding noise, generating the mask of the area, and the features of the white background image after removing the mask area, it is also necessary to inject the features extracted by UNET-T2I and Control Net, where the input of Control Net is the Canny of the outer contour of the clothing after tiling.

[0131] Figure 4 is a schematic diagram of an image generation model structure according to an embodiment of the present application, such as Figure 4 As shown, the image generation model includes a control network (Control Net), a feature extraction model (UNET-T2I), and an image generation model (UNET-INP). The feature extraction model is used as an information extraction module to extract clothing features in clothing images. The image generation model can input the clothing features after noise addition, clothing features, and features extracted by the control network to generate a mask image of the region.

[0132] During the training process of the image generation model, the input of the feature extraction model can be the features of any clothing image, which may or may not contain a model. The input of the control network can be the outer contour edge of the white-background flat clothing image corresponding to any clothing image. The input of the image generation model can be multiple different features, namely, the features of the white-background flat clothing image after adding noise, the mask image containing the clothing position, and the features of the white background image after removing the area where the mask image is located. At the same time, the intermediate features of the control network and the feature extraction model can be injected into the intermediate layer.

[0133] During the inference process of the image generation model, the input of the feature extraction model can be the features of any clothing image, which may or may not contain a model. The control network matches the features of any clothing image to obtain the outer contour edge, or the outer contour edge of the clothing is selected by the user. The input of the image generation model can be multiple different features, such as noise features, mask images containing clothing positions, and white background features after removing the mask image area. At the same time, the intermediate features of the control network and the feature extraction model can be injected into the intermediate layer.

[0134] Figure 5 is a training flow chart of an image generation model according to an embodiment of the present application, such as Figure 5 As shown, the method includes:

[0135] Step S501, collecting any clothing image and a sample white-background flat clothing image corresponding to the any clothing image;

[0136] Step S502, extracting a mask image corresponding to a clothing position in the white-background flat clothing image;

[0137] Step S503, extracting features of the arbitrary clothing image and the white-background flat clothing image respectively;

[0138] Step S504, adding noise to the features and inputting them into the image generation model, and the model outputs a flat clothing image with a white background;

[0139] Step S505 , constructing a loss function according to the sample white-background flat clothing image and the white-background flat clothing image, and updating the model parameters of the image generation model based on the loss function.

[0140] Optionally, after updating the model parameters of the image generation model, it can be determined whether the termination condition of the model training is met. If the termination condition is met, the current model parameters can be retained and the training process of the image generation model is terminated. If the termination condition is not met, step S501 can be continued until the training process meets the termination condition of the model training, the final model parameters are retained, and the training is terminated.

[0141] The above-mentioned termination conditions refer to the conditions under which training automatically stops when the training process reaches a certain predetermined standard or meets specific requirements. These conditions are set to avoid unnecessary long-term training while ensuring that the model achieves the expected performance. The termination condition can be to set a maximum number of iterations, and the model training automatically stops when this number is reached. The termination condition can also be that the loss function value no longer decreases significantly in several consecutive training cycles, or drops below a predetermined threshold, which indicates that the model has learned most of the patterns in the data, and continuing training will not improve the performance much, and may even start to overfit. In practical applications, multiple conditions can be combined to decide when to stop training to ensure that the model can learn fully without being overtrained. For example, a maximum number of iterations can be set, and the performance of the validation set and the loss function value can be monitored at the same time. Once the preset convergence condition is reached or the performance of the validation set is no longer improved, the training is stopped even if the maximum number of iterations has not been reached.

[0142] Figure 6 is a schematic diagram of the reasoning process of an image generation model according to an embodiment of the present application, such as Figure 6 As shown, the method includes:

[0143] Step S601, inputting any clothing image and clothing outline into the image generation model;

[0144] Step S602: the image generation model outputs a flat clothing image with a white background.

[0145] In an optional embodiment, during the processing of the image generation model, the edge of the outer contour of the clothing provided by the control network can be replaced with the shape of the clothing generated by controlling the mask image, that is, when the model is input, the outer contour shape of the mask image and the clothing can be kept consistent.

[0146] In this application, there is no need to obtain the flat clothing images in advance. Any clothing images containing clothing can be input into the image generation model. Therefore, this application can significantly reduce the cost and threshold of white-background flat clothing images. This application adopts a new network architecture and uses two U-Nets to generate white-background flat clothing images, which removes the restrictions on input clothing images. Any clothing images can be input. The input clothing images can be worn on models, can be partially folded, and can be tilted, reducing the restrictions on input clothing images.

[0147] In the field of clothing image processing, this application can convert casually taken clothing images (including model pictures, real clothing pictures, etc.) into standard white-background flat clothing images. When the clothing is worn on the model, or the clothing is folded and the background is messy, the algorithm of this application can generate white-background flat clothing images. A new SDXL-based model structure is used to generate white-background flat clothing images. The model structure contains two U-Nets, and Control Net is mounted as the conditional space. It only needs to obtain any image containing clothing to complete the operation of converting any clothing image into a white-background flat clothing image, which can significantly reduce the threshold for obtaining white-background flat clothing images.

[0148] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0149] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0150] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0151] According to an embodiment of the present application, a model training method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here. Figure 7is a flow chart of a model training method according to an embodiment of the present application, such as Figure 7 As shown, the method includes:

[0152] Step S702, obtaining a sample to-be-processed image and a sample target image corresponding to the sample to-be-processed image;

[0153] The sample images to be processed include sample objects.

[0154] Step S704, extracting features from the sample image to be processed using a feature extraction model to obtain sample item features of the sample item;

[0155] Step S706, based on the sample object feature and the sample edge information of the sample object feature, guiding the initial image generation model to transform the sample image to be processed to obtain a sample transformed image;

[0156] Step S708, constructing a loss function based on the sample conversion image and the sample target image;

[0157] Step S710, using the loss function to update the model parameters of the initial image generation model to obtain the image generation model.

[0158] Through the above steps, a sample image to be processed and a sample target image corresponding to the sample image to be processed are obtained, wherein the sample image to be processed includes a sample object; a feature extraction model is used to extract features of the sample image to be processed to obtain sample object features of the sample object; based on the sample object features and the sample edge information of the sample object features, the initial image generation model is guided to transform the sample image to be processed to obtain a sample transformed image; a loss function is constructed based on the sample transformed image and the sample target image; the model parameters of the initial image generation model are updated using the loss function to obtain an image generation model, thereby achieving the purpose of improving the image processing effect; it is easy to notice that the feature extraction module can be used to capture the details of the target object, thereby forming the object features, and the edge information of the target object is used to provide conditional guidance for the image generation model to transform the image to be processed, thereby ensuring that the boundaries of the target object in the generated target image are clear, and the area where the non-target object is located can be filled with a preset texture, while ensuring that the target object is in a preset state, thereby improving the visual effect of the target image, thereby improving the display effect of the target object in the target image, and thus solving the technical problem of poor image processing effect in the related technology.

[0159] According to an embodiment of the present application, an image processing method is also provided. Figure 8 is a flowchart of an image processing method according to an embodiment of the present application, such as Figure 8 As shown, the method includes:

[0160] Step S802, obtaining clothing images;

[0161] The clothing image includes target clothing.

[0162] Step S804, extracting features from the clothing image using a feature extraction model to obtain clothing features of the target clothing;

[0163] Step S806 , based on the clothing features and the edge information of the target clothing, guiding the image generation model to transform the clothing image to obtain the target clothing image.

[0164] Among them, other areas in the target clothing image except the area where the target clothing is located are displayed through preset textures, and the target clothing in the target clothing image is in a preset state.

[0165] The above steps S802 to S806 describe in detail the process of generating a target clothing image for standard display from any clothing image. This process combines feature extraction and image generation technology in deep learning to improve the efficiency and quality of image processing. The specific beneficial effects are as follows:

[0166] The feature extraction model is used to perform deep feature extraction on clothing images, ensuring that the model can accurately capture the key features of clothing such as texture, color, and shape. This precise feature extraction is the basis for generating high-quality target clothing images, which enables the model to understand the essential attributes of clothing even in complex backgrounds or non-tiled clothing states. The edge information of the target clothing (such as the contour extracted by the Canny algorithm) can be used to guide the image transformation process. The addition of edge information helps the model to more accurately retain the shape and boundaries of clothing, prevents distortion of clothing shapes during the generation process, and ensures the accuracy of clothing shapes in the generated images.

[0167] The non-clothing area of ​​the target clothing image is displayed through preset textures, while ensuring that the clothing is in the preset state. This standardized processing not only unifies the background of the image, but also ensures the consistency of clothing display, greatly improving the visual effect and attractiveness of the image. Especially for e-commerce scenarios, the unified display standard enhances the recognition and professionalism of the products.

[0168] This application significantly reduces the resources and time required to produce standard clothing images through automated feature extraction and image generation. Merchants can quickly obtain high-quality clothing images that meet display needs without a dedicated shooting environment and post-processing team, greatly reducing costs and improving production efficiency. Regardless of the original state of the clothing image, whether it is worn by a model, folded, or with a complex background, the model can process and convert it into a standard tiled white background image. This wide adaptability allows merchants to use existing image resources without the need to take additional clothing tile images, enhancing the flexibility of image processing. The generated standardized target clothing images have a clean background and clear clothing display, which helps to improve the product display effect of the e-commerce platform, enhance the user browsing experience, and thereby increase the attractiveness of the product and the sales conversion rate.

[0169] Taking the e-commerce scenario as an example, suppose a merchant has a photo of a model wearing a specific style of clothing, but in order to display the product, a flat-laying white background clothing image is required. According to the above steps, first obtain the clothing image containing the target clothing; then, use the feature extraction model to extract the features of the clothing; next, combine the edge information and feature information, and use the image generation model to convert the original image into a target clothing image with a uniform white background and clothing flat-laying state. This process can automatically process a variety of clothing images and generate display images that meet the standards, significantly improving the efficiency and quality of image processing, providing merchants with convenient image editing tools, and also providing consumers with a better shopping experience.

[0170] Through the above steps, a clothing image is obtained, wherein the clothing image includes target clothing; a feature extraction model is used to extract features of the clothing image to obtain clothing features of the target clothing; based on the clothing features and the edge information of the target clothing, the image generation model is guided to transform the clothing image to obtain the target clothing image, wherein other areas of the target clothing image except the area where the target clothing is located are displayed by preset textures, and the target clothing in the target clothing image is in a preset state, thereby achieving the purpose of improving the image processing effect; it is easy to notice that the feature extraction module can be used to capture the details of the target clothing to form clothing features, and the edge information of the target clothing is used to provide conditional guidance for the image generation model to transform the clothing image, ensuring that the boundaries of the target clothing in the generated target clothing image are clear, and the area where the non-target clothing is located can be filled by the preset texture, while ensuring that the target clothing is in a preset state, thereby improving the visual effect of the target clothing image, thereby improving the display effect of the target clothing in the target clothing image, and thus solving the technical problem of poor image processing effect in the related art.

[0171] According to an embodiment of the present application, an image processing method is also provided. Fig. 9is a flowchart of an image processing method according to an embodiment of the present application, such as Fig. 9 As shown, the method includes:

[0172] Step S902, in response to an input instruction acting on the interactive interface, displaying the image to be processed on the interactive interface.

[0173] Wherein, the image to be processed includes a target object;

[0174] Step S904, in response to the processing instruction acting on the interactive interface, the target image corresponding to the image to be processed is displayed on the interactive interface.

[0175] Among them, other areas in the target image except the area where the target object is located are displayed by preset textures, and the target object in the target image is in a preset state. The target image is obtained by guiding the image generation model to transform the image to be processed according to the object features of the target object and the edge information of the target object, and the object features are obtained by extracting features from the image to be processed using the feature extraction model.

[0176] Steps S902 and S904 involve a functional description of an interactive image processing system, and its beneficial effects are mainly reflected in the following aspects:

[0177] The image to be processed is displayed in response to the input command acting on the interactive interface, so that the user can directly view and select the image to be processed in a friendly interface without having professional image processing knowledge or skills. This intuitive operation mode greatly simplifies the complexity of the user's participation in the image processing process and improves the user experience. By displaying the target image on the interactive interface, the user can instantly preview the processing results, including the preset state of the target object and the preset texture of the background, which provides the user with the opportunity to adjust and optimize the generated image. For example, the user can check whether the shape of the object is correct, whether the background is clean, or whether the state of the object meets the requirements, so as to make necessary modifications or adjustments to ensure that the final output image meets specific standards or requirements.

[0178] The user's processing instructions on the interactive interface (such as adjusting the shape of an object, modifying edge information, changing preset textures, etc.) can be immediately reflected in the displayed target image. This real-time feedback mechanism enables the user to quickly and directly see the effect of the instructions, improving the efficiency and flexibility of image processing.

[0179] The above steps combine automated image processing with user customization needs. The automated feature extraction and image generation process lowers the threshold for generating high-quality white-ground flat clothing images, while the user's adjustments and instructions on the interactive interface meet the needs of personalization and specific scenarios, such as adjusting the display angle of items and correcting edge details.

[0180] First, select or upload the image to be processed containing the target item on the interactive interface provided by the system. Then, the system automatically processes the image based on the user's instructions and generates the target image, i.e., a flat clothing image with a white background, using the feature extraction model and edge information. The generated target image is displayed on the interactive interface, and the user can visually check the processing results and optimize the image through further processing instructions until it meets specific needs.

[0181] The above interactive processing flow not only simplifies the complexity of image processing and reduces the requirements for professional skills, but also enhances the control and satisfaction of users, ensures the high quality and customization of generated images, and has a significant positive impact on improving the product display effect and user experience of e-commerce platforms. For example, merchants or individual users can quickly generate white-bottomed flat clothing images that meet e-commerce display standards without professional image editing knowledge, greatly improving the efficiency and quality of product listing.

[0182] Through the above steps, in response to the input instruction acting on the interactive interface, the image to be processed is displayed on the interactive interface, wherein the image to be processed includes the target object; in response to the processing instruction acting on the interactive interface, the target image corresponding to the image to be processed is displayed on the interactive interface, wherein other areas in the target image except the area where the target object is located are displayed by preset textures, and the target object in the target image is in a preset state, and the target image is obtained by guiding the image generation model to transform the image to be processed according to the object features of the target object and the edge information of the target object, and the object features are obtained by extracting the features of the image to be processed using the feature extraction model, thereby achieving the purpose of improving the image processing effect; it is easy to notice that the feature extraction module can be used to capture the details of the target clothing, thereby forming clothing features, and the edge information of the target clothing is used to provide conditional guidance for the image generation model to transform the clothing image, ensuring that the boundaries of the target clothing in the generated target clothing image are clear, and the area where the non-target clothing is located can be filled by the preset texture, while ensuring that the target clothing is in a preset state, thereby improving the visual effect of the target clothing image, thereby improving the display effect of the target clothing in the target clothing image, and thus solving the technical problem of poor image processing effect in the related art.

[0183] According to an embodiment of the present application, an image processing device for implementing the above-mentioned image processing method is also provided. Fig.10 is a schematic diagram of an image processing device according to an embodiment of the present application, such as Fig.10 As shown, the device 1000 includes: an acquisition module 1002 , an extraction module 1004 , and a conversion module 1006 .

[0184] Among them, the acquisition module is used to acquire the image to be processed, wherein the image to be processed includes the target object; the extraction module is used to use the feature extraction model to extract features from the image to be processed to obtain the object features of the target object; the transformation module is used to guide the image generation model to transform the image to be processed based on the object features and the edge information of the target object to obtain the target image, wherein other areas in the target image except the area where the target object is located are displayed by a preset texture, and the target object in the target image is in a preset state.

[0185] It should be noted that the acquisition module 1002, the extraction module 1004, and the conversion module 1006 correspond to steps S202 to S206 in the above embodiment, and the three modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors, and the above modules can also be run in the server 10 provided in the above embodiment as part of the device.

[0186] In the above embodiments of the present application, the conversion module is used to determine the position information of the target object in the image to be processed based on the object features; the image to be processed is masked using the position information to obtain a mask image; based on the mask image, object features and edge information, the image generation model is guided to perform conversion processing on the image to be processed to obtain a target image.

[0187] In the above embodiment of the present application, the conversion module is used to use the feature extraction model to extract features from the mask image to obtain the first region feature and the second region feature, and to extract features from the edge information to obtain edge features, wherein the first region feature is the region feature of the region where the target object is located in the image to be processed, and the second region feature is the region feature of other regions in the image to be processed except the region where the target object is located; the feature extraction model is used to extract features from the edge information to obtain edge features; and the image to be processed is converted based on the first region feature, the second region feature, the edge feature and the object feature to obtain the target image.

[0188] In the above embodiment of the present application, the conversion module is used to input the first area feature, the second area feature, the edge feature and the object feature into the input layer of the image generation model, input the first intermediate feature of the feature extraction model into the intermediate layer of the image generation model, and output the target image through the output layer of the image generation model, wherein the first intermediate feature is the feature output by the intermediate layer of the feature extraction model.

[0189] In the above embodiments of the present application, the image generation model further includes: a control network, and the device further includes: a detection module.

[0190] Among them, the detection module is used to use the control network to perform edge detection on the target object in the image to be processed to obtain edge information.

[0191] In the above embodiments of the present application, the conversion module is used to input the first region feature, the second region feature, the edge feature and the object feature into the input layer of the image generation model, and respectively input the first intermediate feature of the feature extraction model and the second intermediate feature of the control network into the intermediate layer of the image generation model, and output the target image through the output layer of the image generation model, wherein the second intermediate feature is the feature output by the intermediate layer of the image generation model.

[0192] In the above embodiments of the present application, the extraction module is also used to extract the contour information of the target object in the image to be processed based on the mask image; the detection module is used to perform edge detection on the image to be processed based on the contour information to obtain edge information.

[0193] In the above embodiments of the present application, the device further includes: a generating module.

[0194] The generation module is used to respond to the drawing instructions acting on the image to be processed, and generate edge information of the target object according to the drawing instructions.

[0195] In the above embodiments of the present application, the device further includes: a construction module and an update module.

[0196] Among them, the acquisition module is used to obtain the sample image to be processed and the sample target image corresponding to the sample image to be processed, wherein the sample image to be processed includes the sample object; the extraction module is used to use the feature extraction model to extract features of the sample image to be processed to obtain the sample item features of the sample item; the transformation module is used to guide the initial image generation model to transform the sample image to be processed based on the sample item features and the sample edge information of the sample item features to obtain the sample transformed image; the construction module is used to construct a loss function based on the sample transformed image and the sample target image; the update module is used to use the loss function to update the model parameters of the initial image generation model to obtain the image generation model.

[0197] According to an embodiment of the present application, a model training device for implementing the above-mentioned model training method is also provided. Fig.11 is a schematic diagram of a model training device according to an embodiment of the present application, such as Fig.11 As shown, the device 1100 includes: an acquisition module 1102 , an extraction module 1104 , a conversion module 1106 , a construction module 1108 , and an update module 1110 .

[0198] Among them, the acquisition module is used to obtain the sample image to be processed and the sample target image corresponding to the sample image to be processed, wherein the sample image to be processed includes a sample object; the extraction module is used to use the feature extraction model to extract features of the sample image to be processed to obtain the sample item features of the sample item; the transformation module is used to guide the initial image generation model to transform the sample image to be processed based on the sample item features and the sample edge information of the sample item features to obtain the sample transformed image; the construction module is used to construct a loss function based on the sample transformed image and the sample target image; the update module is used to use the loss function to update the model parameters of the initial image generation model to obtain the image generation model.

[0199] It should be noted that the acquisition module 1102, extraction module 1104, conversion module 1106, construction module 1108, and update module 1110 correspond to steps S702 to S710 in the above embodiment, and the five modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors, and the above modules can also be run in the server 10 provided in the above embodiment as part of the device.

[0200] According to an embodiment of the present application, an image processing device for implementing the above-mentioned image processing method is also provided. Fig.12 is a schematic diagram of an image processing device according to an embodiment of the present application, such as Fig.12 As shown, the device 1200 includes: an acquisition module 1202 , an extraction module 1204 , and a conversion module 1206 .

[0201] Among them, the acquisition module is used to acquire a clothing image, wherein the clothing image includes a target clothing; the extraction module is used to use a feature extraction model to extract features from the clothing image to obtain clothing features of the target clothing; the transformation module is used to guide the image generation model to transform the clothing image based on the clothing features and edge information of the target clothing to obtain the target clothing image, wherein other areas in the target clothing image except the area where the target clothing is located are displayed through a preset texture, and the target clothing in the target clothing image is in a preset state.

[0202] It should be noted that the acquisition module 1202, the extraction module 1204, and the conversion module 1206 correspond to steps S802 to S806 in the above embodiment, and the three modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors, and the above modules can also be run as part of the device in the server 10 provided in the above embodiment.

[0203] According to an embodiment of the present application, an image processing device for implementing the above-mentioned image processing method is also provided. Fig.13 is a schematic diagram of an image processing device according to an embodiment of the present application, such as Fig.13 As shown, the device 1300 includes: a first display module 1302 and a second display module 1304 .

[0204] Among them, the first display module is used to respond to the input instructions acting on the interactive interface, and display the image to be processed on the interactive interface, wherein the image to be processed includes the target object; the second display module is used to respond to the processing instructions acting on the interactive interface, and display the target image corresponding to the image to be processed on the interactive interface, wherein other areas of the target image except the area where the target object is located are displayed through preset textures, and the target object in the target image is in a preset state, and the target image is obtained by guiding the image generation model to transform the image to be processed according to the object features of the target object and the edge information of the target object, and the object features are obtained by extracting features from the image to be processed using the feature extraction model.

[0205] It should be noted that the first display module 1302 and the second display module 1304 correspond to steps S902 to S904 in the above embodiment, and the two modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors, and the above modules can also be run as part of the device in the server 10 provided in the above embodiment.

[0206] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in the above embodiments, as well as the application scenario and implementation process, but is not limited to the scheme provided in the above embodiments.

[0207] The embodiment of the present application may provide an electronic device, which may be any electronic device in a group of electronic devices. Optionally, in this embodiment, the electronic device may also be replaced by a terminal device such as a mobile terminal.

[0208] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0209] In this embodiment, the computer terminal can execute the program code in the method.

[0210] Optionally, Fig.14 is a structural block diagram of an electronic device according to an embodiment of the present application. Fig.14 As shown, the electronic device A may include: one or more ( Fig.14 (only one is shown) processor 102, memory 104, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0211] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the methods in the above embodiments. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0212] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: obtain an image to be processed, wherein the image to be processed includes a target object; use a feature extraction model to extract features of the image to be processed to obtain object features of the target object; based on the object features and edge information of the target object, guide the image generation model to transform the image to be processed to obtain a target image, wherein other areas in the target image except the area where the target object is located are displayed using a preset texture, and the target object in the target image is in a preset state.

[0213] Through the above steps, an image to be processed is obtained, wherein the image to be processed includes a target object; a feature extraction model is used to extract features of the image to be processed to obtain object features of the target object; based on the object features and the edge information of the target object, the image generation model is guided to transform the image to be processed to obtain the target image, wherein other areas in the target image except the area where the target object is located are displayed by preset textures, and the target object in the target image is in a preset state, thereby achieving the purpose of improving the image processing effect; it is easy to notice that the feature extraction module can be used to capture the details of the target object to form object features, and the edge information of the target object provides conditional guidance for the process of the image generation model transforming the image to be processed, ensuring that the boundaries of the target object in the generated target image are clear, and the area where the non-target object is located can be filled by the preset texture, while ensuring that the target object is in a preset state, thereby improving the visual effect of the target image, thereby improving the display effect of the target object in the target image, and thus solving the technical problem of poor image processing effect in the related technology.

[0214] Those skilled in the art will understand that Fig.14 The structure shown is for illustration only. The electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. The figure does not limit the structure of the above electronic devices. For example, the electronic device A may include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have the same Fig.14 Different configurations are shown.

[0215] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0216] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the method provided in the above embodiment.

[0217] Optionally, in this embodiment, the above storage medium may be located in any electronic device in a group of electronic devices in a computer network, or in any mobile terminal in a group of mobile terminals.

[0218] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: obtaining an image to be processed, wherein the image to be processed includes a target object; performing feature extraction on the image to be processed using a feature extraction model to obtain object features of the target object; and guiding an image generation model to transform the image to be processed based on the object features and edge information of the target object to obtain a target image, wherein areas of the target image other than the area where the target object is located are displayed using a preset texture, and the target object in the target image is in a preset state.

[0219] Through the above steps, an image to be processed is obtained, wherein the image to be processed includes a target object; a feature extraction model is used to extract features of the image to be processed to obtain object features of the target object; based on the object features and the edge information of the target object, the image generation model is guided to transform the image to be processed to obtain the target image, wherein other areas in the target image except the area where the target object is located are displayed by preset textures, and the target object in the target image is in a preset state, thereby achieving the purpose of improving the image processing effect; it is easy to notice that the feature extraction module can be used to capture the details of the target object to form object features, and the edge information of the target object provides conditional guidance for the process of the image generation model transforming the image to be processed, ensuring that the boundaries of the target object in the generated target image are clear, and the area where the non-target object is located can be filled by the preset texture, while ensuring that the target object is in a preset state, thereby improving the visual effect of the target image, thereby improving the display effect of the target object in the target image, and thus solving the technical problem of poor image processing effect in the related technology.

[0220] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and the computer program implements the method provided in the embodiment when executed by a processor.

[0221] The embodiments of the present application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which may be used to store a computer program, and when the computer program is executed by a processor, the method provided in the embodiments is implemented.

[0222] The embodiment of the present application further provides a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, the method provided in the above embodiment is implemented.

[0223] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0224] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0225] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0226] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0227] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0228] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire an image to be processed, wherein the image to be processed includes a target object; Using a feature extraction model to extract features from the image to be processed, to obtain object features of the target object; Based on the object features and the edge information of the target object, the image generation model is guided to transform the image to be processed to obtain a target image, wherein other areas of the target image except the area where the target object is located are displayed by a preset texture, and the target object in the target image is in a preset state.

2. The image processing method according to claim 1, characterized in that: Based on the object features and the edge information of the target object, guiding the image generation model to transform the image to be processed to obtain a target image, including: Determine the position information of the target object in the image to be processed based on the object feature; Using the position information, performing mask processing on the image to be processed to obtain a mask image; Based on the mask image, the object features and the edge information, the image generation model is guided to perform transformation processing on the image to be processed to obtain the target image.

3. The image processing method according to claim 2, characterized in that: Based on the mask image, the object features and the edge information, guiding the image generation model to transform the image to be processed to obtain the target image, including: Performing feature extraction on the mask image using the feature extraction model to obtain a first region feature and a second region feature, and performing feature extraction on the edge information to obtain an edge feature, wherein the first region feature is a region feature of a region where the target object is located in the image to be processed, and the second region feature is a region feature of other regions in the image to be processed except the region where the target object is located; Using the feature extraction model to extract features from the edge information to obtain edge features; The image to be processed is transformed based on the first region feature, the second region feature, the edge feature and the object feature to obtain the target image.

4. The image processing method according to claim 3, characterized in that: The process of converting the image to be processed based on the first region feature, the second region feature, the edge feature and the object feature to obtain the target image includes: The first region feature, the second region feature, the edge feature and the object feature are input into the input layer of the image generation model, the first intermediate feature of the feature extraction model is input into the intermediate layer of the image generation model, and the target image is output through the output layer of the image generation model, wherein the first intermediate feature is the feature output by the intermediate layer of the feature extraction model.

5. The image processing method according to claim 3, characterized in that: The image generation model further includes: a control network, and the method further includes: The control network is used to perform edge detection on the target object in the image to be processed to obtain the edge information.

6. The image processing method according to claim 5, characterized in that: The process of converting the image to be processed based on the first region feature, the second region feature, the edge feature and the object feature to obtain the target image includes: The first region feature, the second region feature, the edge feature and the object feature are input into the input layer of the image generation model, the first intermediate feature of the feature extraction model and the second intermediate feature of the control network are respectively input into the intermediate layer of the image generation model, and the target image is output through the output layer of the image generation model, wherein the second intermediate feature is a feature output by the intermediate layer of the image generation model.

7. The image processing method according to claim 1, characterized in that: The method further comprises: Extracting contour information of the target object in the image to be processed based on the mask image; Perform edge detection on the image to be processed based on the contour information to obtain the edge information.

8. The image processing method according to claim 1, characterized in that: The method further comprises: In response to a drawing instruction applied to the image to be processed, edge information of the target object is generated according to the drawing instruction.

9. The image processing method according to claim 1, characterized in that: The method further comprises: Acquire a sample image to be processed and a sample target image corresponding to the sample image to be processed, wherein the sample image to be processed includes a sample object; Using a feature extraction model to extract features from the sample image to be processed, to obtain sample item features of the sample item; Based on the sample object feature and the sample edge information of the sample object feature, guiding the initial image generation model to transform the sample image to be processed to obtain a sample transformed image; constructing a loss function based on the sample conversion image and the sample target image; The loss function is used to update the model parameters of the initial image generation model to obtain the image generation model.

10. A model training method, characterized in that: include: Acquire a sample image to be processed and a sample target image corresponding to the sample image to be processed, wherein the sample image to be processed includes a sample object; Using a feature extraction model to extract features from the sample image to be processed, to obtain sample item features of the sample item; Based on the sample object feature and the sample edge information of the sample object feature, guiding the initial image generation model to transform the sample image to be processed to obtain a sample transformed image; constructing a loss function based on the sample conversion image and the sample target image; The loss function is used to update the model parameters of the initial image generation model to obtain the image generation model.

11. An image processing method, characterized in that: include: Acquire a clothing image, wherein the clothing image includes a target clothing; Extracting features of the clothing image using a feature extraction model to obtain clothing features of the target clothing; Based on the clothing features and the edge information of the target clothing, the image generation model is guided to perform transformation processing on the clothing image to obtain a target clothing image, wherein other areas of the target clothing image except the area where the target clothing is located are displayed by a preset texture, and the target clothing in the target clothing image is in a preset state.

12. An image processing method, characterized in that: include: In response to an input instruction acting on an interactive interface, displaying an image to be processed on the interactive interface, wherein the image to be processed includes a target object; In response to a processing instruction acting on the interactive interface, a target image corresponding to the image to be processed is displayed on the interactive interface, wherein other areas in the target image except the area where the target object is located are displayed by a preset texture, and the target object in the target image is in a preset state, and the target image is obtained by guiding an image generation model to transform the image to be processed according to the object features of the target object and the edge information of the target object, and the object features are obtained by extracting features from the image to be processed using a feature extraction model.

13. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 12 when running.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 12.

15. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images