Image-based refitting method and device, computer device and storage medium

Abstract

The application relates to an image-based dressing method and device, computer equipment and a storage medium. The method comprises the following steps: performing region segmentation processing on a to-be-dressed image to obtain a clothing region and a clothing adjacent region; the clothing adjacent region refers to a region adjacent to the clothing region in each region obtained through the region segmentation processing; deleting image content in the clothing region from the to-be-dressed image to obtain a no-clothing image; determining a to-be-repaired region in the no-clothing image; the to-be-repaired region is a region in a clothing template image that cannot be covered by new clothing in the clothing region of the no-clothing image; repairing the to-be-repaired region according to the clothing adjacent region in the no-clothing image to obtain a repaired image; and performing layer merging on the repaired image and the clothing template image to obtain a dressing result image. The method can improve the dressing quality.

Description

Technical Field

[0001] This application relates to the fields of computer technology and image processing technology, and in particular to an image-based dressing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of image processing technology, various image processing methods have emerged, providing many conveniences for people's lives and work. Image clothing replacement technology is one such important application. For example, it can replace the clothes of people in electronic ID photos, allowing them to wear new clothes to meet the requirement that ID photos require people to wear formal attire.

[0003] In traditional methods, the template image of the new garment is usually directly pasted onto the image to be dressed. This can easily result in unrealistic and unnatural dressing results, leading to poor dressing effects and affecting the image quality after the dressing is done. Summary of the Invention

[0004] Therefore, it is necessary to provide an image-based dressing method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the quality of dressing in order to address the aforementioned technical problems.

[0005] Firstly, this application provides an image-based dress-up method. The method includes:

[0006] The image of the clothing change is processed by region segmentation to obtain the clothing region and the clothing adjacent region; the clothing adjacent region refers to the region that is adjacent to the clothing region in each region obtained by the region segmentation process.

[0007] Delete the image content in the clothing area from the image to be dressed, and obtain an image without clothing;

[0008] Identify the area to be repaired in the image without clothing; the area to be repaired is the area in the clothing template image that cannot be covered by the clothing area in the image without clothing.

[0009] Based on the adjacent areas of clothing in the image without clothing, the area to be repaired is repaired to obtain a repaired image;

[0010] The repaired image and the clothing template image are merged into layers to obtain the clothing change result image.

[0011] Secondly, this application also provides an image-based dressing device. The device includes:

[0012] The region segmentation module is used to perform region segmentation processing on the image to be dressed up, to obtain the clothing region and the clothing adjacent region; the clothing adjacent region refers to the region that is adjacent to the clothing region in each region obtained by the region segmentation processing; the image content in the clothing region is deleted from the image to be dressed up, to obtain an image without clothing;

[0013] The region repair module is used to determine the region to be repaired in the image without clothing; the region to be repaired is the area that the new clothing in the clothing template image cannot cover in the clothing area of ​​the image without clothing; the region to be repaired is repaired according to the adjacent areas of the clothing in the image without clothing to obtain a repaired image;

[0014] The layer merging module is used to merge the repaired image and the clothing template image to obtain the dressing result image.

[0015] In one embodiment, the adjacent area of ​​the clothing includes a neck area; the area repair module is further configured to perform interpolation processing based on the neck area in the clothing-free image to cover the first area to be repaired in the clothing-free image, thereby obtaining a repaired image; the first area to be repaired is the area within the neckline of the new clothing in the clothing template image that corresponds to the area in the area to be repaired.

[0016] In one embodiment, the adjacent area of ​​the clothing includes a hair area; the area repair module is further configured to perform interpolation processing based on the hair area in the clothing-free image to cover a second area to be repaired in the clothing-free image, thereby obtaining a repaired image; the second area to be repaired is the area in the area to be repaired that corresponds to the area outside the neckline of the new clothing in the clothing template image.

[0017] In one embodiment, the device further includes:

[0018] A color tone adjustment module is used to adjust the color attribute information of the clothing template image based on a reference image to obtain a harmonious clothing template image; the color attributes of the harmonious clothing template image and the reference image are harmonious; the reference image includes any one of the image to be changed into, the image without clothing, or the image to be repaired;

[0019] The layer merging module is also used to merge the repaired image and the harmonized clothing template image to obtain the dressing result image.

[0020] In one embodiment, the tone adjustment module is further configured to adjust the color attribute information of the clothing template image by means of a color attribute adjustment model, based on a reference image and pre-learned color attribute conversion rules, to obtain a harmonious clothing template image.

[0021] In one embodiment, the layer merging module is further configured to dynamically adjust the relative position between the repair image and the clothing template image based on the positioning points in the clothing template image and the positioning points in the repair image, until the exposed neck area in the repair image meets the preset alignment conditions, so that the repair image and the clothing template image are aligned; and then the aligned repair image and clothing template image are merged to obtain the dressing result image.

[0022] In one embodiment, the layer merging module is further configured to determine the positioning points in the repaired image based on at least one of eye-related positioning points, face-width-related positioning points, and neck-related positioning points;

[0023] The eye-related positioning point is a positioning point with a horizontal coordinate determined based on the eye position in the restored image; the face width-related positioning point is a positioning point with a horizontal coordinate determined based on the face width in the restored image; and the neck-related positioning point is a positioning point determined based on the intersection of the clothing area and the neck area.

[0024] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the image-based dress-up method described in the embodiments of this application.

[0025] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, causes the processor to perform the steps of the image-based dress-up method described in the embodiments of this application.

[0026] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, causes the processor to perform the steps of the image-based dress-up method described in the embodiments of this application.

[0027] The aforementioned image-based dress-up method, apparatus, computer equipment, storage medium, and computer program product perform region segmentation processing on the image to be dressed up, obtaining the clothing region and adjacent clothing regions. Image content in the clothing region is deleted from the image to be dressed up, resulting in a clothing-free image. Then, the region to be repaired in the clothing-free image is determined. Based on the adjacent clothing regions in the clothing-free image, the region to be repaired is repaired, thereby repairing areas in the clothing template image that cannot be covered by the clothing region in the clothing-free image. Finally, the repaired image and the clothing template image are merged to obtain the dress-up result image, making the dress-up result image more realistic and natural, with a better dress-up effect, and improving the image quality after the dress-up. Attached Figure Description

[0028] Figure 1 This is an application environment diagram of an image-based dress-up method in one embodiment;

[0029] Figure 2 This is a flowchart illustrating an image-based costume change method in one embodiment.

[0030] Figure 3 This is a schematic diagram of the image to be changed into and the clothing template image in one embodiment;

[0031] Figure 4 This is a schematic diagram of the result of region segmentation processing in one embodiment;

[0032] Figure 5 This is a schematic diagram of the area to be repaired before and after repair in one embodiment;

[0033] Figure 6 This is a schematic diagram showing the repaired first and second regions to be repaired in one embodiment.

[0034] Figure 7 This is a schematic diagram illustrating the effect of harmonizing a clothing template image in one embodiment.

[0035] Figure 8 This is a schematic diagram of the positioning points in one embodiment;

[0036] Figure 9 This is a schematic diagram showing the before and after replacement in one embodiment;

[0037] Figure 10 This is a schematic diagram of the overall process of an image-based dress-up method in one embodiment;

[0038] Figure 11 This is a structural block diagram of an image-based dressing device in one embodiment;

[0039] Figure 12 This is a structural block diagram of an image-based dressing device in another embodiment;

[0040] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] The image-based costume-changing method provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Users can upload an image of the garment to be changed into via terminal 102 and select the desired new clothing. Server 104 can obtain the image of the garment to be changed into from terminal 102 and obtain the clothing template image corresponding to the new clothing. It performs region segmentation processing on the image of the garment to be changed into, obtaining the clothing area and adjacent clothing areas. It deletes the image content from the clothing area of ​​the image of the garment to be changed into, obtaining a clothing-free image. Then, based on the areas in the clothing template image that the new clothing cannot cover in the clothing area of ​​the clothing-free image, it determines the areas to be repaired in the clothing-free image. Then, based on the adjacent clothing areas in the clothing-free image, it repairs the areas to be repaired, obtaining a repaired image. Finally, it merges the repaired image and the clothing template image into layers to obtain the changed-up result image. Server 104 can send the obtained changed-up result image to terminal 102 for display. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0043] In other embodiments, the image-based dress-up methods in the various embodiments of this application can also be executed directly through the terminal without uploading to the server for execution.

[0044] In one embodiment, such as Figure 2 As shown, an image-based dress-up method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the following steps are included:

[0045] Step 202: Perform region segmentation processing on the image to be dressed to obtain the clothing region and the clothing adjacent region; the clothing adjacent region refers to the region that is adjacent to the clothing region in each region obtained by the region segmentation processing.

[0046] Among them, the image to be dressed up is the image in which the object needs to be dressed up. For example... Figure 3 The image shows a person in a dress that needs to be dressed up. It is understood that, to protect portrait rights, the faces in the images shown in the accompanying drawings are obscured; however, in practical applications, the images do include the faces.

[0047] Region segmentation is the process of dividing an image into multiple different regions based on the different semantic meanings of its content. The clothing region refers to the region corresponding to the clothing in the image to be changed into.

[0048] In one embodiment, a user can upload an image of clothing to be changed into via a terminal, which can then send the image to a server. The server can perform region segmentation processing on the image to obtain the clothing area and adjacent areas.

[0049] In one embodiment, the server can use a semantic segmentation method to perform region segmentation processing on the image to be dressed up, obtaining multiple regions with different semantics, from which the clothing region and the clothing adjacent region are obtained.

[0050] In one embodiment, the regions obtained by performing region segmentation processing on the image to be dressed up may include, in addition to the clothing region, at least one of the following: neck region, hair region, face region, accessory region, and background region.

[0051] The image includes the following regions: Neck region (corresponding to the neck of the object in the image to be dressed up), Hair region (corresponding to the hair of the object in the image to be dressed up), Face region (corresponding to the face of the object in the image to be dressed up), Accessory region (corresponding to the accessories worn by the object in the image to be dressed up), and Background region (corresponding to the background of the image excluding the object).

[0052] In one embodiment, the adjacent area of ​​the clothing may include at least one of the neck area and the hair area.

[0053] like Figure 4 The diagram shows the result of the region segmentation process, with each region represented by a different color. As can be seen, 402 is the clothing region, 404 is the hair region, and 406 is the neck region.

[0054] In one embodiment, the object in the image to be dressed up can be a person. In other embodiments, the object in the image to be dressed up can also be an animal wearing clothing, such as a cat or a dog, and there is no limitation.

[0055] In one embodiment, the image to be changed into can be a passport photo. In other embodiments, the image to be changed into can also be other types of photos, such as professional photos, artistic photos, or casual photos, etc., without limitation.

[0056] Step 204: Delete the image content in the clothing area from the image to be dressed, and obtain an image without clothing.

[0057] Among them, the image without clothing refers to the image obtained by deleting the image content in the clothing area of ​​the image to be changed.

[0058] Step 206: Determine the area to be repaired in the image without clothing; the area to be repaired is the area in the clothing template image that cannot be covered by the new clothing in the clothing area of ​​the image without clothing.

[0059] The clothing template image is the template image corresponding to the new clothing to be replaced in the image to be changed. The uncovered area refers to the area where the clothing area exceeds the new clothing in the clothing template image.

[0060] In one embodiment, the clothing template image can be an image corresponding to a new garment without a background. For example... Figure 3 The image shown is a clothing template. It can be seen that the clothing template image only contains new clothing, and does not contain anything else.

[0061] In one embodiment, a user can select a new garment they wish to wear via a terminal, and the terminal can notify the server. The server can then determine a corresponding garment template image based on the selected new garment. Alternatively, the terminal can determine a corresponding garment template image based on the selected new garment and send the garment template image to the server.

[0062] In one embodiment, the server can compare the new garment in the clothing template image with the clothing area to determine the area to be repaired in the image without clothing. Specifically, the server can use the difference area between the new garment in the clothing template image and the clothing area as the area to be repaired in the image without clothing.

[0063] like Figure 5 As shown in the figure, the area 502 enclosed by the black line and the areas 504 and 506 enclosed by the white lines are all areas to be repaired.

[0064] It is understandable that the image without clothing is obtained by deleting the image content in the clothing area. What is deleted is the image content, while the clothing area represents a region. Therefore, the image without clothing still contains the clothing area. Thus, the area to be repaired is the area that the new clothing in the clothing template image cannot cover in the clothing area of ​​the image without clothing. The server can compare the clothing template image with the clothing area in the image without clothing to determine the area to be repaired in the image without clothing.

[0065] In one embodiment, the server can subtract the clothing area from the new clothing in the clothing template image. If the difference is less than or equal to zero, it indicates that the new clothing can completely cover the clothing area and there is no area to be repaired. The server can directly merge the layers of the image without clothing and the clothing template image to obtain the dressing result image. If the difference is greater than zero, it indicates that the new clothing cannot completely cover the clothing area and there is an area to be repaired in the image without clothing. The server can then perform step 206 and subsequent steps.

[0066] In one embodiment, the new garment in the clothing template image can be formal wear. In other embodiments, the new garment in the clothing template image can also be other types of clothing, such as school uniforms, uniforms, or work clothes, etc., without limitation.

[0067] In one embodiment, the image-based clothing change method in various embodiments of this application can be applied to a scenario where a person in an ID photo (image to be changed) is dressed in formal attire (new clothing). In other embodiments, it can also be applied to other clothing change scenarios, without limitation.

[0068] Step 208: Based on the adjacent areas of clothing in the image without clothing, repair the area to be repaired to obtain the repaired image.

[0069] In one embodiment, the server can perform interpolation based on adjacent regions of clothing in an image without clothing to cover the area to be repaired in the image without clothing, thus obtaining a repaired image. For example... Figure 5 The images before and after restoration are shown respectively. It can be seen that, compared with the images before restoration, the image content in the areas to be restored, 502, 504 and 506, has been covered.

[0070] In one embodiment, there can be multiple areas to be repaired, and each area can be repaired based on different adjacent areas of the garment. In another embodiment, the server can determine which adjacent garment area to repair based on the positional relationship between the area to be repaired and the new garment in the garment template image.

[0071] In one embodiment, the server can determine which adjacent garment region to repair based on the positional relationship between the region to be repaired and the neckline of the new garment in the garment template image.

[0072] In one embodiment, if the area to be repaired corresponds to the area inside the neckline of a new garment in a garment template image, then the area to be repaired is repaired based on the neck area.

[0073] In one embodiment, if the area to be repaired corresponds to the area outside the neckline of a new garment in a garment template image, then the area to be repaired is repaired based on the hair area.

[0074] Step 210: Merge the repaired image and the clothing template image into layers to obtain the clothing change result image.

[0075] The "Clothing Change Result Image" is an image where the clothing in the image to be changed is replaced with the new clothing in the image template image. "Layer Merging" combines two images into a single image.

[0076] In one embodiment, the server can align the repair image and the clothing template image based on the positioning points of the repair image and the clothing template image, and then merge the aligned repair image and clothing template image into layers to obtain the dressing result image.

[0077] In one embodiment, the server can directly obtain the positioning points pre-annotated for the clothing template image. In another embodiment, the server can determine the positioning points in the clothing template image by traversing each pixel in the clothing template image.

[0078] In one embodiment, the server can determine the positioning points in the repaired image based on at least one of eye-related positioning points, face-width-related positioning points, and neck-related positioning points.

[0079] In one embodiment, the server can send the resulting transformation image to the terminal, which can then display the image. Users can download the transformation image via the terminal.

[0080] The image-based dress-up method described above performs region segmentation on the image to be dressed up, obtaining the clothing region and adjacent clothing regions. Image content within the clothing region is then removed from the image to be dressed up, resulting in a clothing-free image. Next, the areas to be repaired in the clothing-free image are identified and repaired based on the adjacent clothing regions. This ensures that areas in the clothing template image that cannot be covered by the clothing region in the clothing-free image are repaired. Finally, the repaired image and the clothing template image are merged to obtain the final dress-up image, resulting in a more realistic and natural dress-up effect and improved image quality. Furthermore, the dress-up is performed automatically, eliminating the need for professional retouchers to spend significant time and effort on image editing and dress-up, thus improving efficiency and enabling the rapid and accurate generation of realistic and natural dress-up images.

[0081] In one embodiment, the adjacent area of ​​the clothing includes a neck area. Repairing the area to be repaired based on the adjacent area of ​​the clothing in the image without clothing to obtain a repaired image includes: performing interpolation processing on the neck area in the image without clothing to cover a first area to be repaired in the image without clothing; the first area to be repaired is the area within the neckline of the new clothing in the clothing template image within the area to be repaired.

[0082] In one embodiment, the server can compare the new garment in the garment template image with the area to be repaired to determine the positional relationship between the area to be repaired and the new garment in the garment template image, and determine the first area to be repaired within the neckline of the new garment in the garment template image based on the positional relationship.

[0083] In one embodiment, the server may use nearest neighbor interpolation to generate neck image content with a larger coverage area by spreading it outwards from the neck region in the unclothed image, thereby covering the first area to be repaired in the unclothed image and obtaining the repaired image.

[0084] like Figure 6 602 in the image represents the first area to be repaired. It can be seen that after repairing the first area to be repaired based on the neck region, the first area to be repaired is covered by the neck image content, thus achieving repair.

[0085] In the above embodiments, by repairing the first area to be repaired within the neckline of the new garment in the clothing template image based on the neck area, it is possible to avoid the problem that part of the neck is obscured due to the original garment's neckline being too high or hair obscuring it in the image to be changed, resulting in a blank area within the neckline of the new garment in the final changed image. This makes the changed image more realistic and natural, improves the changing effect, and enhances the quality of the changed image.

[0086] In one embodiment, the adjacent area of ​​the clothing includes a hair area. Repairing the area to be repaired based on the adjacent area of ​​the clothing in the image without clothing, to obtain a repaired image, includes: performing interpolation processing based on the hair area in the image without clothing to cover a second area to be repaired in the image without clothing; the second area to be repaired is the area outside the neckline of the new clothing in the clothing template image within the area to be repaired.

[0087] In one embodiment, the server can compare the new garment in the garment template image with the area to be repaired to determine the positional relationship between the area to be repaired and the new garment in the garment template image, and determine a second area to be repaired in the area to be repaired, corresponding to the neckline of the new garment in the garment template image, based on the positional relationship.

[0088] In one embodiment, the server can perform local liquefaction processing on the second area to be repaired based on the image content in the hair region of the unclothed image, so as to deform the image content in the hair region into a hair image content with a larger coverage area, so as to cover the second area to be repaired in the unclothed image, thereby obtaining a repaired image.

[0089] like Figure 6 In the image, 604 and 606 represent the second area to be repaired. It can be seen that after repairing the second area based on the hair region, the second area to be repaired is covered by the hair image content, thus achieving repair.

[0090] In the above embodiments, by repairing the second area to be repaired in the area to be repaired, which corresponds to the neckline of the new garment in the clothing template image, based on the hair area, it is possible to avoid the problem that some hair is obscured due to the original garment's neckline being too high or accessories obscuring it in the image to be changed, resulting in a blank area outside the neckline of the new garment in the final image of the change of clothes. This makes the image of the change of clothes more realistic and natural, with a better effect, and improves the quality of the image of the change of clothes.

[0091] In one embodiment, merging the repaired image and the clothing template image to obtain the dressing result image includes: adjusting the color attribute information of the clothing template image based on the reference image to obtain a harmonized clothing template image; the color attributes of the harmonized clothing template image and the reference image are harmonious; the reference image includes any one of the image to be dressed, an image without clothing, or a repaired image; merging the repaired image and the harmonized clothing template image to obtain the dressing result image.

[0092] Among them, color attribute refers to the attribute related to the color of the image. Color attribute information is the value of the color attribute.

[0093] In one embodiment, color attributes may include at least one of hue, saturation, or contrast.

[0094] This is understandable, because the image without clothing is obtained by deleting the clothing area from the image to be dressed, so the color attributes of the image without clothing and the image to be dressed are the same. The repaired image is obtained based on the image without clothing, so the color attributes of the repaired image are also the same as the image without clothing. Therefore, any one of the images to be dressed, the image without clothing, or the repaired image can be used as a reference image.

[0095] In one embodiment, the server can adjust the color attribute information of the clothing template image based on the color attribute information of the reference image to obtain a harmonious clothing template image.

[0096] In the above embodiments, the color attribute information of the clothing template image is adjusted based on the reference image to obtain a harmonized clothing template image. The repaired image and the harmonized clothing template image are then merged into layers to obtain the dressing result image. This process ensures that the color attributes of the harmonized clothing template image and the repaired image are in harmony, resulting in a more realistic and natural dressing result image with a better dressing effect and improved quality.

[0097] In one embodiment, adjusting the color attribute information of a clothing template image based on a reference image to obtain a harmonious clothing template image includes: adjusting the color attribute information of the clothing template image according to the reference image and pre-learned color attribute conversion rules through a color attribute adjustment model to obtain a harmonious clothing template image.

[0098] In one embodiment, the server can pre-input training samples into the color attribute adjustment model to be trained, and the color attribute adjustment model learns color attribute conversion rules based on the training samples to obtain the trained color attribute adjustment model.

[0099] The training samples include a sample image of clothing to be changed into, a sample clothing template image, and a sample harmonious clothing template image. The color attributes of the sample image of clothing to be changed into and the sample harmonious clothing template image are harmonious. The color attribute conversion rule is the rule for adjusting the color attributes of the sample clothing template image to match the color attributes of the sample harmonious clothing template image based on the sample image of clothing to be changed into.

[0100] In one embodiment, the color attribute conversion rule can be similar to a three-dimensional lookup table.

[0101] A 3D Lookup Table (3DLUT) is a list of corresponding values ​​that allows you to look up an input value and its corresponding output value. 3D lookup tables are a technique used in color conversion. In a 3D lookup table, one input color attribute corresponds to one output color attribute.

[0102] In one embodiment, the server can adjust the color attribute model by determining the target color attribute information corresponding to each pixel of the clothing template image based on the reference image and pre-learned color attribute conversion rules, and adjusting each pixel of the clothing template image to the target color attribute information to obtain a harmonized clothing template image.

[0103] In the above embodiments, the color attribute adjustment model adjusts the color attribute information of the clothing template image according to the reference image and pre-learned color attribute conversion rules to obtain a harmonized clothing template image. This ensures that the color attributes of the harmonized clothing template image are in harmony with the repaired image, resulting in a more realistic and natural dressing effect and improved image quality. Furthermore, by learning color attribute conversion rules through the color attribute adjustment model, the color attribute information of the clothing template image can be adjusted quickly and accurately without the need for pre-setting complex color attribute conversion rules, thus improving both the dressing effect and efficiency.

[0104] like Figure 7 As shown, the effects before and after harmonization in the above embodiments are illustrated. It can be seen that the harmonized clothing template image is more harmonious with the color attributes of the image to be dressed up compared to the unharmonized clothing template image. The clothing in the dressing result image generated based on the harmonized clothing template image is more harmonious with other image content in the image, unlike the clothing in the dressing result image generated based on the unharmonized clothing template image, which appears abrupt and unnatural. Therefore, the dressing effect is improved, and the dressing result is more realistic and natural.

[0105] In one embodiment, merging the repaired image and the clothing template image to obtain the dressing result image includes: dynamically adjusting the relative position between the repaired image and the clothing template image based on the positioning points in the clothing template image and the positioning points in the repaired image until the exposed neck area in the repaired image meets the preset alignment conditions, so that the repaired image and the clothing template image are aligned; merging the aligned repaired image and the clothing template image to obtain the dressing result image.

[0106] In one embodiment, the positioning points in the clothing template image are a pair of points (i.e., two points). The positioning points in the repaired image are also a pair of points (i.e., two points).

[0107] In one embodiment, the positioning points in the garment template image can be the highest points of the collars on both sides of the new garment, such as... Figure 8 The pair of points shown in 806.

[0108] In one embodiment, the positioning points in the clothing template image can be as follows: Figure 8 The pair of points shown in 804.

[0109] In one embodiment, the server can align the positioning points in the clothing template image with the positioning points in the repair image, and make fine adjustments within a preset range at the corresponding positions until the exposed neck area in the repair image meets the preset alignment conditions, so that the repair image and the clothing template image are aligned.

[0110] In one embodiment, there may be multiple sets of positioning points in the repair image. The server can sequentially align each set of positioning points in the repair image with the positioning points in the clothing template image to adjust the repair image and the clothing template image to different relative positions until the exposed neck area in the repair image meets the preset alignment conditions, so that the repair image and the clothing template image are aligned.

[0111] In another embodiment, there can be multiple sets of positioning points in the repaired image. The server can sequentially align each set of positioning points in the repaired image with the positioning points in the clothing template image to adjust the repaired image and the clothing template image to different relative positions, and make fine adjustments within a preset range at the corresponding positions until the exposed neck area in the repaired image meets the preset alignment conditions, so that the repaired image and the clothing template image are aligned.

[0112] In one embodiment, the preset alignment condition may be a preset length range set for the length of the exposed neck area in the repaired image. In one embodiment, if the length of the exposed neck area in the repaired image is within the preset length range, then the exposed neck area in the repaired image meets the preset alignment condition.

[0113] In another embodiment, the preset alignment condition may be a preset area range set for the area of ​​the exposed neck region in the repaired image. In one embodiment, if the area of ​​the exposed neck region in the repaired image is within the preset area range, then the exposed neck region in the repaired image meets the preset alignment condition.

[0114] In the above embodiments, the relative positions between the repaired image and the clothing template image are dynamically adjusted according to the positioning points in the clothing template image and the positioning points in the repaired image until the exposed neck area in the repaired image meets the preset alignment conditions, so that the repaired image and the clothing template image are aligned. The aligned repaired image and the clothing template image are then merged into layers to obtain the dressing result image, making the exposed neck area in the dressing result image more realistic, making the dressing result image more realistic and natural, and improving the dressing effect, thus improving the quality of the dressing result image.

[0115] In one embodiment, the method further includes: determining positioning points in the repaired image based on at least one of eye-related positioning points, face-width-related positioning points, and neck-related positioning points. Wherein, the eye-related positioning point is a positioning point whose horizontal coordinate is determined based on the eye position in the repaired image; the face-width-related positioning point is a positioning point whose horizontal coordinate is determined based on the face width in the repaired image; and the neck-related positioning point is a positioning point determined based on the intersection of the clothing area and the neck area.

[0116] In one embodiment, the server can determine the coordinates of the intersection of the clothing area and the neck area as the location coordinates of the neck-related positioning point. For example... Figure 8 Point 802 in the image represents the neck-related localization points. Point 804 represents the localization point in the final reconstructed image.

[0117] In one embodiment, the server can determine the x-coordinate of the eye's position coordinates in the repaired image as the x-coordinate of the eye's associated location point. In another embodiment, the server can determine the eye's position coordinates in the repaired image using a face detection algorithm.

[0118] In one embodiment, the server can move within a preset range according to the horizontal coordinate of the eye-associated positioning point and change the vertical coordinate of the eye-associated positioning point to dynamically adjust the relative position between the repaired image and the clothing template image.

[0119] In one embodiment, the server can determine the x-coordinate of the face width-related positioning point based on the x-coordinate of the position coordinates of the facial edge points in the repaired image that conform to a preset face width. For example... Figure 8 The points at both ends of the horizontal line in the mid-face area are the facial edge points that conform to the preset face width.

[0120] In one embodiment, the server can move within a preset range according to the horizontal coordinate of the face width-associated positioning point and change the vertical coordinate of the face width-associated positioning point to dynamically adjust the relative position between the repaired image and the clothing template image.

[0121] In the above embodiments, the positioning points in the repaired image are determined based on at least one of the eye-related positioning points, face width-related positioning points, and neck-related positioning points, thereby enabling accurate alignment of the repaired image and the clothing template image and improving the quality of the clothing change result image.

[0122] like Figure 9 The image shown is a comparison between the image of the clothing change result obtained by the image-based clothing change method in various embodiments of this application and the image of the clothing to be changed before the change. It can be seen that the clothing change result image accurately replaces the clothing in the image of the clothing to be changed with new clothing, and the clothing change effect is realistic, natural and of good quality.

[0123] like Figure 10 The diagram shown is a schematic flowchart of the image-based dress-up method in various embodiments of this application. First, the image to be dressed and the clothing template image are input. Then, the image to be dressed is segmented to obtain the clothing area and adjacent clothing areas. Next, the area to be repaired is repaired based on the adjacent clothing areas to obtain a repaired image. Then, the color attribute information of the clothing template image is adjusted based on a reference image (i.e., adaptive hue adjustment) to obtain a harmonized clothing template image. Finally, the relative positions of the repaired image and the harmonized template image are dynamically adjusted for alignment. Then, layers are merged to obtain the dress-up result image and output.

[0124] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0125] Based on the same inventive concept, this application also provides an image-based dressing device for implementing the image-based dressing method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more image-based dressing device embodiments provided below can be found in the limitations of the image-based dressing method described above, and will not be repeated here.

[0126] In one embodiment, such as Figure 11As shown, an image-based dressing device 1100 is provided, including: a region segmentation module 1102, a region repair module 1104, and a layer merging module 1106, wherein:

[0127] The region segmentation module 1102 is used to perform region segmentation processing on the image to be changed into clothing, and obtain clothing region and clothing adjacent region; the clothing adjacent region refers to the region that is adjacent to the clothing region in each region obtained by the region segmentation processing; the image content in the clothing region is deleted from the image to be changed into clothing, and an image without clothing is obtained.

[0128] The region repair module 1104 is used to determine the region to be repaired in the image without clothing; the region to be repaired is the area that the new clothing in the clothing template image cannot cover in the clothing area of ​​the image without clothing; based on the adjacent areas of the clothing in the image without clothing, the region to be repaired is repaired to obtain the repaired image.

[0129] The layer merging module 1106 is used to merge the repaired image and the clothing template image to obtain the dressing result image.

[0130] In one embodiment, the adjacent area of ​​the garment includes a neck area. The area repair module 1104 is further configured to perform interpolation processing based on the neck area in the image without clothing to cover the first area to be repaired in the image without clothing, thereby obtaining a repaired image; the first area to be repaired is the area within the neckline of the new garment in the clothing template image that corresponds to the area to be repaired.

[0131] In one embodiment, the adjacent area of ​​the clothing includes a hair area. The area repair module 1104 is further configured to perform interpolation processing based on the hair area in the image without clothing to cover the second area to be repaired in the image without clothing, thereby obtaining a repaired image; the second area to be repaired is the area outside the neckline of the new clothing in the clothing template image within the area to be repaired.

[0132] In one embodiment, such as Figure 12 As shown, the device 1100 also includes:

[0133] The color tone adjustment module 1108 is used to adjust the color attribute information of the clothing template image based on the reference image to obtain a harmonious clothing template image; the color attributes of the harmonious clothing template image and the reference image are harmonious; the reference image includes any one of the image to be changed, the image without clothing, or the image to be repaired.

[0134] The layer merging module 1106 is also used to merge the repaired image and the harmonious clothing template image to obtain the dressing result image.

[0135] In one embodiment, the color adjustment module 1108 is further configured to adjust the color attribute information of the clothing template image by means of a color attribute adjustment model, based on a reference image and pre-learned color attribute conversion rules, to obtain a harmonious clothing template image.

[0136] In one embodiment, the layer merging module 1106 is further configured to dynamically adjust the relative position between the repair image and the clothing template image based on the positioning points in the clothing template image and the positioning points in the repair image, until the exposed neck area in the repair image meets the preset alignment conditions, so that the repair image and the clothing template image are aligned; and merge the aligned repair image and the clothing template image to obtain the dressing result image.

[0137] In one embodiment, the layer merging module 1106 is further configured to determine the positioning points in the repaired image based on at least one of eye-related positioning points, face-width-related positioning points, and neck-related positioning points.

[0138] Among them, the eye-related positioning point is a positioning point whose horizontal coordinate is determined based on the position of the eyes in the restored image; the face width-related positioning point is a positioning point whose horizontal coordinate is determined based on the width of the face in the restored image; and the neck-related positioning point is a positioning point determined based on the intersection between the clothing area and the neck area.

[0139] The aforementioned image-based dress-up device performs region segmentation processing on the image to be dressed up, obtaining the clothing region and adjacent clothing regions. It then deletes the image content in the clothing region from the image to be dressed up, obtaining a clothing-free image. Next, it identifies the areas to be repaired in the clothing-free image and repairs them based on the adjacent clothing regions in the clothing-free image. This allows areas in the clothing template image that cannot be covered by the clothing region in the clothing-free image to be repaired. Finally, it merges the repaired image and the clothing template image into layers to obtain the dress-up result image, making the dress-up result image more realistic and natural, with a better dress-up effect and improved image quality after the dress-up.

[0140] The modules in the aforementioned image-based dressing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0141] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an image-based dress-up method.

[0142] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0143] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0144] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0145] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0146] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0147] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0148] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image-based rebranding method, characterized by, The method comprises: performing region segmentation processing on the image to be refitted to obtain a clothing region and a clothing adjacent region; the clothing adjacent region refers to a region adjacent to the clothing region in each region obtained by the region segmentation processing; the clothing adjacent region comprises a neck region and a hair region; deleting image content in the clothing region from the image to be refitted to obtain a clothing-free image; the clothing-free image refers to an image obtained by deleting image content in the clothing region from the image to be refitted, and the clothing-free image has the clothing region; determining a region to be repaired in the clothing-free image, comprising: subtracting the clothing region from a new clothing in a clothing template image, if a difference value is less than or equal to zero, it indicates that the new clothing completely covers the clothing region, and there is no region to be repaired; if the difference value is greater than zero, it indicates that the new clothing cannot cover the clothing region, and there is a region to be repaired in the clothing-free image; the region to be repaired is a difference region between the new clothing in the clothing template image and the clothing region of the clothing-free image; performing interpolation processing on the neck region and the hair region in the clothing-free image to cover the region to be repaired in the clothing-free image to obtain a repaired image, comprising: if the region to be repaired corresponds to inside a collar of the new clothing in the clothing template image, repairing the region to be repaired according to the neck region; if the region to be repaired corresponds to outside the collar of the new clothing in the clothing template image, repairing the region to be repaired according to the hair region; performing layer merging on the repaired image and a harmonized clothing template image to obtain a refitting result image, comprising: determining a positioning point in the repaired image according to at least one of an eye-related positioning point, a face width-related positioning point and a neck-related positioning point, aligning the repaired image and the harmonized clothing template image according to the positioning point in the repaired image and a positioning point in the harmonized clothing template image, and performing layer merging on the aligned repaired image and the harmonized clothing template image to obtain the refitting result image; wherein the harmonized clothing template image is obtained by adjusting color attribute information of the clothing template image according to color attribute information of a reference image, and the reference image comprises any one of the image to be refitted, the clothing-free image or the repaired image; or, performing layer merging on the clothing-free image and the harmonized clothing template image to obtain the refitting result image.

2. The method of claim 1, wherein, The clothing adjacent region comprises a neck region; and performing repairing on the region to be repaired according to the clothing adjacent region in the clothing-free image to obtain a repaired image, comprising: performing interpolation processing on the neck region in the clothing-free image to cover a first region to be repaired in the clothing-free image to obtain a repaired image; the first region to be repaired is a region in the region to be repaired corresponding to inside a collar of the new clothing in the clothing template image.

3. The method of claim 1, wherein, The clothing adjacent region includes a hair region; the repairing the to-be-repaired region according to the clothing adjacent region in the clothing-free image to obtain a repaired image includes: performing interpolation processing according to the hair region in the clothing-free image to cover a second to-be-repaired region in the clothing-free image to obtain a repaired image; the second to-be-repaired region is a region in the to-be-repaired region corresponding to a collar of the new clothing in the clothing template image.

4. The method of claim 1, wherein, The layer merging of the repaired image and the clothing template image to obtain a clothing changing result image includes: adjusting color attribute information of the clothing template image based on a reference image to obtain a harmonized clothing template image; the color attributes of the harmonized clothing template image and the reference image are harmonious; performing layer merging of the repaired image and the harmonized clothing template image to obtain a clothing changing result image.

5. The method of claim 4, wherein, The adjusting of the color attribute information of the clothing template image based on the reference image to obtain the harmonized clothing template image includes: adjusting, by a color attribute adjustment model, the color attribute information of the clothing template image according to a reference image and a pre-learned color attribute conversion rule to obtain a harmonized clothing template image.

6. The method according to any one of claims 1 to 5, characterized in that, The layer merging of the repaired image and the harmonized clothing template image to obtain a clothing changing result image includes: dynamically adjusting a relative position between the repaired image and the harmonized clothing template image according to a positioning point in the harmonized clothing template image and a positioning point in the repaired image until a neck exposure region in the repaired image meets a preset alignment condition, so as to align the repaired image and the harmonized clothing template image; performing layer merging of the aligned repaired image and the harmonized clothing template image to obtain a clothing changing result image.

7. The method of claim 6, wherein, The method further includes: The method further includes: The eye-related positioning point is a positioning point whose horizontal coordinate is determined according to an eye position in the repaired image; the face width-related positioning point is a positioning point whose horizontal coordinate is determined according to a face width in the repaired image; and the neck-related positioning point is a positioning point determined according to an intersection between the clothing region and a neck region.

8. An image-based re-casing device, characterized by The device includes: The region segmentation module is configured to perform region segmentation processing on the to-be-changed clothing image to obtain a clothing region and a clothing adjacent region; the clothing adjacent region refers to a region adjacent to the clothing region in each region obtained by the region segmentation processing; the clothing adjacent region includes a neck region and a hair region; the image content in the clothing region is deleted from the to-be-changed clothing image to obtain a clothing-free image; the clothing-free image refers to an image obtained by deleting the image content in the clothing region in the to-be-changed clothing image, and the clothing-free image has the clothing region; The region repairing module is configured to determine a region to be repaired in the garment-free image, including: subtracting the garment region from a new garment in the garment template image, and if the difference is less than or equal to zero, it indicates that the new garment completely covers the garment region, and there is no region to be repaired; if the difference is greater than zero, it indicates that the new garment cannot cover the garment region, and there is a region to be repaired in the garment-free image; the region to be repaired is a difference region between the new garment in the garment template image and the garment region in the garment-free image; and performing interpolation processing on the neck region and the hair region in the garment-free image to cover the region to be repaired in the garment-free image, to obtain a repaired image, including: if the region to be repaired corresponds to inside of a collar of the new garment in the garment template image, repairing the region to be repaired according to the neck region; and if the region to be repaired corresponds to outside of the collar of the new garment in the garment template image, repairing the region to be repaired according to the hair region. The layer merging module is configured to perform layer merging on the repaired image and a harmonized garment template image to obtain a result image of clothes changing, including: determining a positioning point in the repaired image according to at least one of an eye-associated positioning point, a face width-associated positioning point and a neck-associated positioning point, aligning the repaired image and the harmonized garment template image according to the positioning point in the repaired image and a positioning point in the harmonized garment template image, and performing layer merging on the aligned repaired image and the harmonized garment template image to obtain the result image of clothes changing; wherein the harmonized garment template image is obtained by adjusting color attribute information of the garment template image according to color attribute information of a reference image, and the reference image includes any one of the image to be changed, the garment-free image or the repaired image; or performing layer merging on the garment-free image and the harmonized garment template image to obtain the result image of clothes changing. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7.

11. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7.

Images