Image preprocessing method, virtual dressing system and computer readable storage medium

Abstract

The invention discloses a virtual dressing image preprocessing method, a virtual dressing system and a computer readable storage medium. The method comprises the following steps: obtaining a clothes image comprising clothes and a human body image comprising a human body, and generating a human body skeleton diagram corresponding to the human body; determining a skeleton image area in the human skeleton diagram based on the clothes image; and intercepting a specific image region corresponding to the skeleton image region from the human body image.

Description

Technical Field

[0001] The present invention relates to an image pre-processing method, and in particular to an image pre-processing method for virtual dressing, a virtual dressing system and a computer-readable storage medium. Background Art

[0002] The virtual dressing market is developing rapidly, and with the development of e-commerce and network technology, more and more retailers and brands are beginning to use virtual dressing technology to increase sales and customer experience. At the same time, due to the impact of the epidemic, more and more people are shopping online, allowing customers to try on virtual clothes without leaving home to enhance the shopping experience, and the demand for virtual dressing technology has also increased.

[0003] Most of the existing virtual dressing technologies are published in research papers. In order to facilitate comparison between different methods, most of these research papers use standard and high-quality image data. Since such image data has been pre-processed, researchers only focus on the development of core algorithms to optimize the final virtual dressing results.

[0004] However, it is difficult to ensure the quality of image data used in practical applications is the same as that of research papers, and the required pre-processing methods also need to be completed separately, making it difficult to directly apply the results of related technologies. Summary of the invention

[0005] In view of this, the present invention provides a virtual dressing image pre-processing method, a virtual dressing system and a computer-readable storage medium, which can be used to solve the above technical problems.

[0006] An embodiment of the present invention provides an image pre-processing method for virtual dressing, which is suitable for a virtual dressing system, comprising: obtaining a clothing image including a clothing and a human body image including a human body, and generating a human body skeleton map corresponding to the human body; determining a skeleton image area in the human body skeleton map based on the clothing image; and intercepting a specific image area corresponding to the skeleton image area from the human body image.

[0007] An embodiment of the present invention provides a virtual dressing system, comprising a storage circuit and a processor. The storage circuit stores a program code. The processor is coupled to the storage circuit and accesses the program code to execute: obtaining a clothing image including a clothing and a human body image including a human body, and generating a human body skeleton image corresponding to the human body; determining a skeleton image area in the human body skeleton image based on the clothing image; and intercepting a specific image area corresponding to the skeleton image area from the human body image.

[0008] An embodiment of the present invention provides a computer-readable storage medium, which records an executable computer program. The executable computer program is loaded by a virtual dressing system to execute the following steps: obtaining a clothing image including a clothing and a human body image including a human body, and generating a human body skeleton map corresponding to the human body; determining a skeleton image area in the human body skeleton map based on the clothing image; and capturing a specific image area corresponding to the skeleton image area from the human body image. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of a virtual dressing system according to an embodiment of the present invention;

[0010] FIG. 2A to FIG. 2C is a diagram of a virtual dressing application scenario depicted in one embodiment of the present invention;

[0011] Figure 3 is a flow chart of an image pre-processing method for virtual dressing according to an embodiment of the present invention;

[0012] FIG. 4A to FIG. 4E is a schematic diagram of image pre-processing according to the first embodiment of the present invention;

[0013] Figure 5 is a flow chart of a method for removing a person's background according to an embodiment of the present invention;

[0014] FIG. 6A to FIG. 6C is a schematic diagram of removing a person's background according to a second embodiment of the present invention;

[0015] Figure 7 A flow chart of a method for fixing the aspect ratio of a clothing image according to an embodiment of the present invention;

[0016] FIG. 8A to FIG. 8C is a schematic diagram of a fixed clothing image aspect ratio according to a third embodiment of the present invention;

[0017] Fig. 9 It is a schematic diagram of application results corresponding to different original images and original clothing images according to an embodiment of the present invention.

[0018] Explanation of symbols

[0019] 100: Virtual Dressing System

[0020] 102: Storage circuit

[0021] 104: Processor

[0022] 201: User Interface

[0023] 202,203: Upload button

[0024] 204,420,816: Clothing images

[0025] 205,411: Human body images

[0026] 206: Trigger button

[0027] 207,440,450,615,616,821,822: Virtual dressing result images

[0028] 411a: Human body

[0029] 412: Human skeleton diagram

[0030] 412a,412a',412b,412b',412c,412c': Area of ​​interest

[0031] 420a,811a: Clothing

[0032] 430: Reference result image

[0033] B: Skeleton image area

[0034] R: Specific image area

[0035] 611: Original image

[0036] 611a: human body image area

[0037] 611b: Background area

[0038] 612: Character Area

[0039] 613: Background Image

[0040] 613a: Empty area

[0041] 614: Reference background image

[0042] 811: Original clothing image

[0043] 811b: clothing image area

[0044] 812: Clothing area

[0045] 813: Reference clothing image

[0046] 813a: Clothing image range

[0047] 813b: Reference clothing image area

[0048] 814: Image

[0049] 815a,815b: Center point

[0050] H': specific height

[0051] W': Specific width

[0052] H: Height

[0053] W: Width

[0054] S310~S330, S510~S530, S710~S740: Steps DETAILED DESCRIPTION

[0055] Please refer to Figure 1 , which is a schematic diagram of a virtual dressing system according to an embodiment of the present invention. In different embodiments, the virtual dressing system 100 can be implemented as various smart devices and / or computer devices, but is not limited thereto.

[0056] exist Figure 1 In the embodiment, the virtual dressing system 100 includes a storage circuit 102 and a processor 104 .

[0057] The storage circuit 102 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk or other similar devices or a combination of these devices, and can be used to record multiple program codes or modules.

[0058] The processor 104 is coupled to the memory circuit 102 and may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), any other type of integrated circuit, a state machine, an Advanced RISC Machine (ARM) based processor, and the like.

[0059] In an embodiment of the present invention, the processor 104 can access the modules and program codes recorded in the storage circuit 102 to implement the method proposed by the present invention, so that the virtual dressing system 100 can execute the relevant image processing method for virtual dressing, and then generate a virtual dressing result image.

[0060] Please refer to FIG. 2A to FIG. 2C , which is a diagram of a virtual dressing application scenario according to an embodiment of the present invention. Figure 2AIn the embodiment, the user can, for example, select a clothing image to be virtually dressed in the user interface 201 provided by the virtual dressing system 100 (which can correspond to clothing items on a shopping website, but is not limited thereto), or upload the clothing image to be virtually dressed to the virtual dressing system 100 by himself through the upload button 202.

[0061] In addition, the user can also upload a human body image including a human body to the virtual dressing system 100 through the upload button 203, so that the virtual dressing system 100 can virtually wear the clothes in the clothing image on the human body in the human body image.

[0062] exist Figure 2B In the embodiment, assuming that the user uploads the clothing image 204 and the human body image 205 through the upload buttons 202 and 203 respectively, the virtual dressing system 100 may display a trigger button 206 in the user interface 201 for the user to trigger the virtual dressing system 100 to execute the relevant image processing method for virtual dressing.

[0063] exist Figure 2C In the embodiment, after the user clicks the trigger button 206, the virtual dressing system 100 can, for example, use the above-mentioned image processing method to virtually wear the clothes in the clothing image 204 on the human body in the human body image 205 to generate a virtual dressing result image 207 for the user's reference, and the user can also download the virtual dressing result image 207 for subsequent use, but it is not limited to this.

[0064] In the embodiment of the present invention, in order to enable the virtual dressing system 100 to provide a virtual dressing result image with better quality, the virtual dressing system 100 may execute the virtual dressing image pre-processing method proposed by the present invention, which will be further described below.

[0065] Please refer to Figure 3 , which is a flow chart of a method for pre-processing an image for virtual dressing according to an embodiment of the present invention. The method of this embodiment can be Figure 1 The virtual dressing system 100 is executed, and the following is matched Figure 1 Description of components shown Figure 3 To make the concept of the present invention easier to understand, the following is supplemented with FIG. 4A to FIG. 4E To illustrate, FIG. 4A to FIG. 4E FIG. 4 is a schematic diagram of image pre-processing according to the first embodiment of the present invention.

[0066] First, in step S310 , the processor 104 obtains a clothing image (hereinafter referred to as A) including clothing (hereinafter referred to as C) and a human body image 411 including a human body 411 a , and generates a human body skeleton image 412 corresponding to the human body 411 a .

[0067] exist Figure 4A In the clothing image A, the clothing type (hereinafter referred to as T) of the clothing C is, for example, upper body clothing (such as the T-shirt shown), lower body clothing (such as the trousers shown) or full body clothing (such as the dress shown), but is not limited thereto. In different embodiments, the clothing image A may be from a shopping website or uploaded by a user.

[0068] In one embodiment, the human body image 411 can be uploaded by the user. In another embodiment, the user can also take a full-body photo of himself / herself as the human body image 411 by using an imaging device in a clothing store when shopping for clothing. The clothing image A under consideration can be selected by the user from the website of the clothing store, or can be obtained by directly photographing the clothing in the clothing store by the imaging device, but is not limited thereto.

[0069] In one embodiment, the processor 104 may generate the human skeleton image 412 corresponding to the human body 411 a based on a related human posture recognition algorithm (eg, DeepPose), but is not limited thereto.

[0070] exist Figure 4A In the figure, the human skeleton graph 412 includes nodes corresponding to a plurality of joints on the human body 411a, and each node may have a corresponding number, and the relevant numbering principle may be determined according to the selected human posture recognition algorithm.

[0071] Thereafter, in step S320 , the processor 104 determines a skeleton image region (hereinafter referred to as B) in the human skeleton image 412 based on the clothing image A.

[0072] In one embodiment, the processor 104 may, for example, identify the clothing type T of the clothing C in the clothing image A, and determine the skeleton image region B in the human skeleton image 412 based on the clothing type T.

[0073] In different embodiments, the processor 104 may use different methods to identify the clothing type T of the clothing C. In one embodiment, if the clothing C is selected on-site at a clothing store, the user (or other relevant personnel) may use a relevant reader (e.g., a barcode reader) to directly read the identification information (e.g., a label barcode) of the clothing C, and the above reader may provide the read identification information to the virtual dressing system 100. Afterwards, the processor 104 may query the clothing database of the clothing store to obtain relevant information of the clothing C, such as the clothing type T, etc., but is not limited thereto.

[0074] In another embodiment, the processor 104 may feed the clothing image A into an image classification model (hereinafter referred to as M), wherein the image classification model M may predict the clothing type T of the clothing C in response to the clothing image A.

[0075] In different embodiments, the image classification model M may be implemented based on architectures such as Visual Geometry Group (VGG), MobileNet, and EfficientNet, but is not limited thereto.

[0076] In one embodiment, in order to enable the image classification model M to have the above capabilities, during the training process of the image classification model M, the designer may feed specially designed training data into the image classification model M so that the image classification model M can perform corresponding learning. For example, after obtaining clothing images that are labeled as corresponding to a certain clothing type (such as upper body clothing, lower body clothing, or full body clothing), they can be fed into the image classification model M in training as a piece of training data so that the image classification model M can learn what characteristics the clothing images corresponding to a certain clothing type have.

[0077] Based on the similarity concept, after feeding clothing images corresponding to different clothing types into the image classification model M in training, the image classification model M can learn the features of clothing images corresponding to different clothing types. Based on this, when a certain unknown clothing image is fed into the trained image classification model M, the image classification model M can predict / determine the corresponding clothing type accordingly, but it is not limited thereto.

[0078] In one embodiment, the processor 104 may determine the range of interest corresponding to the upper body skeleton, the lower body skeleton, or the whole body skeleton in the human skeleton diagram 412 based on the clothing type T, such as Figure 4B exemplified.

[0079] exist Figure 4B In response to determining that the clothing type T is upper body clothing, the processor 104 may determine an interest range 412a corresponding to the upper body skeleton (which includes, for example, nodes numbered 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 15, 16, 17, and 18) in the human skeleton graph 412. In response to determining that the clothing type T is lower body clothing, the processor 104 determines an interest range 412b corresponding to the lower body skeleton (which includes, for example, nodes numbered 8, 9, 10, 11, 12, 13, 14, 19, 20, 21, 22, 23, and 24) in the human skeleton graph 412. In response to determining that the clothing type T is full body clothing, the processor 104 determines an interest range 412c corresponding to the full body skeleton (for example, the entire human skeleton graph 412) in the human skeleton graph 412.

[0080] Afterwards, the processor 104 may adjust the considered interest range according to a specific ratio, and determine the image region corresponding to the adjusted interest range in the human skeleton image 412 as the skeleton image region B. In one embodiment, the adjusted interest range may not exceed the size of the human skeleton image 412 .

[0081] For example, if the considered interest range is the interest range 412a (i.e., the context in which the clothing type T is upper body clothing), the processor 104 may multiply the interest range 412a by a specific ratio (e.g., 1.4) to adjust the interest range 412a to the interest range 412a'. Thereafter, the processor 104 may determine the image region corresponding to the interest range 412a' as the skeleton image region B.

[0082] For another example, if the considered range of interest is the range of interest 412b (i.e., the situation where the clothing type T is lower body clothing), the processor 104 may multiply the range of interest 412b by a specific ratio (e.g., 1.4) to adjust the range of interest 412b to the range of interest 412b'. Thereafter, the processor 104 may determine the image region corresponding to the range of interest 412b' as the skeleton image region B.

[0083] For another example, if the considered interest range is the interest range 412c (i.e., the clothing type T is a situation of full-body clothing), the processor 104 may multiply the interest range 412c by a specific ratio (e.g., 1) to adjust the interest range 412c to the interest range 412c', but is not limited thereto. Thereafter, the processor 104 may determine the image region corresponding to the interest range 412c' as the skeleton image region B.

[0084] Next, in step S330 , the processor 104 extracts a specific image region (hereinafter referred to as R) corresponding to the skeleton image region B from the human body image 411 .

[0085] exist Figure 4C In the scenario, assuming that the adjusted interest range under consideration is the interest range 412a′ (i.e., the scenario in which the clothing type T is upper body clothing), the processor 104 may accordingly determine the skeleton image region B in the human skeleton image 412. Thereafter, the processor 104 may intercept a specific image region R corresponding to the skeleton image region B from the human image 411.

[0086] In the embodiment of the present invention, by adjusting the range of interest based on the specific ratio, it is possible to avoid the captured specific image region R including a truncated human body part (eg, a truncated top of the head and / or the end of a limb).

[0087] In one embodiment, after obtaining a specific image area R including a human body part (e.g., the upper body), the processor 104 may virtually wear the clothing C in the clothing image A on the human body part in the specific image area R, and combine the human body part wearing the clothing C with the human body image 411 to generate a virtual dressing result image.

[0088] exist Figure 4DIn the scenario, assuming that the clothing image A under consideration is a clothing image 420 including clothing 420a (for example, upper body clothing), the processor 104 may, after obtaining the specific image area R, virtually wear the clothing 420a on a human body part (for example, the upper body) in the specific image area R to generate a reference result image 430.

[0089] In different embodiments, the processor 104 may virtually wear the clothing 420a on a human body part in a specific image area R based on various existing virtual dressing algorithms (e.g., “High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditions”, “Style-Based Global Appearance Flow for Virtual Try-On”, etc.).

[0090] Afterwards, the processor 104 may combine the reference result image 430 with the human body image 411 to form a virtual dressing result image 440 (which may be referred to as a first virtual dressing result image). For example, the processor 104 may cover a specific image region R in the human body image 411 with the reference result image 430 to obtain the virtual dressing result image 440.

[0091] pass Figure 3 The method shown can enable the virtual dressing result image 440 to provide a better virtual dressing effect, thereby improving the user experience.

[0092] exist Figure 4E In the example, the virtual dressing result image 450 is not previously Figure 3 The method performs image pre-processing, and directly applies the existing virtual dressing algorithm to virtually wear the clothing 420a in the clothing image 420 on the human body 411a in the human body image 411.

[0093] Depend on Figure 4E It can be seen that the position of the clothing pattern in the virtual dressing result image 450 is obviously too low, so it cannot provide a good virtual dressing effect. In contrast, the virtual dressing result image 440 can provide a more accurate virtual dressing effect. Figure 3 After the method shown obtains the specific image region R corresponding to the skeleton image region B (which can also be understood as obtaining the corresponding upper body image region, lower body image region or full body image region based on the clothing type T), the subsequent virtual dressing algorithm can more accurately virtually wear the clothing 420a in the clothing image 420 on the human body part in the specific image region R. Thus, a better virtual dressing effect can be provided.

[0094] In the above embodiment, since the human body image 411 is of the type without background (or with a relatively simple background), it can be directly used to perform Figure 3 However, for images with complex backgrounds, the complex background may lead to uncertainty in the virtual dressing result, which may cause the quality of the final virtual dressing result to deteriorate. In view of this, the embodiment of the present invention further proposes a method for removing the background of a person to try to solve the above technical problem.

[0095] Please refer to Figure 5 , which is a flow chart of a method for removing a person's background according to an embodiment of the present invention. The method of this embodiment can be Figure 1 The virtual dressing system 100 is executed, and the following is matched Figure 1 Description of components shown Figure 5 To make the concept of the present invention easier to understand, the following is supplemented with FIG. 6A to FIG. 6C To illustrate, FIG. 6A to FIG. 6C FIG. 4 is a schematic diagram of removing a person's background according to the second embodiment of the present invention.

[0096] First, in step S510, the processor 104 obtains an original image 611, wherein the original image 611 includes a background area 611b (which includes, for example, a white wall) and a human image area 611a corresponding to the human body 411a. In this embodiment, the original image 611 is, for example, uploaded by a user, or is taken on-site in a clothing store, but is not limited thereto.

[0097] In one embodiment, the processor 104 may, for example, execute an existing person region segmentation algorithm based on the original image 611 to obtain the person region 612 , and accordingly determine the human image region 611 a and the background region 611 b in the original image 611 .

[0098] In step S520, the processor 104 separates the human image region 611a from the original image 611 to generate the human image 411 and a background image 613 corresponding to the background region 611b. Fig. 6A In the original image 611, the background image 613 may include an empty area 613a corresponding to the human image area 611a, and the processor 104 may fill the empty area 613a with black pixels. In addition, after separating the human image area 611a from the original image 611, the processor 104 may fill the portion corresponding to the background area 611b with white pixels to form the human image 411, but is not limited thereto.

[0099] In step S530, the processor 104 fills the empty area 613a to convert the background image 613 into a reference background image 614. Figure 6BIn one embodiment, the processor 104 may fill the empty area 613a based on, for example, the Inpaint technology or other similar filling technologies, but is not limited thereto.

[0100] exist Figure 6B In the embodiment of FIG. 5 , the processor 104 may further generate a virtual dressing result image 615 (which may be referred to as a second virtual dressing result image) by combining the virtual dressing result image 440 with the reference background image 614 .

[0101] pass Figure 5 The method shown can enable the virtual dressing result image 615 to provide a better virtual dressing effect, thereby improving the user experience.

[0102] exist Figure 6C In the example, the virtual dressing result image 616 is not previously Figure 5 The method removes the background of the person, and directly applies the existing virtual dressing algorithm to virtually wear the clothing 420a in the clothing image 420 on the human body 411a in the original image 611.

[0103] Depend on Figure 6C It can be seen that the background near the hand in the virtual dressing result image 615 has obvious noise, so it cannot provide a good virtual dressing effect. In contrast, the virtual dressing result image 614 can avoid excessive noise in the background, thereby providing a better virtual dressing effect.

[0104] In some embodiments, different clothing brands may have different ways of photographing their clothing, which may lead to differences in the final virtual dressing results. For example, if different clothing images have different aspect ratios due to different shooting methods, the final virtual dressing results may not be ideal.

[0105] In view of this, an embodiment of the present invention further proposes a method for fixing the aspect ratio of clothing images to attempt to solve the above technical problem.

[0106] Please refer to Figure 7 , which is a flow chart of a method for fixing the aspect ratio of a clothing image according to an embodiment of the present invention. The method of this embodiment can be performed by Figure 1 The virtual dressing system 100 is executed, and the following is matched Figure 1 Description of components shown Figure 7 To make the concept of the present invention easier to understand, the following is supplemented with FIG. 8A to FIG. 8C To illustrate, FIG. 8A to FIG. 8C FIG. 4 is a schematic diagram of fixing the aspect ratio of a clothing image according to a third embodiment of the present invention.

[0107] First, in step S710 , the processor 104 obtains an original clothing image 811 including clothing 811 a , and identifies a clothing image region 811 b corresponding to the clothing 811 a in the original clothing image 811 .

[0108] In one embodiment, the processor 104 may, for example, execute an existing clothing region segmentation algorithm (which may be the same as the above-mentioned person region segmentation algorithm) based on the original clothing image 811 to obtain the clothing region 812 , and determine the clothing image region 811 b in the original clothing image 811 accordingly.

[0109] In step S720, the processor 104 fills the original clothing image 811 into a reference clothing image 813, and determines the clothing image range 813a in the reference clothing image 813 based on the clothing image area 811b. In one embodiment, the processor 104 may, for example, expand the height and width of the original clothing image 811 (whose resolution is, for example, 500×500) by 2 times, and fill the portion that does not belong to the original clothing image 811 with white pixels to form a reference clothing image 813 (whose resolution is, for example, 1000×1000). Similarly, the processor 104 may also fill the clothing area 812 accordingly to form an image 814 (whose resolution is, for example, 1000×1000).

[0110] Afterwards, the processor 104 may use a rectangular frame as the clothing image range 813a, wherein the top, bottom, left and right sides of the rectangular frame correspond to the top, bottom, leftmost and rightmost pixels of the clothing image area 811b respectively, but is not limited thereto.

[0111] In step S730 , the processor 104 captures a reference clothing image region 813 b including a clothing image range 813 a in the reference clothing image 813 , wherein the reference clothing image region 813 b has a preset aspect ratio (eg, height:width is 4:3).

[0112] exist Figure 8B , the reference clothing image region 813b has a specific height H' and a specific width W'.

[0113] In one embodiment, in response to determining that the aspect ratio of the clothing image range 813a satisfies a preset condition (for example, the aspect ratio is greater than 1.33 times), the processor 104 may expand the width W of the clothing image range 813a by a first preset multiple (for example, 1.25 times) to determine a specific width W', and expand the specific width W' by a second preset multiple (for example, 4 / 3 times) to determine the specific height H'.

[0114] In another embodiment, in response to determining that the aspect ratio of the clothing image range 813a does not meet a preset condition (for example, the aspect ratio is not greater than 1.33 times), the processor 104 may expand the height H of the clothing image range 813a by a first preset multiple to determine a specific height H', and expand the specific height H' by a second preset multiple to determine a specific width W'.

[0115] In addition, Figure 8B , the center point 815b of the reference clothing image region 813b corresponds to the center point 815a of the clothing image range 813a, but is not limited thereto.

[0116] Then, in step S740, the processor 104 converts the resolution of the reference clothing image area 813b into a preset resolution (eg, 1024x768) to generate a clothing image 816, such as Figure 8C shown.

[0117] exist Figure 8C , the processor 104 may execute a virtual dressing algorithm based on the original clothing image 811 and the original image 611 to obtain a virtual dressing result image 821. In addition, the processor 104 may execute a virtual dressing algorithm based on the clothing image 816 and the original image 611 to obtain a virtual dressing result image 822.

[0118] Depend on Figure 8C It can be seen that if you do not first execute Figure 7 If the virtual dressing algorithm is directly executed based on the original clothing image 811 and the original image 611, the virtual dressing result image 821 may have unsatisfactory effects (for example, the sleeves may not fit the arms correctly). Figure 7 The original clothing image 811 is converted into a clothing image 816 by the method, and then the virtual dressing algorithm is executed accordingly, so that the virtual dressing result image 822 can have a better effect (for example, the sleeves can correctly match the arms).

[0119] Please refer to Fig. 9 , which is a schematic diagram of application results corresponding to different original images and original clothing images according to an embodiment of the present invention.

[0120] Depend on Fig. 9 It can be seen that compared to directly executing the virtual dressing algorithm based on the original image and the original clothing image, first applying the virtual dressing image pre-processing method, the character background elimination method and / or the clothing image aspect ratio fixing method of the embodiments of the present invention and then applying the virtual dressing algorithm can obtain a virtual dressing result image with better effect.

[0121] In other embodiments, after executing the image pre-processing method for virtual dressing, the method for removing the background of a person, and / or the method for fixing the aspect ratio of a clothing image according to the embodiments of the present invention, the obtained clothing image (for example Figure 8C clothing image 816) and / or a specific image region corresponding to the skeleton image region (e.g. Figure 4D Other pre-processing methods may be used for the specific image area R), such as "High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditions", "Style-Based Global Appearance Flow for Virtual Try-On" and other pre-processing methods corresponding to various virtual dressing algorithms, but are not limited to these.

[0122] In summary, the embodiments of the present invention can improve the virtual dressing result image provided by the subsequent virtual dressing algorithm through the virtual dressing image pre-processing method, the character background elimination method and / or the clothing image aspect ratio fixing method, so as to provide a better user experience.

[0123] In addition, since the quality of the virtual dressing result image can be improved, the method of the embodiment of the present invention can at least achieve the following effects: (1) Improve shopping experience: Allow consumers to try on clothes without leaving home, thereby increasing their online shopping experience; (2) Reduce return rate: Allow consumers to know in advance the wearing effect of the clothing and whether it fits their own dressing style, thereby reducing the return rate of consumers who are dissatisfied with the clothes after receiving them; (3) Increase sales: Compared with other clothing brands that have not introduced virtual dressing products, this function can be used as a publicity gimmick to attract consumers to use this virtual dressing product, thereby increasing their sales; (4) Increase customer repurchase rate: With the clothing recommendation algorithm, the recommended clothes can be tried on immediately to attract consumers to buy, thereby increasing consumers' stickiness to the brand; (5) Reduce costs: Through virtual dressing technology, the inventory cost of store clothes is reduced, and at the same time, product damage caused by trying on is reduced, such as leaving lipstick marks on the clothes when trying on, and the clothes tried on can only be sold as welfare products.

[0124] Although the present invention is disclosed in conjunction with the above embodiments, they are not intended to limit the present invention. Any ordinary technician in the technical field may make slight changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the attached claims.

Claims

1. A virtual dressing image pre-processing method, suitable for a virtual dressing system, comprising: Acquire a clothing image including clothing and a human body image including a human body, and generate a human body skeleton image corresponding to the human body; Determine a skeleton image region in the human skeleton image based on the clothing image; as well as A specific image region corresponding to the skeleton image region is captured from the human body image.

2. The image pre-processing method according to claim 1, wherein the step of intercepting the skeleton image region from the human skeleton image based on the clothing image comprises: identifying the clothing type of the clothing in the clothing image; as well as The skeleton image region is determined in the human skeleton image based on the type of clothing.

3. The image pre-processing method as claimed in claim 2, wherein the step of determining the skeleton image region in the human skeleton image based on the type of clothing comprises: Determine, in the human skeleton image, a range of interest corresponding to an upper body skeleton, a lower body skeleton, or a full body skeleton based on the type of clothing; as well as The range of interest is adjusted according to a specific ratio, and an image region corresponding to the adjusted range of interest is determined in the human skeleton image as the skeleton image region.

4. The image pre-processing method as claimed in claim 3, wherein the step of determining the range of interest corresponding to the upper body skeleton, the lower body skeleton or the whole body skeleton in the human skeleton image based on the type of clothing comprises: In response to determining that the clothing type is upper body clothing, determining the range of interest corresponding to the upper body skeleton in the human skeleton image; In response to determining that the clothing type is lower body clothing, determining the range of interest corresponding to the lower body skeleton in the human skeleton image; and In response to determining that the clothing type is full-body clothing, the range of interest corresponding to the full-body skeleton is determined in the human skeleton image.

5. The image pre-processing method as claimed in claim 2, wherein the step of identifying the clothing type of the clothing in the clothing image comprises: feeding the clothing image into an image classification model, wherein the image classification model predicts the clothing type of the clothing in response to the clothing image; or The identification information of the clothing is obtained, and the clothing type of the clothing is determined based on the identification information.

6. The image pre-processing method according to claim 1, further comprising: Acquire an original image, wherein the original image includes a background area and a human body image area corresponding to the human body; Separating the human body image region from the original image to generate the human body image and a background image corresponding to the background region, wherein the background image includes a blank region corresponding to the human body image region; The background image is converted into a reference background image by filling the empty area.

7. The image pre-processing method as claimed in claim 1, wherein the specific image area includes a human body part, and the method further comprises: Virtually wearing the clothing in the clothing image on the human body part in the specific image area to generate a reference result image; as well as The reference result image and the human body image are combined into a first virtual dressing result image.

8. The image pre-processing method according to claim 7, further comprising: A second virtual dressing result image is generated by combining the first virtual dressing result image with the reference background image.

9. The image pre-processing method according to claim 1, further comprising: Acquire an original clothing image including the clothing, and identify a clothing image region corresponding to the clothing in the original clothing image; Filling the original clothing image as a reference clothing image, and determining a clothing image range in the reference clothing image based on the clothing image area; capturing a reference clothing image region including the clothing image range in the reference clothing image, wherein the reference clothing image region has a preset aspect ratio; as well as The resolution of the reference clothing image region is converted into a preset resolution to generate the clothing image.

10. The image pre-processing method according to claim 9, wherein the reference clothing image region has a specific height and a specific width, and the method further comprises: In response to determining that the aspect ratio of the clothing image range satisfies a preset condition, the width of the clothing image range is enlarged by a first preset multiple to determine the specific width, and the specific width is enlarged by a second preset multiple to determine the specific height; In response to determining that the aspect ratio of the clothing image range does not meet the preset condition, the height of the clothing image range is enlarged by the first preset multiple to determine the specific height, and the specific height is enlarged by the second preset multiple to determine the specific width; The center point of the reference clothing image area corresponds to the center point of the clothing image range.

11. A virtual dressing system, comprising: a storage circuit storing program code; as well as A processor coupled to the storage circuit and accessing the program code to execute: Acquire a clothing image including clothing and a human body image including a human body, and generate a human body skeleton image corresponding to the human body; Based on the clothing image, a skeleton image region is intercepted from the human skeleton image; as well as A specific image region corresponding to the skeleton image region is captured from the human body image.

12. The virtual dressing system of claim 11, wherein the processor executes: identifying the clothing type of the clothing in the clothing image; and The skeleton image region is determined in the human skeleton image based on the type of clothing.

13. The virtual dressing system of claim 12, wherein the processor executes: Determining a range of interest corresponding to an upper body skeleton, a lower body skeleton, or a full body skeleton in the human skeleton image based on the type of clothing; and The range of interest is adjusted according to a specific ratio, and an image region corresponding to the range of interest is determined in the human skeleton image as the skeleton image region.

14. The virtual dressing system of claim 13, wherein the processor executes: In response to determining that the clothing type is upper body clothing, determining the range of interest corresponding to the upper body skeleton in the human skeleton image; In response to determining that the clothing type is lower body clothing, determining the range of interest corresponding to the lower body skeleton in the human skeleton image; and In response to determining that the clothing type is full-body clothing, the range of interest corresponding to the full-body skeleton is determined in the human skeleton image.

15. The virtual dressing system of claim 12, wherein the processor executes: feeding the clothing image into an image classification model, wherein the image classification model predicts the clothing type of the clothing in response to the clothing image; or The identification information of the clothing is obtained, and the clothing type of the clothing is determined based on the identification information.

16. The virtual dressing system of claim 11, wherein the processor further executes: Acquire an original image, wherein the original image includes a background area and a human body image area corresponding to the human body; Separating the human body image region from the original image to generate the human body image and a background image corresponding to the background region, wherein the background image includes a blank region corresponding to the human body image region; The background image is converted into a reference background image by filling the empty area.

17. The virtual dressing system as claimed in claim 11, wherein the specific image area includes a human body part, and the processor further executes: Virtually wearing the clothing in the clothing image on the human body part in the specific image area to generate a reference result image; as well as combining the reference result image and the human body image into a first virtual dressing result image; as well as A second virtual dressing result image is generated by combining the first virtual dressing result image with the reference background image.

18. The virtual dressing system of claim 11, wherein the processor further executes: Acquire an original clothing image including the clothing, and identify a clothing image region corresponding to the clothing in the original clothing image; Filling the original clothing image as a reference clothing image, and determining a clothing image range in the reference clothing image based on the clothing image area; capturing a reference clothing image region including the clothing image range in the reference clothing image, wherein the reference clothing image region has a preset aspect ratio; as well as The resolution of the reference clothing image region is converted into a preset resolution to generate the clothing image.

19. The virtual dressing system as claimed in claim 18, wherein the reference clothing image region has a specific height and a specific width, and the processor further executes: In response to determining that the aspect ratio of the clothing image range satisfies a preset condition, the width of the clothing image range is enlarged by a first preset multiple to determine the specific width, and the specific width is enlarged by a second preset multiple to determine the specific height; In response to determining that the aspect ratio of the clothing image range does not meet the preset condition, the height of the clothing image range is enlarged by the first preset multiple to determine the specific height, and the specific height is enlarged by the second preset multiple to determine the specific width; The center point of the reference clothing image area corresponds to the center point of the clothing image range.

20. A computer-readable storage medium, wherein the computer-readable storage medium records an executable computer program, wherein the executable computer program is loaded by a virtual dressing system to perform the following steps: Acquire a clothing image including clothing and a human body image including a human body, and generate a human body skeleton image corresponding to the human body; Extracting a skeleton image region from the human skeleton image based on the clothing image; and A specific image region corresponding to the skeleton image region is captured from the human body image.

Images