AI ornament virtual wearing system and method

Abstract

The invention discloses an AI ornament virtual wearing system and method, and the system comprises an image collection module, an ornament selection module, an image generation module and a display module, and the image collection module is used for obtaining a portrait image of a user; the ornament selection module is used for selecting a target ornament by a user and receiving target ornament information; the image generation module is used for performing fusion processing on the portrait image and the target ornament information to generate a virtual wearing effect image for virtually wearing the target ornament on the portrait image; the display module is used for displaying the virtual wearing effect image; a user portrait image is acquired through an image acquisition module, a plurality of candidate virtual wearing effect images are generated by a generator of an image generation module in combination with a random noise vector, the portrait image and target ornament information, a discriminator performs authenticity evaluation on the candidate images and outputs confidence, and an optimal effect image conforming to a preset threshold is screened.

Description

Technical Field

[0001] This invention relates to the fields of image processing and artificial intelligence technology, specifically to an AI-powered virtual jewelry wearing system and method. Background Technology

[0002] As living standards improve, consumers' demand for fashionable and personalized jewelry is growing, and the purchasing scenarios for jewelry are gradually extending from traditional physical stores to online platforms. Among the current mainstream purchasing methods, trying on jewelry in physical stores requires individual operation, which is inefficient and limited by geographical location and store inventory; online shopping relies solely on pictures or videos, and users cannot intuitively perceive the fit and overall effect of the jewelry after wearing it, which can easily lead to returns and exchanges due to discrepancies between expectations and reality, reducing the shopping experience and transaction efficiency.

[0003] While existing image generation technologies have made virtual try-on possible, they still have significant shortcomings: on the one hand, the fusion effect between virtual jewelry and human images is poor, easily resulting in problems such as misaligned contours and inconsistent lighting, making it difficult to reproduce the feel of real wearing; on the other hand, the application of the technology has a high degree of uncertainty, and the generation effect lacks stability, failing to efficiently meet users' core needs for rapid screening and accurate matching. Therefore, the applicant proposes an AI-powered virtual jewelry wearing system and method. Summary of the Invention

[0004] This invention aims to improve the problems of low efficiency in purchasing jewelry, unintuitive online experience, and insufficient accuracy of image fusion and poor stability of generated effects in existing virtual try-on technology.

[0005] This invention is implemented as follows: According to a first aspect of the present invention, the present invention provides an AI-powered virtual jewelry wearing system, comprising: The image acquisition module is used to acquire the user's portrait image; The jewelry selection module is used by users to select target jewelry and receive information about the target jewelry. The image generation module is used to fuse the portrait image with the target jewelry information to generate a virtual wearing effect image in which the target jewelry is virtually worn on the portrait image; The display module is used to display the virtual wearing effect image.

[0006] Furthermore, the image generation module is built based on a generative adversarial network and includes: A generator is used to receive a noise vector randomly sampled from a normal distribution, the portrait image, and the target jewelry information, and to generate multiple candidate virtual wearing effect images by mapping them to an upsampling layer through a deconvolutional network; The discriminator is used to evaluate the authenticity of an image through a convolutional network and outputs a confidence level between 0 and 1, where 1 represents certainty that the image is true and 0 represents certainty that the image is false.

[0007] Furthermore, the confidence evaluation result of the discriminator is used to screen the optimal image. The screening criterion is candidate images with confidence ≥ a preset threshold. If there are multiple images that meet the conditions, the image with the highest confidence is selected as the final virtual wearing effect image.

[0008] Furthermore, it also includes: The preprocessing module is used to perform key point detection and standardization on the portrait image, and to detect the wearing position area; The posture guidance module is used to provide users with posture adjustment guidance during image acquisition.

[0009] According to a second aspect of the present invention, the present invention provides an AI-powered virtual jewelry wearing method, using the aforementioned AI-powered virtual jewelry wearing system, comprising the following steps: S1. Acquire the user's portrait image through the image acquisition module; S2. Obtain the target jewelry information corresponding to the target jewelry selected by the user through the jewelry selection module; S3. The image generation module fuses the portrait image with the target jewelry information to generate a virtual wearing effect image of the target jewelry being virtually worn on the portrait image. S4. Display the virtual wearing effect image through the display module.

[0010] Furthermore, the image generation module is constructed based on a generative adversarial network, including a generator and a discriminator trained adversarially. Step S3 specifically includes: S31. The generator receives a random noise vector, the portrait image, and the target jewelry information, and generates multiple candidate virtual wearing effect images; S32. The discriminator compares and evaluates the candidate image with the real image of the jewelry being worn, and outputs the confidence score. S33. Filter candidate images with confidence levels ≥ preset thresholds to determine the final virtual wearing effect image.

[0011] Furthermore, the AI ​​jewelry virtual wearing system also includes a posture guidance module, and before step S1, it further includes: S0. The posture guidance module detects the user's posture, and triggers the image acquisition module to acquire images when the posture meets the preset conditions.

[0012] Furthermore, the AI ​​jewelry virtual wearing system also includes a preprocessing module, and step S1 further includes: S11. The preprocessing module performs key point detection on the portrait image to locate at least one jewelry wearing area; S12. The image of the wearing area is standardized by the preprocessing module. The standardization process includes at least one of size normalization, illumination equalization, and background simplification.

[0013] Furthermore, the target jewelry information includes at least one of the jewelry outline mask image and the product image, wherein the outline mask image is extracted from the image of the target jewelry selected by the user through image segmentation technology.

[0014] Furthermore, the adversarial training process for the generator and discriminator includes the following steps performed alternately: First training phase: Fix generator parameters, optimize discriminator, and maximize discriminator's ability to distinguish between real image x and generated image G(z) by using loss function LD=-[log(D(x))+log(1-D(G(z)))]; The second training phase involves fixing the discriminator parameters and optimizing the generator. The realism of the generated images is improved by using the loss function LG=-log(D(G(z))), making the confidence level of the discriminator output close to 1.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention acquires the user's portrait image through the image acquisition module, selects the target jewelry through the jewelry selection module and receives the outline mask image of the target jewelry, the product image and other information, and then the generator trained by the adversarial image generation module based on the generative adversarial network combines the random noise vector, the portrait image and the target jewelry information to generate multiple candidate virtual wearing effect images. The discriminator evaluates the authenticity of the candidate images and outputs the confidence level. The optimal effect image that meets the preset threshold is selected and displayed through the display module. Attached Figure Description

[0016] Figure 1 This is a flowchart of the AI-powered virtual jewelry wearing method provided by the present invention; Figure 2 This is a block diagram of the module structure of the AI ​​jewelry virtual wearing system provided by the present invention. Detailed Implementation

[0017] The following description, in conjunction with the accompanying drawings and specific embodiments, provides further details: Example 1

[0018] This embodiment provides an AI-powered virtual jewelry wearing system, such as... Figure 2As shown, the system includes an image acquisition module, a posture guidance module, an accessory selection module, a preprocessing module, an image generation module, and a display module. The system requires specific hardware and software environments. The hardware environment includes: a 4-core CPU (1.5GHz) and 4GB of RAM; a 4-megapixel camera supporting autofocus and low-light enhancement, with a shooting frame rate ≥30fps to ensure clear and real-time image capture; and a touch-enabled Android screen with 2K resolution, touch response latency ≤50ms, and color reproduction ≥95% to ensure smooth user interaction and realistic image display. The software environment is Android 12, and the programming languages ​​are Kotlin and Java.

[0019] Among the above modules, the image acquisition module is used to acquire the user's portrait image, with the acquisition range limited to the user's upper body or hands. The posture guidance module provides posture adjustment guidance to the user during image acquisition, displaying text prompts, schematic outlines, and dynamic posture diagrams through the display module, while also providing voice guidance. Acquisition is triggered when the user's posture meets preset conditions for 2 consecutive seconds. The preprocessing module performs key point detection and standardization on the portrait image, and detects the wearing position area. The MediaPipe human posture recognition algorithm is used to accurately locate 128 key feature points such as facial contours, neck, ears, and hands, with a wearing area positioning error of ≤2 pixels; after standardization, the image size is uniformly 512×512 pixels, and the pixel values ​​are normalized to the [0,1] range to eliminate the influence of light intensity differences within ±30%. The jewelry selection module is used for the user to select target jewelry and receives target jewelry information, which includes at least one of the jewelry's outline mask image and product image. The image generation module fuses the portrait image with the target jewelry information to generate a virtual wearing effect image that virtually places the target jewelry onto the portrait image. The display module shows the virtual wearing effect image, supporting two modes: full-screen display of a single image and split-screen comparison display of multiple candidate effects. Users can zoom in and out with two fingers to view details, and the module also supports screenshot saving.

[0020] The image generation module is built on a generative adversarial network (GAN) and includes a generator and a discriminator trained adversarially. The generator uses a 3-layer deconvolutional network + 2-layer upsampling layer network structure. Its input channels are 3 (portrait image) + 3 (jewelry image) + 100 (noise vector), and its output channel is 3. The output channel receives a noise vector (100-dimensional, mean 0, variance 1) randomly sampled from a normal distribution, the portrait image, and the target jewelry information. It generates multiple candidate virtual wearing effect images by mapping these noise vectors to the upsampling layer through the deconvolutional network. The number of generated images can be configured from 5 to 8. The discriminator uses a 4-layer convolutional network + 2-layer fully connected layer network structure. The convolutional kernel size is 3×3, and the stride is 2. It evaluates the realism of the image through the convolutional network and outputs a confidence score between 0 and 1, where 1 represents certainty of being true and 0 represents certainty of being false. The discriminator's confidence assessment result is used to select the optimal image. The selection criterion is candidate images with a confidence level ≥ a preset threshold. The preset threshold range can be appropriately selected according to the actual situation, for example, a number between 0.7 and 0.9 can be chosen. If multiple images meet the criteria, the image with the highest confidence level is selected as the final virtual wearing effect image. If the highest confidence level is lower than the preset threshold, the generator is automatically triggered to regenerate candidate images, and the maximum number of retries can be set.

[0021] The adversarial training process for the generator and discriminator includes the following steps performed alternately: The first training phase involves fixing the generator parameters and optimizing the discriminator. The training batch size is 32, and the number of iterations is 10,000. The discriminator's ability to distinguish between the real image x and the generated image G(z) is maximized using the loss function LD = -[log(D(x)) + log(1-D(G(z)))]. The real image x comes from a labeled dataset containing over 100,000 images of different people and different accessories, covering users aged 18-60 and with a height of 150-190cm.

[0022] The second training phase involves fixing the discriminator parameters and optimizing the generator. The batch size and number of iterations are the same as in the first training phase. The fidelity of the generated images is improved using the loss function LG = -log(D(G(z))), making the confidence level of the discriminator output close to 1. The Adam optimizer is used during training, with an initial learning rate of 0.0002, which decreases to 0.9 times the original rate every 2000 iterations. The final generated images have an FID value ≤ 30 and an IS value ≥ 12. Example 2

[0023] This embodiment provides an AI-powered virtual jewelry wearing method, using the AI-powered virtual jewelry wearing system provided in Embodiment 1, such as... Figure 1 As shown, it includes the following steps: S0. The posture guidance module detects the user's posture and triggers the image acquisition module to acquire images when the posture meets the preset conditions.

[0024] S1. Acquire the user's portrait image through the image acquisition module, and preprocess the acquired portrait image through the preprocessing module. The preprocessing includes: S11. The preprocessing module performs key point detection on the portrait image to locate at least one jewelry wearing area. The corresponding wearing area is automatically matched according to the type of jewelry, such as the neck area for necklaces, the earrings for earrings, and the hand area for bracelets. It supports locating multiple wearing areas at the same time to realize the simulation of wearing multiple jewelry combinations.

[0025] S12. The image of the wearing area is standardized by the preprocessing module. The standardization process includes at least one of size normalization, illumination equalization, and background simplification.

[0026] S2. The jewelry selection module retrieves the target jewelry information corresponding to the user's selected target jewelry. This target jewelry information includes at least one of the following: a contour mask image of the jewelry and a product image. The contour mask image is extracted from the image of the user-selected target jewelry using image segmentation technology, preserving details such as the jewelry's openwork and inlays during the segmentation process. The jewelry selection module provides a graphical user interface, displaying jewelry for users to choose from in lists, grids, or categories. Category dimensions include jewelry type, style, material, and price range, and keyword search is supported. The jewelry selection module receives user selection actions, including touch clicks, swipe selections, or voice commands. After the user selects a target jewelry item through the jewelry selection module, the display module simultaneously displays the original product image of the selected jewelry, along with textual information such as material descriptions, size parameters, and applicable scenarios, ensuring readability.

[0027] S3. The image generation module fuses the portrait image with the target jewelry information to generate a virtual wearing effect image that virtually places the target jewelry onto the portrait image; the specific steps are as follows: S31. The generator receives a random noise vector, a portrait image, and information about the target accessory, and generates multiple candidate virtual wearing effect images. S32. The discriminator compares and evaluates the candidate image with the real image of the jewelry being worn, and outputs the confidence score. S33. Filter candidate images with confidence levels ≥ preset thresholds to determine the final virtual wearing effect image.

[0028] S4. Display the virtual wearing effect image through the display module. Users can operate and provide feedback on whether they are satisfied with the virtual wearing effect. If satisfied, they can submit and purchase the desired jewelry. If not satisfied, they can return to step S3 to regenerate the virtual wearing effect image, or return to step S2 to reselect jewelry, or provide feedback to step S1 to re-capture the image.

[0029] In summary, this invention acquires a user's portrait image through an image acquisition module, selects a target jewelry item through a jewelry selection module, and receives information such as the target jewelry's contour mask image and the product image. Then, a generator trained on adversarial principles by a generative adversarial network-based image generation module combines random noise vectors, the portrait image, and the target jewelry information to generate multiple candidate virtual wearing effect images. A discriminator evaluates the realism of the candidate images and outputs a confidence score. The optimal effect image, meeting a preset threshold, is then displayed through a display module. Furthermore, user satisfaction feedback can be incorporated to create an adjustable interactive flow, ultimately enabling virtual jewelry try-on and purchase.

[0030] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An AI-powered virtual jewelry wearing system, characterized in that, include: The image acquisition module is used to acquire the user's portrait image; The jewelry selection module is used by users to select target jewelry and receive information about the target jewelry. The image generation module is used to fuse the portrait image with the target jewelry information to generate a virtual wearing effect image in which the target jewelry is virtually worn on the portrait image; The display module is used to display the virtual wearing effect image.

2. The AI-powered virtual jewelry wearing system according to claim 1, characterized in that, The image generation module is built based on a generative adversarial network and includes: A generator is used to receive a noise vector randomly sampled from a normal distribution, the portrait image, and the target jewelry information, and generate multiple candidate virtual wearing effect images; The discriminator is used to evaluate the authenticity of an image and outputs a confidence level between 0 and 1, where 1 represents certainty that it is true and 0 represents certainty that it is false.

3. The AI-powered virtual jewelry wearing system according to claim 2, characterized in that, The confidence evaluation result of the discriminator is used to select the optimal image. The selection criterion is candidate images with confidence ≥ preset threshold. If there are multiple images that meet the conditions, the image with the highest confidence is selected as the final virtual wearing effect image.

4. The AI-powered virtual jewelry wearing system according to claim 1, characterized in that, Also includes: The preprocessing module is used to perform key point detection and standardization on the portrait image, and to detect the wearing position area; The posture guidance module is used to provide users with posture adjustment guidance during image acquisition.

5. A method for virtual wearing of AI jewelry, using the AI ​​jewelry virtual wearing system according to any one of claims 1-4, characterized in that, Includes the following steps: S1. Acquire the user's portrait image through the image acquisition module; S2. Obtain the target jewelry information corresponding to the target jewelry selected by the user through the jewelry selection module; S3. The image generation module fuses the portrait image with the target jewelry information to generate a virtual wearing effect image of the target jewelry being virtually worn on the portrait image. S4. Display the virtual wearing effect image through the display module.

6. The AI-powered virtual jewelry wearing method according to claim 5, characterized in that, The image generation module is built based on a generative adversarial network, including a generator and a discriminator trained adversarially. Step S3 specifically includes: S31. The generator receives a random noise vector, the portrait image, and the target jewelry information, and generates multiple candidate virtual wearing effect images; S32. The discriminator compares and evaluates the candidate image with the real image of the jewelry being worn, and outputs the confidence score. S33. Filter candidate images with confidence levels ≥ preset thresholds to determine the final virtual wearing effect image.

7. The AI-powered virtual jewelry wearing method according to claim 5, characterized in that, The AI-powered virtual jewelry wearing system also includes a posture guidance module, and prior to step S1, it further includes: S0. The posture guidance module detects the user's posture, and triggers the image acquisition module to acquire images when the posture meets the preset conditions.

8. The AI-powered virtual jewelry wearing method according to claim 5, characterized in that, The AI ​​jewelry virtual wearing system also includes a preprocessing module, and step S1 further includes: S11. The preprocessing module performs key point detection on the portrait image to locate at least one jewelry wearing area; S12. The image of the wearing area is standardized by the preprocessing module. The standardization process includes at least one of size normalization, illumination equalization, and background simplification.

9. The AI-powered virtual jewelry wearing method according to claim 5, characterized in that, The target jewelry information includes at least one of the jewelry outline mask image and the product image, wherein the outline mask image is extracted from the image of the target jewelry selected by the user through image segmentation technology.

10. The AI-powered virtual jewelry wearing method according to claim 6, characterized in that, The adversarial training process for the generator and discriminator includes the following steps performed alternately: First training phase: Fix generator parameters, optimize discriminator, and maximize discriminator's ability to distinguish between real image x and generated image G(z) by using loss function LD=-[log(D(x))+log(1-D(G(z)))]; The second training phase involves fixing the discriminator parameters and optimizing the generator. The realism of the generated images is improved by using the loss function LG=-log(D(G(z))), making the confidence level of the discriminator output close to 1.

Images