Personalized clothing virtual try-on method and system, terminal and storage medium

Abstract

The invention relates to the technical field of virtual reality, and discloses a personalized clothing virtual try-on method and system, a terminal and a storage medium. The method comprises the following steps: S1, acquiring a three-dimensional body model of a user, and constructing a high-precision three-dimensional body model by analyzing a multi-angle picture uploaded by the user and utilizing a deep learning technology; s2, according to the three-dimensional body model and personal attributes of the user, performing personalized adjustment on the virtual clothes, including color, size and material; and S3, superposing the adjusted virtual costume on the three-dimensional body model of the user by using an augmented reality technology. Through combination of multi-angle image analysis and a deep learning three-dimensional reconstruction technology, a high-precision human body modeling effect is realized. The purpose that the user can quickly generate the three-dimensional body model only through the daily photos is achieved. Compared with a modeling method depending on a depth camera or somatosensory equipment in the prior art, the user operation process is remarkably simplified, and the problems that a traditional scheme is high in threshold and limited in application scene are solved.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and specifically to a method, system, terminal and storage medium for virtual try-on of personalized clothing. Background Art

[0002] With the development of Internet technology, e-commerce has gradually become an important way for people's daily shopping. In the clothing purchase scenario, consumers are increasingly inclined to online shopping. However, traditional online shopping lacks the try-on experience, resulting in a relatively high return rate after purchase. To improve this situation, virtual fitting technology has been introduced in the prior art. Through technologies such as image processing and human face recognition, the user's body model is combined with virtual clothing to achieve a virtual try-on experience. However, the existing virtual fitting technologies generally have the following problems: First, the matching accuracy is not high, and the display effect of virtual clothing on the user's body model is not realistic; second, the degree of personalization is insufficient, and it cannot be accurately adjusted according to personal attributes such as the user's body characteristics and skin color; third, the interaction experience is poor, and the operations during the virtual try-on process are complex for users, affecting the user experience. The prior art usually adopts a virtual try-on method based on 2D images. By using the photos uploaded by the user or the images captured in real time by the camera, the user's body contour is combined with virtual clothing using image processing algorithms. The virtual try-on method based on 2D images has limitations in dealing with human body posture changes, clothing texture mapping, etc., resulting in an unrealistic virtual try-on effect and a poor user experience.

[0003] Currently, the adjustment logic of virtual clothing is generally rather rigid. The color is just a texture change, the size scaling is a whole stretch, and the material change only stays at visual camouflage. It seems to have changed, but in fact it's of little use. There is a lack of linked modeling based on the user's real body data, and the physical properties' influence on the try-on dynamics is not considered, so the experience is naturally not realistic. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a method, system, terminal and storage medium for virtual try-on of personalized clothing, which solves the problems of low personalization degree, poor dynamic authenticity and lack of user interaction reference in the existing virtual try-on technology.

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: A method for virtual try-on of personalized clothing, including the following steps; S1. Obtain the user's three-dimensional body model. By analyzing the multi-angle photos uploaded by the user, use deep learning technology to construct a high-precision three-dimensional body model; S2. According to the user's three-dimensional body model and personal attributes, perform personalized adjustments on the virtual clothing, including color, size, and material; S3. Use augmented reality technology to overlay the adjusted virtual clothing on the user's 3D body model to achieve a real-time virtual fitting experience; S4. During the virtual fitting process, use a physics engine to simulate the dynamic effects of the clothing so that the virtual clothing swings naturally during the user's movement; S5. Use emotion recognition technology to automatically adjust the atmosphere of the virtual environment according to the user's emotional state to enhance the fitting experience; S6. During the virtual fitting process, the user can customize the design by adjusting the style, pattern, and material of the virtual clothing, and ensure the virtual fitting effect through 3D reconstruction and physical simulation.

[0006] Preferably, the specific steps in step S1 include the following steps; The user uploads at least three full-body photos from different angles; Perform image enhancement processing on the uploaded photos to improve the contrast of the images and remove the noise generated during shooting; Use a convolutional neural network to detect key points of the uploaded photos to obtain the two-dimensional coordinates of the body key points; According to the positions of the feature points of the photos from different angles, use the principle of triangulation to convert the two-dimensional feature points into three-dimensional coordinates to construct the user's 3D body model.

[0007] Preferably, the specific steps in step S2 include the following steps; Convert the color of the virtual clothing from the RGB color space to the HSV color space, and adjust the hue, saturation, and lightness according to the user's needs; According to the user's 3D body model and the original size of the virtual clothing, use linear interpolation and non-linear deformation algorithms to adjust the size of the virtual clothing to ensure that the clothing can match the user's body; Through physical rendering technology, adjust the glossiness and roughness parameters of the virtual clothing according to the material type selected by the user.

[0008] Preferably, the steps of the physics engine in step S4 include; According to the material and structure parameters of the virtual clothing, use the physics engine to calculate the morphological changes of the virtual clothing under the user's dynamic movement in real time; Simulate the physical properties of the weight, elasticity, and drapability of the virtual clothing through a particle spring model to make the dynamic effects of the virtual clothing more natural and smooth.

[0009] Preferably, the emotion recognition technology in step S5 includes: Identify the user's emotional state by analyzing the user's facial expressions and speech intonations; Adjust the background music and lighting in the virtual environment according to the user's emotional state to enhance the immersion and user experience of the virtual fitting.

[0010] Preferably, when performing feature point matching based on the triangulation principle, the following formula is used: P i (X i ,Y i ,Z i ) = f(p1, p2, p3); Where: P i (X i ,Y i ,Z i ) represents the coordinates of the i-th body feature point of the user in three-dimensional space, where X i is the X coordinate of this point in three-dimensional space, Y i is the Y coordinate of this point in three-dimensional space, and Z i is the Z coordinate of this point in three-dimensional space; f(p1, p2, p3) represents a mapping function from two-dimensional feature points to three-dimensional coordinates; p1, p2, p3 represent three two-dimensional feature point data used for triangulation.

[0011] Preferably, the key measurement parameters of the three-dimensional body model are compared with the corresponding dimensions of the virtual clothing model to calculate the size adjustment ratio, and the virtual clothing model is scaled according to the following formula: Where: R is the size adjustment ratio; M u is the measurement size of the corresponding part of the user; M g is the original size of the virtual clothing at this part; The key vertex coordinates in the three-dimensional mesh model of the virtual clothing are transformed by applying the size adjustment ratio R to make the virtual clothing fit the three-dimensional body model of the user.

[0012] The personalized clothing virtual try-on system includes: A data collection module for collecting the user's body information and preference data; A three-dimensional modeling module, connected to the data collection module, which constructs the user's three-dimensional body model using deep learning technology; A personalized adjustment module, connected to the three-dimensional modeling module, which adjusts the color, size, and material of the virtual clothing according to the user's body data; A virtual try-on module, connected to the personalized adjustment module, which uses augmented reality technology to superimpose the adjusted virtual clothing on the user's three-dimensional body model and display the virtual try-on effect in real time; The physical engine module, together with the virtual try-on module, simulates the dynamic effect of the natural swing of virtual clothing during the user's movement; the emotion recognition module, together with the virtual try-on module, identifies the user's emotional state and adjusts the atmosphere of the virtual environment to enhance the user experience.

[0013] Personalized clothing virtual try-on terminal, including; A processor, used to execute various steps of the personalized clothing virtual try-on method; A storage module, used to store the user's body data, virtual clothing information, and program codes required for system operation; A display module is used to display the user's virtual try-on effect; The input module is used to receive user photo uploads, personalization adjustments and try-on operations.

[0014] A storage medium stores computer-executable instructions, which, when executed in a processor, enable the processor to execute the personalized clothing virtual try-on method.

[0015] The present invention provides a personalized clothing virtual try-on method, system, terminal and storage medium. It has the following beneficial effects: 1. The present invention achieves high-precision human body modeling by combining multi-angle image analysis with deep learning 3D reconstruction technology. It achieves the goal that users can quickly generate a complete 3D body model based on daily photos. Compared with the existing modeling methods that rely on depth cameras or somatosensory devices, it significantly simplifies the user operation process and solves the problems of high threshold and limited application scenarios of traditional solutions.

[0016] 2. The present invention adopts a joint personalized adjustment strategy based on HSV color space adjustment, size interpolation fitting and PBR material mapping, so that virtual clothing can accurately match the structural characteristics of the user's body model, and finally present a more natural and fitting visual effect. Different from the traditional method of stretching and deforming by fixed templates, this solution effectively solves the problem of virtual clothing deformation and distortion and unrealistic material performance.

[0017] 3. Through the user's emotional perception and the dynamic adjustment mechanism of the virtual environment, the system can automatically identify the user's facial expression or voice status and synchronously adjust the background light and sound atmosphere to enhance the sense of immersion. Compared with the existing fitting system that only relies on visual simulation, the present invention realizes the leap from "seeing" to "feeling", solving the problem of the lack of interactive emotional feedback in the existing technology.

[0018] 4. The present invention provides a clothing customization function during the fitting process, including style reconstruction, pattern overlay, and material replacement, and ensures the stability of its dynamic presentation based on local three-dimensional reconstruction and physical simulation. This is hardly achievable in traditional systems. Existing technologies are often limited to static replacement of preset models, while this solution solves the problems of easy intersection and lag in dynamic editing, truly integrating personalized design and fitting experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flowchart of the method of the present invention; Figure 2 is a system framework diagram of the present invention; Figure 3 is a schematic diagram of the virtual fitting terminal of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0021] Please refer to the attached Figure 1 , the embodiments of the present invention provide a method for virtual fitting of personalized clothing, including the following steps; S1. Obtain the three-dimensional body model of the user, and construct a high-precision three-dimensional body model by analyzing the multi-angle photos uploaded by the user using deep learning technology; Specifically, in this embodiment, to realize the functions of subsequent precise fitting and dynamic simulation presentation of virtual clothing, it is first necessary to construct a three-dimensional body model of the user himself. This three-dimensional model not only serves as the basic data for subsequent virtual clothing size adjustment and dynamic mapping, but also directly relates to the spatial registration accuracy of the personalized rendering and emotion recognition modules. Therefore, in the early stage of the virtual fitting process, it is necessary to model the original image information provided by the user and ensure that the modeling result has high reducibility and structural integrity.

[0022] In this embodiment, the user can upload full-body images of different angles of himself through terminal devices (such as mobile devices, wearable devices, desktop devices, etc.). Generally, it is recommended to provide at least three photos covering the front, left side, and back to meet the spatial baseline requirements of three-dimensional geometric reconstruction. As an option, the image can be a color image or a depth image, but preferably a high-definition color photo, and the image resolution should generally not be lower than 720×1280 to ensure the accuracy of subsequent feature point extraction.

[0023] After uploading the image, the system performs image preprocessing operations. In one possible implementation, the image preprocessing includes operations such as brightness normalization, image enhancement, edge smoothing, and noise removal. For example, image enhancement can be based on the histogram equalization algorithm to improve the distinguishability of dark details. The denoising operation can be implemented through algorithms such as Gaussian filtering and bilateral filtering to reduce the impact of image interference information on the judgment results of the neural network.

[0024] Subsequently, the system performs human key point recognition based on a deep learning model. Specifically, in this embodiment, a trained convolutional neural network (CNN) structure is used to model the human body posture in the image. The CNN model can refer to the OpenPose framework or the HRNet structure, which can identify the two-dimensional spatial positions of key nodes including the head, shoulders, elbows, wrists, hips, knees, and ankles. Here, the two-dimensional key point coordinates are represented as: p ij =(x ij ,y ij ); Where: p ij represents the position of the i-th key point of the user in the j-th image; x ij and y ij are the horizontal and vertical coordinates of this point on the image plane, respectively.

[0025] To further reconstruct the three-dimensional body model, it is necessary to map the two-dimensional key point information at multiple angles to the three-dimensional space through triangulation technology. Generally, an image pair with a good parallax baseline is selected as the input, and the position differences of the corresponding key points at different perspectives are used to estimate the depth information. In one implementation, the calculation of the three-dimensional point position P i can be based on the following mapping function: P i (X i ,Y i ,Z i )=f(p1,p2,p3); Where: P i (X i ,Y i ,Z i ) represents the coordinates of the i-th body feature point of the user in the three-dimensional space, X i is the X coordinate of this point in the three-dimensional space, Y i is the Y coordinate of this point in the three-dimensional space, and Z i is the Z coordinate of this point in the three-dimensional space; f(p1,p2,p3) represents a mapping function from two-dimensional feature points to three-dimensional coordinates; p1, p2, and p3 represent three two-dimensional feature point data for triangulation.

[0026] In some embodiments, the above mapping function can further combine the camera internal parameters and distortion parameters, and adopt structured light reconstruction, bundle adjustment, or the SFM algorithm based on sparse reconstruction to improve the spatial accuracy of the three-dimensional point cloud fitting.

[0027] After the three-dimensional point reconstruction is completed, the system performs a surfacing process on the point cloud through a triangular mesh construction algorithm to generate a connected three-dimensional body surface model. This model has a compact structure and continuous topology and can be directly used for the subsequent clothing simulation matching process.

[0028] S2. According to the user's three-dimensional body model and personal attributes, perform personalized adjustments on the virtual clothing, including color, size, and material; Specifically, in this embodiment, after the construction of the three-dimensional body model is completed, to ensure the accurate spatial matching relationship between the virtual clothing and the user's body and enhance the user's personalized experience, the system needs to adjust the parameters of the virtual clothing according to the three-dimensional model. This step not only plays a key role in connecting subsequent augmented reality overlay, physical simulation, etc., but also directly affects the spatial coupling accuracy and visual consistency during the fitting process.

[0029] Generally, the virtual clothing has standard model parameters in the initial state, usually the general human template size, the default color scheme, and the basic material attributes. Therefore, it is necessary to perform dynamic adjustment operations by comparing and analyzing the structural dimensions of the user's three-dimensional body model and the original data of the clothing model. This adjustment process includes but is not limited to the personalized setting of the clothing color, the scaling and deformation of the size, and the adaptation of the material perception parameters.

[0030] In this embodiment, the system first performs a color space conversion operation on the virtual clothing model selected by the user. Specifically, the original clothing color is generally defined in the RGB space, that is, it consists of three channels: Red, Green, and Blue. However, directly performing color operations in the RGB space is likely to cause non-linear color deviation. Therefore, it is preferably to convert the color information to the HSV color space. Among them, HSV is a combination of Hue, Saturation, and Value.

[0031] In a possible implementation manner, the HSV adjustment function of the clothing color can be expressed as follows: H′ = H + ΔH, S′ = S × α s , V′ = V × α v ; Where: H′, S′, and V′ are the adjusted hue, saturation, and value respectively; H, S, and V are the HSV values of the original virtual clothing color; ΔH represents the hue offset set by the user; α s , α v are the saturation adjustment factor and brightness adjustment factor set by the user, both being positive real numbers.

[0032] As an option, the user can set the above parameters by means of a sliding control or a voice command. The system performs a color mapping update on the clothing model to ensure that the color change takes effect in real time within the three-dimensional visual area.

[0033] After completing the color adjustment, the system further performs a size adjustment process on the virtual clothing model according to the structural parameters of the user's body model. Specifically, this process is based on the measurement results of the spatial distances between key human body parts and combines with the original corresponding sizes of the clothing model to calculate the scaling ratio.

[0034] In some embodiments, the size adjustment ratio R can be calculated according to the following formula: Where: R i represents the scaling ratio of the i-th measurement part; M u,i represents the actual size data of the user at this part (such as shoulder width, chest circumference, waist circumference, trouser length, etc.), and the unit can be centimeter (cm); M g,i represents the reference size of the original virtual clothing model at this part; In a possible implementation, the system generates a spatial scaling vector according to the R of each key part i and performs a point-by-point coordinate transformation operation on the three-dimensional mesh vertices of the clothing model. The specific transformation process can be expressed as follows: V j ′ = R i ·V j , if V j ∈S i ; Where: V j ′ represents the transformed vertex coordinates, which are the new coordinates obtained after the scaling operation; R i represents the scaling matrix corresponding to the i-th part of the human body, which may scale in one or more directions in three-dimensional space; V j represents the original clothing mesh vertex coordinates, which are the values before the transformation; The "if" represents "if", which is a logical conditional judgment used to indicate that the subsequent transformation is only performed when a specific condition is met; V j ∈S i represents the vertex V j belongs to the clothing area S i , where S i is the clothing area corresponding to the i-th part of the human body.

[0035] This scaling operation can be performed uniaxially (such as only scaling the shoulder width in the X direction), or it can be performed jointly in three axes to achieve anisotropic scaling in space.

[0036] To ensure the continuity of deformation, the system will also perform transitional interpolation processing on the edge area to avoid clothing cracks or visual distortions caused by sudden scaling changes.

[0037] After the size adjustment is completed, the system performs adaptive processing of material properties based on the user's preference information and the original material data of the clothing. Specifically, each material block in the virtual clothing model is defined with a set of PBR (Physically Based Rendering) parameters. This parameter set usually includes metallicity, roughness, reflectance, and normal map, etc.

[0038] S3. Use augmented reality technology to superimpose the adjusted virtual clothing on the user's three-dimensional body model to achieve a real-time virtual try-on experience; Specifically, after completing the personalized adjustment of the virtual clothing, to further enhance the spatial realism and dynamic interactivity of the virtual wearing experience, this step needs to map and bind the adjusted virtual clothing model to the user's three-dimensional body model. This mapping process is a crucial core link in virtual try-on, and its goal is to achieve precise matching between the virtual clothing and the user's body surface in three-dimensional space, ensuring that the clothing model has position continuity, topological consistency, and motion responsiveness in subsequent interactions.

[0039] Generally, the virtual clothing model already includes a bone binding structure or a control right reorganization available for skinning during initial construction. In this embodiment, the system pairs the point cloud information of the user's three-dimensional body model with the clothing model mesh, and assigns the binding position on the user's body model to each clothing mesh vertex, thereby achieving a one-to-one mapping in space.

[0040] In this embodiment, the spatial mapping relationship between the clothing and the body model is represented by the weight function W ij represents; Among them: W ij To represent the influence weight of the i-th human body model point corresponding to the j-th clothing mesh vertex e is the base of the natural logarithm (Euler's constant); ||P j -Q i || 2 is the square of the Euclidean distance between the clothing mesh vertex P j and the user's body point Q i The farther the distance, the larger the value; ||P k -Q i || 2 represents the square of the Euclidean distance between the clothing vertex P k and the human body point Q i The larger the distance, the smaller this term; / σ 2 is the distance square divided by the variance σ 2 to control the attenuation rate of the Gaussian function; k is the summation index variable.

[0041] Specifically, the above weight function embodies a soft binding strategy based on Gaussian attenuation. As an option, this method can replace the traditional rigid pairing method, effectively reducing the mesh penetration or distortion caused by movement or pose changes.

[0042] To further enhance the stability of the mapping, the system performs topological region division on the body model and the clothing model before binding. In one possible implementation, the user's body model can be divided into regions such as the head, torso, upper limbs, and lower limbs, and each region is fitted using a local coordinate system. The material patches of each part in the clothing model are also labeled to the corresponding regions, so that the mapping is only performed within the local regions, thereby reducing the interference between non-related parts.

[0043] In some embodiments, the system also introduces a bone driving mechanism. Specifically, the virtual clothing is bound to the bone structure generated based on pose estimation in the user's body model, and the Linear Blend Skinning (LBS) algorithm is used to achieve real-time action response. The vertex transformation formula in the LBS algorithm is as follows: Among them, V j ′ represents the coordinate of the j-th vertex in the clothing model after pose transformation; V j is the initial vertex position; w jk is the weight of the j-th vertex and the k-th bone; Tk Represents the transformation matrix of the k-th bone, including rotation and translation components; n is the number of bound bones.

[0044] This method allows the system to automatically update the position and deformation state of the clothing mesh when the user makes pose adjustments or movements, thus achieving synchronous response of the clothing to the user's actions.

[0045] As a supplementary technical path, the system can be optionally equipped with a constraint mechanism based on physical simulation, such as elastic constraints, collision detection, etc., to further enhance the dynamic realism of virtual clothing in terms of fit and compliance. At this time, the system needs to define physical parameters such as mass, damping, and elastic coefficient for each binding point to achieve cloth dynamics drive.

[0046] S4. During the virtual fitting process, use a physics engine to simulate the dynamic effects of the clothing, so that the virtual clothing swings naturally during the user's movement; Specifically, in this embodiment, after the personalized adjustment of the virtual clothing model and the spatial binding of the user's body model are completed, in order to further achieve user visual interaction and realistic fitting feedback, the system needs to enter the rendering and output processing stage. The main goal of this step is to present the aforementioned mapping fusion result to the user in the form of an image in real time and achieve dynamic response update to the user's actions. As the final visual layer output, this link is directly related to the immersion, interactivity, and image quality of the user's fitting experience, and is a key end module in the technical process of the present invention.

[0047] Generally, a three-dimensional rendering system relies on a graphics processing unit (GPU) for parallel computing. The system needs to perform operations such as unified view transformation, lighting simulation, texture mapping, and pixel shading on the clothing model and the user's body model, so as to generate a sequence of two-dimensional images for display on the terminal device.

[0048] In this embodiment, the system introduces an image generation engine based on the PBR (Physically Based Rendering) mechanism to perform high-precision modeling on the clothing material and lighting. Specifically, after the clothing model is bound to the user's body, each patch area of it will contain a set of material properties, defined as follows: M = {ρ d , ρ s , r, n, T, N}; Where: ρ d Represents the diffuse albedo; ρ s Represents the specular albedo; r is the surface roughness; n is the normal direction vector; T is the texture map image; N is the normal perturbation map image.

[0049] As an option, the system can set specific material parameter templates according to different types of clothing materials (such as knitted cotton, silk, leather, etc.), and call the corresponding parameter sets according to user options.

[0050] In a possible implementation, the lighting model in the rendering process adopts an improved Cook-Torrance model, and its specular reflection component L s can be expressed as: where: F(θ) is the Fresnel reflection function; D(h) is the micro-surface normal distribution function G(v, l) is the geometric shadowing function n·v and n·l are the cosines of the angles between the normal and the viewing direction, and the normal and the light direction respectively.

[0051] The above parameters are all based on the local coordinate system and are calculated pixel by pixel in the GPU fragment shader to obtain a more realistic lighting and material response.

[0052] Specifically, to achieve dynamic update of the user's actions during the try-on process, the system introduces a pose-driven mechanism in the rendering module. This mechanism is based on the bone information generated during the binding phase, and updates the clothing vertex positions in real time when the user's pose changes, and re-triggers the rendering process.

[0053] In some embodiments, the terminal device uses an OpenGL or Vulkan rendering framework, combined with real-time depth camera input data or an IMU pose perception module, to achieve frame-level action synchronization response. The generation process of each frame of image includes the combined action of the view transformation matrix V, the projection matrix P, and the model matrix M: x screen = P·V·M·x model ; where: x model is the homogeneous coordinate point in the model space; x screen is the point finally projected onto the screen coordinate system.

[0054] To optimize the system response efficiency, in some implementations, the system adopts a deferred rendering architecture, separating the lighting calculation to be executed after the G-buffer, thereby improving the performance in the case of multiple light sources.

[0055] As a supplementary solution, the system can also add a background synthesis module during the output stage. According to the input of the user's current environmental background image or video stream, this module performs foreground matting and background synthesis operations, so that the virtual try-on effect is naturally integrated into the real scene. This process is often completed by combining a semantic segmentation model and an Alpha matte generation network.

[0056] In addition, in some embodiments, the system provides multi-angle view switching and magnified viewing functions, allowing users to rotate the model, zoom in to view the detailed texture of the clothing or the material of the stitching.

[0057] S5. Utilize emotion recognition technology to automatically adjust the atmosphere of the virtual environment according to the user's emotional state, enhancing the try-on experience; Specifically, in this embodiment, after the rendering output of the virtual clothing is completed, in order to achieve personalized feedback and system adaptation optimization during the user interaction process, the system needs to collect, analyze and process the user's operation behaviors and experience results. This step not only provides a data basis for subsequent recommendation engines, interactive interface optimization, and clothing parameter adjustment, but also helps to establish a user preference model to achieve the dynamic evolution of the virtual try-on experience.

[0058] Generally, during the try-on process, the user will have various types of interactions with the system, including but not limited to view rotation, clothing replacement, color switching, zooming operations, pose simulation, screenshot saving, etc. The system needs to completely record the above operation events, and combine parameters such as the system response time, rendering frame rate, and model adaptation status in the rendering output to construct a comprehensive interaction dataset.

[0059] In this embodiment, the system introduces a behavior log module to record the user's operation sequence. Based on the event listening mechanism, this module captures all the input behaviors of the user in the front-end interface in real time and forms an operation trajectory through timestamp serialization. The operation data structure is defined as follows: O i =<t i ,a i ,p i >; Among them, O i represents the i-th operation record; t i is the operation timestamp; a i is the operation type; p i is the operation attached parameter.

[0060] In a possible implementation, every time the user completes a virtual try-on process, the system will automatically generate a complete operation log sequence {O1, O2,..., O n}, and stored in the local cache or cloud database for subsequent analysis.

[0061] Specifically, the system also evaluates the fitting feedback during the user's try-on process. As an option, the system guides the user to subjectively rate the current try-on effect, and is supplemented with a visual annotation module that allows the user to circle problem areas or mark points where the clothing does not fit well. This feedback data is structured and saved in the following form: F j = <s j ,l j ,c j >; Where: F j represents the j-th feedback record; s j is the scoring result; l j is the position coordinate (x, y, z) of the feedback part on the 3D model; c j is the feedback classification.

[0062] In some embodiments, to enhance the feedback perception ability of the system, the system also collects real-time available sensing data in the user's device, such as camera images, voice commands, facial expression recognition, click heat zones, etc., and enhances the accuracy of user intention recognition through multi-modal data fusion technology.

[0063] As a supplementary mechanism, the system can also automatically infer preference tendencies based on operation frequency, dwell time, and user behavior characteristics. For example, when the user stays on a certain type of clothing (such as sports style) for a long time and tries it on frequently, the system can automatically mark this category as the user's preferred clothing type and increase its display priority in the next interaction.

[0064] Through the user behavior collection and feedback analysis mechanism described in this step, the system can establish a complete closed-loop of interaction data, provide an accurate dynamic adjustment basis for the personalized virtual try-on system, and promote the continuous optimization of the model and recommendation strategy, improving the personality adaptability and data availability of the try-on experience.

[0065] S6. During the virtual try-on process, the user customizes the design by adjusting the style, pattern, and material of the virtual clothing, and ensures the virtual try-on effect through 3D reconstruction and physical simulation; Specifically, in this embodiment, after completing the virtual try-on display (step S3) and dynamic simulation (step S4) based on augmented reality technology, to further improve the user's interaction participation and personalized experience, the system provides a visual editing function that allows the user to autonomously define and real-time adjust the style, pattern, and material of the virtual clothing worn during the try-on process.

[0066] First, in this embodiment, the system integrates a clothing customization module in the user's fitting interface. This module can automatically load the model structure and material configuration of the current virtual clothing after being activated by the user. The user can adjust the following elements in real time through the interaction interface: In some embodiments, style customization allows the user to modify key shape parameters such as sleeve length, dress length, neckline depth, skirt amplitude, etc. A set of style control parameters is preset in the system, and the corresponding clothing mesh is locally adjusted through a geometric transformation algorithm. The adjusted style model still retains the topological matching relationship with the user's three-dimensional body model, avoiding problems such as model deviation or fitting distortion.

[0067] As an option, after the clothing structure changes, the system will call the local three-dimensional reconstruction module to refit and optimize the triangular mesh of the modified area to ensure structural continuity and simulation stability. The reconstruction result supports hot update, that is, the visual effect can be updated in real time without reloading the overall clothing model.

[0068] Secondly, the pattern customization function supports the user to add, replace, or edit pattern textures on the clothing surface. The user can select the system-preinstalled patterns or upload custom pictures, and the system maps the patterns based on the UV mapping coordinates. Specifically, the system first extracts the UV unfolding structure of the current clothing model, and after performing an affine transformation on the pattern, fuses it into the material texture map.

[0069] In a possible implementation, pattern fusion uses a simple linear weighting model: T new = α · T base +(1 - α) · T user ; Where: T new represents the fused texture image; T base is the original texture map of the clothing; T user is the pattern image provided by the user; α represents the fusion weight, which can be adjusted by the user.

[0070] In terms of material customization, the user can select various predefined material types (such as cotton, silk, denim, leather, etc.), and the system automatically adjusts the visual parameters of the clothing such as lighting, roughness, and reflectivity, and synchronously updates the material properties for physical simulation to ensure the accuracy of the dynamic simulation effect.

[0071] Please refer to Appendix Figure 1 - Appendix Figure 3 , the personalized clothing virtual fitting system includes; A data acquisition module for collecting users' physical information and preference data; A 3D modeling module, connected to the data acquisition module, which constructs a 3D body model of the user using deep learning technology; A personalized adjustment module, connected to the 3D modeling module, which adjusts the color, size, and material of virtual clothing according to the user's body data; A virtual try-on module, connected to the personalized adjustment module, which uses augmented reality technology to superimpose the adjusted virtual clothing on the user's 3D body model and display the virtual try-on effect in real time; A physical engine module, connected to the virtual try-on module, which simulates the dynamic effect of natural swinging of virtual clothing during the user's movement; An emotion recognition module, connected to the virtual try-on module, which recognizes the user's emotional state and adjusts the atmosphere of the virtual environment to enhance the user experience.

[0072] Please refer to the appendix Figure 1 - Appendix Figure 3 A personalized clothing virtual try-on terminal, including; A processor for executing each step of the personalized clothing virtual try-on method; A storage module for storing the user's body data, virtual clothing information, and program code required for system operation; A display module for displaying the user's virtual try-on effect; An input module for receiving the user's photo upload, personalized adjustment, and try-on operations.

[0073] Please refer to the appendix Figure 1 - Appendix Figure 3 A storage medium storing computer-executable instructions, which, when executed by a processor, cause the processor to execute the personalized clothing virtual try-on method.

[0074] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A personalized clothing virtual try-on method, characterized in that, It includes the following steps: S1. Obtain the user's three-dimensional body model. By analyzing the multi-angle photos uploaded by the user, use deep learning technology to construct a high-precision three-dimensional body model; S2. According to the user's three-dimensional body model and personal attributes, perform personalized adjustments on the virtual clothing, including color, size, and material; S3. Use augmented reality technology to superimpose the adjusted virtual clothing on the user's three-dimensional body model to achieve a real-time virtual try-on experience; S4. During the virtual try-on process, use a physics engine to simulate the dynamic effects of the clothing, so that the virtual clothing swings naturally during the user's movement; S5. Use emotion recognition technology to automatically adjust the atmosphere of the virtual environment according to the user's emotional state to enhance the try-on experience; S6. During the virtual try-on process, the user can customize the design by adjusting the style, pattern, and material of the virtual clothing, and ensure the virtual try-on effect through three-dimensional reconstruction and physical simulation.

2. The personalized clothing virtual try-on method according to claim 1, wherein, Specifically in step S1 It includes the following steps; The user uploads at least three full-body photos from different angles; Perform image enhancement processing on the uploaded photos to improve the contrast of the images and remove the noise generated during shooting; Use a convolutional neural network to perform key point detection on the uploaded photos to obtain the two-dimensional coordinates of the body key points; According to the positions of the feature points in the photos from different angles, use the principle of triangulation to convert the two-dimensional feature points into three-dimensional coordinates to construct the user's three-dimensional body model.

3. The personalized clothing virtual try-on method according to claim 1, characterized in that Specifically in step S2, it includes the following steps; Convert the color of the virtual clothing from the RGB color space to the HSV color space, and adjust the hue, saturation, and lightness according to the user's needs; According to the user's three-dimensional body model and the original size of the virtual clothing, use linear interpolation and non-linear deformation algorithms to adjust the size of the virtual clothing to ensure that the clothing can match the user's body; Through physical rendering technology, adjust the glossiness and roughness parameters of the virtual clothing according to the material type selected by the user.

4. The personalized clothing virtual try-on method according to claim 1, characterized in that, The steps of the physics engine in step S4 include; According to the material and structure parameters of the virtual clothing, use the physics engine to calculate the morphological changes of the virtual clothing under the user's dynamic movement in real time; Simulate the physical properties of the weight, elasticity, and drapability of the virtual clothing through the particle spring model to make the dynamic effect of the virtual clothing more natural and smooth.

5. The personalized clothing virtual try-on method according to claim 1, characterized in that The emotion recognition technology in step S5 includes: By analyzing the user's facial expressions and speech intonations, recognize the user's emotional state; According to the user's emotional state, adjust the background music and lighting in the virtual environment to enhance the immersion and user experience of the virtual try-on.

6. The personalized clothing virtual try-on method according to claim 2, characterized in that, When using the triangulation principle for feature point matching, use the following formula: P i (X i ,Y i ,Z i ) = f(p1, p2, p3); Where: P i (X i , Y i , Z i ) represents the coordinates of the i-th body feature point of the user in three-dimensional space, where X i is the X coordinate of this point in three-dimensional space, Y i is the Y coordinate of this point in three-dimensional space, and Z i is the Z coordinate of this point in three-dimensional space; f(p1, p2, p3) represents a mapping function from two-dimensional feature points to three-dimensional coordinates; p1, p2, p3 represent the data of three two-dimensional feature points used for triangulation.

7. The personalized clothing virtual try-on method according to claim 1, wherein Compare the key measurement parameters of the three-dimensional body model with the corresponding sizes of the virtual clothing model, calculate the size adjustment ratio, and scale the virtual clothing model according to the following formula: Where: R i represents the scaling ratio of the i-th measurement site; M u,i represents the actual dimension data of the user at this part, and the unit can be centimeter; M g,i represents the reference dimension of the virtual clothing original model at this part; Apply a scaling ratio R to the key vertex coordinates in the three-dimensional mesh model of the virtual clothing to adapt the virtual clothing to the user's three-dimensional body model.

8. A personalized clothing virtual try-on system, applied to the personalized clothing virtual try-on method according to any one of claims 1-7, characterized in that, Including; A data collection module for collecting the user's body information and preference data; A three-dimensional modeling module connected to the data collection module, which uses deep learning technology to construct the user's three-dimensional body model; A personalized adjustment module connected to the three-dimensional modeling module, which adjusts the color, size, and material of the virtual clothing according to the user's body data; A virtual try-on module connected to the personalized adjustment module, which uses augmented reality technology to superimpose the adjusted virtual clothing on the user's three-dimensional body model to display the virtual try-on effect in real time; A physics engine module connected to the virtual try-on module, which simulates the dynamic effect of the natural swing of the virtual clothing during the user's movement; An emotion recognition module connected to the virtual try-on module, which recognizes the user's emotional state and adjusts the atmosphere of the virtual environment to enhance the user experience.

9. A personalized clothing virtual try-on terminal, applied to the personalized clothing virtual try-on method according to any one of claims 1-7, characterized in that Including; A processor for executing each step of the personalized clothing virtual try-on method; A storage module for storing the user's body data, virtual clothing information, and program code required for system operation; A display module for displaying the user's virtual try-on effect; An input module for receiving the user's photo upload, personalized adjustment, and try-on operations.

10. A storage medium, characterized in that, Stores computer-executable instructions that, when executed by the processor, cause the processor to execute the personalized clothing virtual try-on method as claimed in claims 1-7.

Images