Virtual fitting system and fitting method based on deep reinforcement learning

Abstract

The invention relates to a virtual try-on system and method based on deep reinforcement learning, and the system comprises a body type recognition and reconstruction module, a material simulation module, an adaptive optimizer, and a rendering output module. The material simulation module is used for carrying out micro-approximate simulation on virtual fitting garment fabric, the self-adaptive optimizer is used for automatically adjusting an optimization strategy according to different user characteristics, and the rendering output module is used for rendering and generating a fitting garment result. According to the design, the problem that the fitting matching degree is low due to the fact that the personalized adaptation capacity is poor and self-adaptive optimization cannot be carried out according to the body shape features and preferences of different users is effectively solved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, and in particular to a virtual try-on system and method based on deep reinforcement learning. Background Technology

[0002] With the development of the times, online shopping has rapidly developed globally. Among the many categories of goods available for purchase on online platforms, clothing is one of the most frequently purchased items. However, the rapid development of online shopping has also brought many concerns to consumers. Issues such as ill-fitting sizes and discrepancies between the clothing and its pictures, leading to returns and exchanges, often trouble both consumers and merchants. Virtual try-on technology has thus emerged to address this need. Traditional virtual try-on methods based on 2D image transformation lack physical simulation of clothing materials, failing to accurately represent the drape, stretch characteristics, and luster of fabrics, resulting in distorted try-on effects. Therefore, a new technology is needed. For example, patent CN114663199A, entitled "A Dynamically Displayed Real-Time 3D Virtual Try-on System and Method," discloses a dynamically displayed real-time 3D virtual try-on system and method. This system divides digital human body modeling into two parts: deep learning-based facial modeling and database-based model selection. This significantly reduces the time required for human body modeling and maximizes the reproduction of the body and facial features of the person trying on the clothes, making the system more realistic. Simultaneously, parametric modeling can greatly protect user privacy.

[0003] However, the aforementioned technical solutions lack personalization capabilities and cannot adaptively optimize based on different users' body shape characteristics and preferences, resulting in low fitting accuracy. Most existing technologies have limited deformation handling capabilities, easily exhibiting phenomena such as self-piercing (interpenetration of clothing mesh) and excessive stretching when dealing with complex clothing deformations, which violate physical laws. They also lack user feedback mechanisms, failing to effectively integrate users' subjective preference feedback and hindering continuous performance improvement. Therefore, it is necessary to develop a technology that employs newer design techniques, possesses high physical realism, and can adaptively optimize based on different users' body shape characteristics and preferences to solve these problems. Summary of the Invention

[0004] The purpose of this invention is to provide a virtual try-on system and method based on deep reinforcement learning, so as to solve the problems of insufficient physical realism, poor personalized adaptation ability, limited deformation handling ability, and lack of user feedback mechanism in the above-mentioned background technology.

[0005] This invention provides the following technical solution: a virtual try-on system based on deep reinforcement learning, comprising a body shape recognition and reconstruction module, a material simulation module, an adaptive optimizer, and a rendering output module. The body shape recognition and reconstruction module is used to collect human body shape parameters. The material simulation module is used to perform differentiable approximate simulation of virtual try-on clothing fabrics. The adaptive optimizer is used to automatically adjust the optimization strategy based on different user characteristics. The rendering output module is used to render the results of the clothing try-on.

[0006] Preferably, a virtual try-on method based on a deep reinforcement learning-based system includes the following steps: S1. Body Shape Recognition and Reconstruction: Based on user-provided multi-view... Figure 2 In the 3D image sequence, through image preprocessing, 3D point cloud generation and processing, and parametric human body model regression, the user's 3D body features and posture information are accurately identified and reconstructed, generating the user's 3D point cloud data and key body parameters. S2. Material Simulation: Based on the material type of the clothing to be virtually tried on, retrieve and instantiate the corresponding physical property parameters from the pre-built material parameter database, and use a differentiable physical simulation system to simulate the physical behavior and deformation state of the virtual clothing under the action of gravity, collision force and damping force, and output quantified physical behavior parameters of the clothing. S3. Adaptive Optimization: Construct a quantile Q-learning algorithm based on deep reinforcement learning. The state space is the 3D point cloud data, the body shape parameters, the clothing posture of the virtual try-on clothing, the overlap measurement between the virtual try-on clothing and the user, and / or the material state of the virtual try-on clothing. The action space is the global rigid body transformation, local non-rigid body deformation, and / or dynamic adjustment of the material parameters of the virtual try-on clothing. The learning process is guided by a multi-objective reward function that integrates visual realism scores, physical constraints, and user preference feedback. The actions proposed by the safety filter mechanism are verified and corrected in real time. The algorithm learns and selects the optimal action to optimize the try-on effect of the virtual try-on clothing. S4. Rendering Output: The virtual try-on clothing model adjusted through the adaptive optimization steps is rendered differently with the 3D point cloud data to generate the final virtual try-on image or video. The parameter fine-tuning mechanism is triggered based on the quality assessment results of the no-reference image quality assessment model to optimize the rendering parameters to ensure the quality of the output image.

[0007] Preferably, the body shape recognition and reconstruction in step S1 includes the following steps: S11. Multi-view Image Acquisition: Acquires frontal, side, and multi-angle images or RGB images of the user. S12. 3D Point Cloud Generation: A depth estimation network is used to predict the depth information of each pixel in the input image, generating a corresponding depth map. Combined with camera intrinsic parameters, the depth map is converted into 3D point cloud data. Noise reduction and completion processing are then performed on the 3D point cloud data. S13. SMPL-X Parameter Regression: A neural network regressor is used to extract the shape parameter β and pose parameter θ of SMPL-X from the 3D point cloud data. The shape parameter β controls the body shape of the body shape parameter, and the pose parameter θ controls the joint angles of the body shape parameter. The output body shape parameter includes height, chest circumference, waist circumference, and hip circumference. S14. Calculation of surface contact probability map: Based on the three-dimensional point cloud data, calculate the probability that each surface point may come into contact with the clothing.

[0008] Preferably, the material simulation in step S2 includes the following steps: S21. Fabric property modeling and material parameter database construction: Construct a material parameter database, wherein the material parameters include elastic modulus, Poisson's ratio, shear modulus, bending stiffness, density, coefficient of friction, damping coefficient, and thickness. Input the fabric properties of the virtual try-on garment, and retrieve the material parameters corresponding to the fabric properties of the virtual try-on garment from the material parameter database. S22. Differentiable physical simulation: A differentiable fabric simulation framework based on a mass-spring model is adopted. The three-dimensional mesh of the virtual try-on garment is discretized into a series of mass points and connected by structural springs, shear springs and bending springs. Under the action of external forces such as gravity, human-garment collision force and garment self-collision force, the simulation framework uses the numerical method of implicit Euler integral for time stepping and ensures that all physical parameters and state variables in the simulation process are differentiable in the entire simulation calculation graph. S23. Key Parameter Output: After the differentiable physical simulation, a series of key parameters that quantify the physical behavior and visual characteristics of the garment are output, including drape parameters, stretch parameters, and gloss parameters.

[0009] Preferably, the clothing posture includes the grid vertex coordinates and normal vector of the virtual try-on clothing, the overlap metric includes the distance field between the user's body surface and the virtual try-on clothing, the puncture detection result of the virtual try-on clothing, and the material state includes the deformation degree of the virtual try-on clothing and the stress distribution of the virtual try-on clothing.

[0010] Preferably, the multi-objective reward function is: R(s,a)=α·R_visual+β·R_physics+γ·R_preference; Where α, β, and γ are positive weighting coefficients used to balance the importance of different reward items, R_visual is the visual realism score, R_physics is the physical constraint term, and R_preference is the user preference feedback.

[0011] Preferably, the security filter mechanism includes real-time verification of the action after the agent selects an action but before actually executing the action, and correction or rejection when necessary. The verification and correction strategy includes: S301. Maximum stretching ratio limit for virtual try-on clothing: Evaluate the local stretching ratio of the clothing mesh that may be caused by the action. If the stretching ratio of any edge exceeds the preset material elastic limit, the action is corrected and the relevant vertex displacement vector or deformation gradient parameter is limited so that it does not exceed the safety threshold. S302. Minimum safe distance maintenance: The minimum distance between clothing and human body is monitored using a body surface contact probability map and a signed distance function. If the predicted minimum distance is less than a preset safety threshold, the action is corrected by applying additional repulsive force in the contact area or adjusting the local vertex displacement to ensure that clothing and body are kept above the minimum safe distance. S303. Self-puncture detection and prevention: Real-time monitoring of self-puncture within the virtual try-on clothing mesh; if any intersection or vertex penetration of a clothing mesh surface is detected, the action is corrected, triggering a local optimization process to eliminate puncture. The security filter mechanism feeds back the corrected action to the quantile Q-learning algorithm to update its strategy.

[0012] Preferably, the design of the motion space includes the following steps: S311. Global Rigid Body Transformation: Rotate and move the virtual try-on clothing in three-dimensional space. The rotation is parameterized by Euler angles, and the translation represents the movement of the clothing in the X, Y, and Z axis directions. S312. Local Non-rigid Deformation: Fine deformation adjustment is performed on local areas of the virtual try-on clothing mesh, including applying small three-dimensional displacement vectors to key vertices of specific areas. The displacement direction can be the normal direction or the tangent direction of the vertex. The deformation field of the local area of ​​the clothing is controlled by deformation gradient parameters. The deformation gradient parameters affect advanced deformation properties such as fold intensity, fold direction or fabric drape curvature.

[0013] Preferably, the quantile Q-learning algorithm operates through the following steps: S321. Modeling the reward distribution: The quantile Q-learning algorithm models the entire conditional distribution P(R|s,a) of the future cumulative reward that can be obtained by performing action a in a specific state s by learning a Q-value function Z(s,a), which is approximated by a set of discrete quantile points; S322. Maintaining multiple Q-value estimates: The quantile Q-learning algorithm maintains Q-value estimates at multiple quantile levels. For each quantile τ_i, a corresponding Q-value Z_i(s,a) is estimated such that P(Z(s,a)≤Z_i(s,a))=τ_i; S323. Quantile Regression Loss Function: During training, the agent uses the quantile regression loss function to update the quantile Q-value estimate. By optimizing this loss function, the quantile Q network can gradually learn the shape of the reward distribution, thereby capturing the uncertainty of the reward and supporting the agent in learning risk-sensitive policies.

[0014] Preferably, the rendering output in step S4 includes the following steps: S41. Differentiable rendering system: It adopts differentiable rasterization technology or differentiable volume rendering technology based on neural radiation field, so that all geometric parameters, material parameters and lighting parameters in the rendering process can be differentiated. The differentiable rendering system combines physical material model to realize physically correct lighting interaction and supports real-time calculation of shadow, reflection and refraction effects. S42. Ambient lighting estimation and fusion: From the background image provided by the user, the ambient lighting information of the scene is inferred and estimated using a deep learning network to generate a high dynamic range ambient lighting probe. During the differentiable rendering process, the ambient lighting probe is fused with the 3D model in the virtual scene to ensure that the virtual try-on clothing is visually highly matched with the lighting effect of the background environment. S43. Quality assessment and fine-tuning mechanism: A deep learning-based no-reference image quality assessment algorithm is used to assess the quality of the final rendered virtual try-on image. When the assessed image quality score is lower than a preset quality threshold, a parameter fine-tuning mechanism is triggered to automatically adjust the lighting parameters, material parameters and / or perform slight geometric fine-tuning in the renderer.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. The present invention provides a virtual try-on system based on deep reinforcement learning, comprising a body shape recognition and reconstruction module, a material simulation module, an adaptive optimizer, and a rendering output module. The body shape recognition and reconstruction module collects human body shape parameters; the material simulation module performs differentiable approximate simulation of the fabric of the virtual try-on clothing; the adaptive optimizer automatically adjusts the optimization strategy according to different user characteristics; and the rendering output module renders the result of the virtual try-on clothing. The body shape recognition and reconstruction module collects the user's body shape parameters to form 3D point cloud data; the material simulation module simulates the material of the virtual try-on clothing to significantly improve the physical realism of the virtual try-on experience; the adaptive optimizer automatically adjusts the optimization strategy according to different user characteristics; and finally, the rendering output module outputs the effect of the user's virtual try-on. This system can be widely applied to virtual try-on services on e-commerce platforms, digital prototype display of clothing designs, etc., effectively solving the problems of poor personalization adaptability and the inability to adaptively optimize according to different users' body shape characteristics and preferences, resulting in low try-on matching.

[0016] 2. The virtual try-on method of the deep reinforcement learning-based virtual try-on system of the present invention includes the following steps: S1. Body shape recognition and reconstruction, S2. Material simulation, S3. Adaptive optimization, and S4. Rendering output. The body shape recognition and reconstruction step collects the user's body shape parameters to form three-dimensional point cloud data. The material simulation step simulates the material of the virtual try-on clothing to significantly improve the physical realism of the virtual try-on experience. The adaptive optimization step automatically adjusts the optimization strategy according to different user characteristics. Finally, the rendering output module outputs the virtual try-on effect. This system can be widely used in virtual try-on services on e-commerce platforms, digital prototype display of clothing designs, etc., effectively solving the problem of poor personalization adaptability and the inability to adaptively optimize according to different users' body shape characteristics and preferences, resulting in low try-on matching.

[0017] 3. The virtual try-on method of the virtual try-on system based on deep reinforcement learning of the present invention includes the following steps in step S1: body shape recognition and reconstruction: S11. Multi-view image acquisition, S12. 3D point cloud generation, S13. SMPL-X parameter regression, and S14. Body surface contact probability map calculation. The multi-view image acquisition step accurately acquires the user's body shape parameters; the 3D point cloud generation step learns the potential representation of 3D shapes from sparse or incomplete point clouds and generates complete and smooth 3D point clouds, especially effective for filling in easily missing areas such as human joints; the SMPL-X parameter regression outputs the extracted body shape parameters in vector form and uses them as part of the state space for subsequent adaptive optimizer use; the body surface contact probability map calculation is used to guide subsequent collision detection and constraint optimization. In the physical simulation stage, high contact probability areas will be given higher collision detection priority or stronger repulsion force to more actively avoid clothing punctures into the human body.

[0018] 4. The virtual try-on system and method based on deep reinforcement learning of the present invention includes the following steps in step S2: S21. Fabric property modeling and material parameter database construction, S22. Differentiable physical simulation and S23. Key parameter output. Through the above steps, the physical properties of the virtual try-on clothing are accurately modeled and differentiable simulated to ensure the physical authenticity and visual rationality of the virtual try-on results.

[0019] 5. The virtual try-on system and method based on deep reinforcement learning of this invention have a multi-objective reward function: R(s,a)=α·R_visual+β·R_physics+γ·R_preference; where α, β, and γ are positive weight coefficients used to balance the importance of different reward items, R_visual is the visual realism score, R_physics is the physical constraint term, and R_preference is the user preference feedback. R_visual (visual realism score): measures the visual quality and realism of the virtual try-on effect; R_physics (physical constraint term): ensures that the deformation of the virtual try-on clothing conforms to physical laws, avoiding unnatural punctures, excessive stretching, or excessive compression; R_preference (user preference feedback): integrates the user's subjective evaluation and preferences, enabling the system to learn and adapt to personalized needs. All reward items are normalized to ensure they are within the same numerical range, thus preventing any one reward from dominating.

[0020] 6. The virtual try-on system and method based on deep reinforcement learning of the present invention includes a safety filter mechanism that verifies the action in real time after the agent selects an action but before actually executing the action, and corrects or rejects it when necessary. The verification and correction strategies include: S301. Limiting the maximum stretch ratio of the virtual try-on clothing, S302. Maintaining the minimum safe distance, and S303. Self-puncture detection and prevention. Through the above steps, the filter verifies the actions proposed by the quantile Q-learning algorithm in real time and corrects or rejects them when necessary.

[0021] 7. The virtual try-on system and method based on deep reinforcement learning of the present invention includes the following steps in the motion space design: S311. Global rigid body transformation and S312. Local non-rigid body deformation. Global rigid body transformation involves rotating and moving the entire garment in three-dimensional space; local non-rigid body deformation involves fine-tuning the deformation of local areas of the garment mesh. The motion space design takes into account both rigid body transformation and non-rigid body deformation, and introduces dynamic adjustment of physical material parameters to achieve more precise and flexible control.

[0022] 8. The virtual try-on system and method based on deep reinforcement learning of the present invention utilizes the quantile Q-learning algorithm through the following steps: S321. Modeling the reward distribution, S322. Maintaining multiple Q-value estimates, and S313. Quantile regression loss function. Through these steps, the quantile Q-learning algorithm can progressively learn the reward distribution obtained by taking different actions in different states. Therefore, given a state, by analyzing the Q-value distribution, it can select the optimal action to optimize the virtual try-on effect.

[0023] 9. The virtual try-on system and method based on deep reinforcement learning of the present invention, wherein the rendering output in step S4 includes the following steps: S41. Differentiable rendering system, S42. Ambient lighting estimation and fusion, and S43. Quality assessment and fine-tuning mechanism. These steps ensure that even under complex and changing user input and environmental conditions, the system can continuously output high-visual-quality virtual try-on results. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the virtual try-on system based on deep reinforcement learning according to the present invention. Figure 2This is a flowchart illustrating the virtual try-on method of the virtual try-on system based on deep reinforcement learning according to the present invention. Figure 3 This is a schematic diagram of the body shape recognition and reconstruction process of the virtual try-on system based on deep reinforcement learning according to the present invention. Figure 4 This is a schematic diagram of the material simulation process in the virtual try-on system based on deep reinforcement learning of the present invention. Figure 5 This is a flowchart illustrating the security filter mechanism of the virtual try-on system and try-on method based on deep reinforcement learning according to the present invention. Figure 6 This is a flowchart illustrating the design of the action space for the virtual try-on system and try-on method based on deep reinforcement learning according to the present invention. Figure 7 This is a flowchart illustrating the design of the quantile Q-learning algorithm for the virtual try-on system and try-on method based on deep reinforcement learning in this invention. Figure 8 This is a schematic diagram illustrating the rendering output design of the virtual try-on system and try-on method based on deep reinforcement learning according to the present invention. Detailed Implementation

[0026] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0027] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0028] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between the components; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0029] In the description of this application, it should be understood that the terms "upper", "lower", "side", "front", "rear", etc., indicate the orientation or positional relationship based on the installation orientation or positional relationship, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0030] In the description of this application, it should be noted that the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0031] It should also be noted that in the embodiments of this application, the same reference numerals are used to represent the same component or part. For the same part in the embodiments of this application, the reference numerals may only be used to mark one part or component as an example. It should be understood that the reference numerals are also applicable to other identical parts or components.

[0032] Furthermore, in this application, the technical features described in an open-ended manner include both closed technical solutions composed of the listed features and open technical solutions that include the listed features.

[0033] To further understand the invention's content, features, and effects, the following embodiments are provided, along with detailed descriptions in conjunction with the accompanying drawings: like Figure 1-8 As shown, the virtual try-on system based on deep reinforcement learning includes a body shape recognition and reconstruction module, a material simulation module, an adaptive optimizer, and a rendering output module. The body shape recognition and reconstruction module is used to collect human body shape parameters. The material simulation module is used to perform differentiable approximation simulation of virtual clothing fabrics. An adaptive optimizer is used to automatically adjust the optimization strategy based on different user characteristics. The rendering output module is used to render the results of the clothing try-on.

[0034] In this embodiment, a body shape recognition and reconstruction module collects the user's body shape parameters to form 3D point cloud data. A material simulation module simulates the material of the virtual try-on clothing to significantly improve the physical realism of the virtual try-on experience. An adaptive optimizer automatically adjusts the optimization strategy based on different user characteristics. Finally, a rendering output module outputs the virtual try-on effect. This system can be widely used in virtual try-on services on e-commerce platforms, digital prototype display of clothing designs, etc. Of course, it is not limited to this; other systems that can achieve the same function can be used.

[0035] Specifically, such as Figure 2 As shown, a virtual try-on method based on deep reinforcement learning includes the following steps: S1. Body Shape Recognition and Reconstruction: Based on user-provided multi-view... Figure 2 In the 3D image sequence, through image preprocessing, 3D point cloud generation and processing, and parametric human body model regression, the user's 3D body shape features and posture information are accurately identified and reconstructed, generating the user's 3D point cloud data and key body shape parameters; in this embodiment, the user-provided multi-view... Figure 2 The 2D image is a full-body image of the user from the front and side, but it can also be an image from other angles. The image undergoes preprocessing, including denoising and enhancement. Then, a depth map is generated from the 2D image using a depth estimation network (such as MiDaS or DPT), and combined with the camera intrinsic parameter matrix to convert it into raw 3D point cloud data. This 3D point cloud data is then denoised and completed, and registered using an algorithm to form complete 3D point cloud data. A neural network regressor is used to extract the SMPL-X shape parameter β and pose parameter θ from the 3D point cloud data. Based on the SMPL-X model, the user's height, chest circumference (in centimeters), waist circumference, hip circumference, and joint angles are calculated. Simultaneously, a human body surface contact probability map is generated. S2. Material Simulation: Based on the material type of the garment to be virtually tried on, the corresponding physical property parameters are retrieved and instantiated from a pre-built material parameter database. Using a differentiable physics simulation system, the physical behavior and deformation state of the virtual garment under gravity, impact force, and damping force are simulated, and quantified physical behavior parameters of the garment are output. In this embodiment, material parameters matching the material of the virtual garment are retrieved from the pre-built material parameter database. These material parameters include elastic modulus, Poisson's ratio, shear modulus, bending stiffness, density, static friction coefficient, dynamic friction coefficient, thickness (in millimeters), and roughness. However, the material parameters are not limited to these; any other parameters capable of reflecting the material state of the virtual garment can be used. The aforementioned material parameters are input into the differentiable physics simulation system, which simulates the physical behavior and deformation state of the virtual garment under gravity, impact force, and damping force, iterating over time steps to initially simulate the draping state of the virtual garment on the user's body. Again, this is not limited to these parameters; any other parameters capable of simulating the draping state of the virtual garment on the user's body can be used. S3. Adaptive Optimization: A quantile Q-learning algorithm based on deep reinforcement learning is constructed. The state space is defined by 3D point cloud data, body shape parameters, the posture of the virtual try-on clothing, the overlap measurement between the virtual try-on clothing and the user, and the material state of the virtual try-on clothing. The action space is defined by performing global rigid body transformations and / or local non-rigid body deformations and / or dynamic adjustments to the material parameters of the virtual try-on clothing. The learning process is guided by a multi-objective reward function that integrates visual realism scores, physical constraints, and user preference feedback. Actions proposed by a safety filter mechanism are verified and corrected in real time to learn and select the optimal action, thereby optimizing the virtual try-on effect. In this embodiment, the state space is constructed as follows: The state space includes body shape parameters, the current vertex position and normal vector of the clothing output by physical simulation, local stress distribution map features, wrinkle density distribution map features, puncture depth between the clothing and the human body and the area of ​​the self-puncture region, and material deformation energy. An action space is constructed so that at each time step, the virtual try-on clothing selects an action from a preset discrete action space. Finally, a learning process guided by a multi-objective reward function that integrates visual realism scores, physical constraints, and user preference feedback is implemented. Actions proposed through a safety filter mechanism are then validated and corrected in real time to learn and select the optimal action, thereby optimizing the virtual try-on effect. Of course, this is not limited to this; any other method that can optimize the virtual try-on effect is acceptable. S4. Rendering Output: The virtual try-on clothing model, adjusted through adaptive optimization steps, is rendered using differentiable rendering with 3D point cloud data to generate the final virtual try-on image or video. A parameter fine-tuning mechanism is triggered based on the quality assessment results of the no-reference image quality assessment model to optimize the rendering parameters and ensure the quality of the output image. In this embodiment, the optimized clothing model is input into the differentiable rendering system; ambient lighting estimation estimates a lighting probe from the user-provided image and fuses it with the clothing model for rendering. The rendered image is evaluated using the NR-IQA (No-Reference Image Quality Assessment) model, and the system automatically fine-tunes material parameters. The final output virtual try-on image displays a highly fitted virtual try-on garment on the user's body. However, this is not a limitation; any other method capable of achieving a highly fitted virtual try-on garment on the user's body can be used.

[0036] Specifically, such as Figure 3 As shown, body shape recognition and reconstruction in step S1 includes the following steps: S11. Multi-view Image Acquisition: Acquire frontal, side, and multi-angle images or RGB images of the user; in this embodiment, the user uploads their own frontal, side, and multi-angle images, and the system performs image denoising and image enhancement preprocessing operations on the received raw two-dimensional images. Specifically, image denoising uses a non-local mean filtering algorithm, and image enhancement uses an adaptive histogram equalization algorithm with limited contrast; thus providing a consistent and standardized input format for subsequent depth estimation; of course, it is not limited to this, and other algorithms that can achieve the same effect can be used; S12. 3D Point Cloud Generation: A depth estimation network is used to predict the depth information of each pixel from the input image, generating a corresponding depth map. Combined with camera intrinsic parameters, the depth map is converted into 3D point cloud data. Noise reduction and completion processing are then performed on the 3D point cloud data. In this embodiment, a high-resolution, high-precision dense depth map is generated from the 2D image using the MiDaS v3 depth prediction network based on the Transformer architecture. Subsequently, the system combines the pre-calibrated camera intrinsic parameter matrix to convert the depth map into 3D point cloud data. The conversion process follows a standard pinhole camera model. For each pixel (u, v) and its corresponding depth value Z_d in the depth map, its 3D coordinates (X_c, Y_c, Z_c) in the camera coordinate system are calculated as follows: X_c = (u - c_x) * Z_d / f_x, Y_c = (v - c_y) * Z_d / f_y, Z_c = Z_d. The set of 3D coordinates of all pixels constitutes the original 3D point cloud. To improve the quality of 3D point clouds, the system performs noise reduction and completion processing on the generated 3D point clouds. Noise reduction adopts a statistical outlier removal algorithm, and 3D point cloud completion adopts a deep learning model based on generative adversarial networks. The 3D point clouds from different perspectives are fused into a unified, higher-density, and higher-precision 3D point cloud to ensure the spatial consistency of multi-view information. Of course, it is not limited to this and can be any other method that can ensure the spatial consistency of multi-view information. S13. SMPL-X Parameter Regression: A neural network regressor is used to extract the shape parameter β and posture parameter θ of SMPL-X from the 3D point cloud data. The shape parameter β controls the body shape parameter, and the posture parameter θ controls the joint angles of the body shape parameter. The output body shape parameters include height, chest circumference, waist circumference, and hip circumference. In this embodiment, the system uses a regressor based on a deep neural network to extract the shape parameter β and posture parameter θ of the SMPL-X model from the 3D point cloud data processed in S12. The shape parameter β typically controls static body shape features such as height, weight, and body type, such as chest width and waist size. The posture parameter θ includes the rotation angles of major joints (such as neck, shoulders, elbows, hips, knees, etc.). The rotation of each joint is represented by three-dimensional axis angles, controlling the dynamic posture of the human body. Through the regression process of SMPL-X parameter regression, the system can accurately reconstruct the 3D human body mesh model under the user's current posture. Of course, it is not limited to this; any other method that can accurately reconstruct the 3D human body mesh model under the user's current posture can be used. S14. Calculation of Body Surface Contact Probability Map: Based on 3D point cloud data, the probability of contact between each surface point and the clothing is calculated. In this embodiment, firstly, the system voxelizes the human body mesh model formed from the 3D point cloud data. Simultaneously, the system acquires the clothing mesh model in its initial pose. Subsequently, the system calculates the Euclidean distance from each vertex of the clothing mesh to the nearest human body mesh surface. By constructing a signed distance field, the system generates a body surface contact probability map based on the distance field. The body surface contact probability map is stored in the form of mesh vertex attributes, with each vertex associated with a contact probability value between 0 and 1, forming a continuous density distribution map. The density distribution map is used to guide subsequent collision detection and constraint optimization; however, it is not limited to this and can be any other method that can achieve the same effect.

[0037] Specifically, such as Figure 4 As shown, the material simulation in step S2 includes the following steps: S21. Fabric Property Modeling and Material Parameter Database Construction: A material parameter database is constructed, containing parameters such as elastic modulus, Poisson's ratio, shear modulus, flexural stiffness, density, coefficient of friction, damping coefficient, and thickness. The system inputs the fabric properties of the clothing for virtual try-on and retrieves the corresponding material parameters from the database. The system receives the clothing material type selected by the user through the interface or automatically obtained from the clothing's metadata. Based on the selected material type, the system retrieves and instantiates the corresponding set of physical parameters from the pre-constructed material parameter database. The material parameter database is a rigorously calibrated and validated structured dataset storing detailed physical properties of hundreds of common fabrics. Material parameters include elastic modulus, Poisson's ratio, shear modulus, flexural stiffness, density, coefficient of friction, damping coefficient, and thickness. However, it is not limited to these parameters; other parameters that reflect the material state of the clothing being virtually tried on can also be used. S22, Differentiable Physics Simulation: A differentiable fabric simulation framework based on a mass-spring model is adopted. The three-dimensional mesh of the virtual try-on clothing is discretized into a series of mass points and connected by structural springs, shear springs, and bending springs. Under the action of external forces such as gravity, human-clothing collision force, and clothing self-collision force, the simulation framework uses the implicit Euler integral numerical method for time stepping and ensures that all physical parameters and state variables in the simulation process are differentiable in the entire simulation calculation graph. In this embodiment, the model is constructed by discretizing the three-dimensional mesh of the clothing into a series of mass points and connecting adjacent mass points using a mass-spring-damping system. The mass of a point mass is typically allocated based on the density of the fabric and the area it represents. Springs are categorized into structural springs, shear springs, and bending springs. Structural springs connect adjacent vertex pairs (i.e., edges of the mesh) to simulate the elastic behavior of the material in the tensile and compressive directions. Shear springs connect diagonal vertex pairs to simulate shear deformation, such as fabric skewing. Bending springs connect vertex pairs separated by one vertex to simulate bending and wrinkling behavior. These springs ensure that the garment forms natural wrinkles and drapes when deformed. Of course, the type of spring is not limited to these; other springs serving the same function can be used. The simulation system calculates and applies various external forces to the garment's points of mass. A preliminary screening is performed using the previously calculated surface contact probability map. For points that may collide, the system calculates their closest distance to the human body mesh surface and the surface normal vector. Subsequently, an elastic repulsive force is applied along the normal direction. Simultaneously, considering the coefficient of friction between the human body and the garment, a frictional force is applied along the tangential direction to simulate the garment sliding on the human body surface. A mesh-mesh cross-detection algorithm based on a spatially partitioned data structure is used to effectively identify puncture areas inside the garment. For self-piercing particle pairs, the system calculates their overlap depth and normal vector, and applies a similar repulsive force to eliminate the piercing. The system employs implicit Euler integrals for time stepping to ensure the numerical stability and efficiency of the simulation process. Implicit integrals update the particle positions and velocities by solving a linear system, and the calculation of its Jacobian matrix preserves gradient information, thus supporting differentiability. The physical simulation is performed in a small iterative loop. Within each time step, the system calculates all forces, updates the particle positions and velocities, and resolves all collisions until a steady state is reached or a specified number of simulation steps are taken to allow the virtual try-on clothing to reach equilibrium under gravity and other forces. Of course, this is not limited to this; other methods can be used to ensure the physical realism and visual plausibility of the virtual try-on results. S23. Key Parameter Output: After differentiable physics simulation, a series of key parameters quantifying the physical behavior and visual characteristics of clothing are output. These key parameters include drape parameters, stretch parameters, and gloss parameters. In this embodiment, after differentiable physics simulation, a series of key parameters quantifying the physical behavior and visual characteristics of clothing are output for the calculation of the multi-objective reward function. The key parameters are drape parameters, stretch parameters, and gloss parameters. The drape parameter describes the natural drape and wrinkle shape of the fabric under gravity. The stretch parameter quantifies the elastic deformation ability and recovery characteristics of the fabric under stress. The gloss parameter simulates the surface reflection characteristics of different materials. Of course, the key parameters are not limited to these; they can be other parameters that can quantify the physical behavior and visual characteristics of clothing.

[0038] Specifically, the clothing pose includes the mesh vertex coordinates and normal vectors of the virtual try-on clothing; the overlap metric includes the distance field between the user's body surface and the virtual try-on clothing, and the puncture detection results of the virtual try-on clothing; and the material state includes the degree of deformation and stress distribution of the virtual try-on clothing. Of course, it is not limited to these; other parameters or indicators that can achieve the same function can be used.

[0039] Specifically, the multi-objective reward function is: R(s,a)=α·R_visual+β·R_physics+γ·R_preference; Where α, β, and γ are positive weighting coefficients used to balance the importance of different reward items, R_visual is the visual realism score, R_physics is the physical constraint term, and R_preference is the user preference feedback.

[0040] In this embodiment, a multi-objective reward function R(s,a) is constructed. This function integrates reward signals from three dimensions and combines them through weighted summation. R_visual is the visual realism score, which measures the visual quality and realism of the virtual try-on effect. R_physics is the physical constraint term, which includes self-puncture penalty and overstretch penalty. The self-puncture penalty function is `R_physics_penetration = (-λ1 * (1 - exp(-k1 * penetration_depth_sum)))`, where `penetration_depth_sum` is the sum of the puncture depths (in millimeters) of all detected puncture points (including punctures between the clothing and the human body and self-punctures of the clothing itself). λ1 and k1 are positive coefficients; the greater the puncture depth, the heavier the penalty. The `exp` function applies a gentler penalty for small punctures and a rapidly increasing penalty for severe punctures. The overstretch penalty function is `R_physics_stretch = (-λ2 * (1 - exp(-k2 * overstretching_ratio_max)))`, where `overstretching_ratio_max` is the maximum stretch ratio of any edge in the clothing mesh relative to its original length (e.g., if the original length is L and the current length is L', the ratio is L' / L; a penalty is triggered if it exceeds 1.2). λ2 and k2 are positive coefficients; the penalty increases rapidly when the stretch ratio exceeds the material's elastic limit, ensuring that the deformation of the virtual try-on clothing conforms to physical laws and avoiding unnatural punctures, overstretching, or overcompression. However, this is not limited to this; any other virtual try-on clothing whose deformation conforms to physical laws can be used. `R_preference` provides user preference feedback, integrating the user's subjective evaluation and preferences, enabling the system to learn and adapt to personalized needs. Again, this is not limited to this; any other function that allows the system to learn and adapt to personalized needs can be used.

[0041] Specifically, such as Figure 5 As shown, the security filter mechanism includes real-time verification of the action after the agent selects an action but before actually executing it, and correction or rejection as necessary. The verification and correction strategies include: S301. Maximum Stretching Ratio Limit: Evaluates the potential local stretching ratio of the clothing mesh caused by the action. If the stretching ratio of any edge exceeds the preset material elastic limit, the action is corrected, limiting the relevant vertex displacement vector or deformation gradient parameter to ensure it does not exceed a safety threshold. In this embodiment, after the system proposes an action, the safety filter first evaluates the potential local stretching of the clothing mesh caused by the action. The system performs a forward-looking simulation step in the simulation environment to calculate the stretching ratio of all edges of the clothing mesh after the action is executed. If the stretching ratio of any edge exceeds the preset material elastic limit, the action is deemed unsafe; otherwise, it is deemed unsafe. In this case, the safety filter will take corrective measures by limiting the relevant vertex displacement vector or deformation gradient parameter to ensure it does not exceed the safety threshold. Of course, this is not limited to this; other methods that achieve the same effect can be used. S302. Minimum Safe Distance Maintenance: The minimum distance between the clothing and the human body is monitored using a surface contact probability map and a signed distance function. If the predicted minimum distance is less than a preset safety threshold, the action is corrected by applying additional repulsive force to the contact area or adjusting local vertex displacement to ensure that the clothing and the body remain above the minimum safe distance. In this embodiment, the safety filter uses a surface contact probability map and a signed distance function to monitor the minimum distance between the clothing and the human body. Before executing an action, the system predicts the distance between the clothing mesh and the human body mesh after the action is executed. If the predicted minimum distance is less than the preset safety threshold, it is determined that the action may lead to puncture or excessive contact. The filter ensures that the clothing and the body remain above the minimum safe distance by applying additional repulsive force to the contact area (as part of the action correction) or adjusting local vertex displacement; of course, it is not limited to this, and other methods can achieve the same effect. S303. Self-Piercing Detection and Prevention: Real-time monitoring of self-piercing within the virtual clothing mesh. If any intersection between mesh faces or a vertex penetrating its own face is detected, the action is corrected, triggering a local optimization process to eliminate the piercing. In this embodiment, a safety filter monitors self-piercing within the clothing mesh in real time. After the action is executed, if any intersection between mesh faces or any vertex penetrating another face of the clothing is detected, it is determined to be an unsafe action. The filter triggers a local optimization process, fine-tuning the displacement or rollback of the relevant vertices and applying additional penalty force to push out the pierced portion to restore physical consistency; however, this is not limited to this, and other methods achieving the same effect can be used. The safety filter mechanism feeds the corrected action back to the quantile Q-learning algorithm to update its strategy.

[0042] Specifically, such as Figure 6As shown, the design of the motion space includes the following steps: S311. Global Rigid Body Transformation: This involves rotating and moving the virtual try-on garment in three-dimensional space. Rotation is parameterized using Euler angles, and translation represents the garment's movement along the X, Y, and Z axes. In this embodiment, the global rigid body transformation includes a rotation matrix and translation vectors. The rotation matrix represents the three-dimensional rotation of the garment relative to the human body or the world coordinate system. Euler angles are used for parameterization, with a range of rotation angles set for each axis. However, this is not limited to this method; other methods that achieve the same effect can be used. S312. Local Non-rigid Deformation: Fine-tuning of local areas of the virtual try-on clothing mesh. This includes applying small 3D displacement vectors to key vertices of a specific area. The displacement direction can be the normal or tangent direction of the vertex. It also involves controlling the deformation field of the local area of ​​the clothing through deformation gradient parameters. These parameters affect advanced deformation properties such as wrinkle intensity, wrinkle direction, and fabric drape. In this embodiment, local non-rigid deformation includes local vertex displacement vectors and deformation gradient parameters. The local vertex displacement vector is applied to a specific area of ​​the clothing mesh, with the displacement direction being either the normal or tangent direction of the vertex. The displacement amplitude can be set. The deformation gradient parameters control the deformation field of the local area of ​​the clothing, indirectly adjusting the mesh deformation through the control point positions of the local coordinate system or the free deformation cage. The deformation gradient parameters can be scalar values ​​affecting advanced deformation properties such as wrinkle intensity and fabric drape, with their adjustment range and step size determined based on the specific deformation model. However, this is not limited to these methods; other methods can achieve the same effect.

[0043] Specifically, such as Figure 7 As shown, the quantile Q-learning algorithm works through the following steps: S321. Modeling the reward distribution: The quantile Q-learning algorithm models the entire conditional distribution P(R|s,a) of the future cumulative reward that can be obtained by performing action a in a specific state s by learning a Q-value function Z(s,a). The distribution is approximated by a set of discrete quantile points. S322. Maintaining Multiple Q-Value Estimates: The quantile Q-learning algorithm maintains Q-value estimates for multiple quantile levels. For each quantile τ_i, a corresponding Q-value Z_i(s,a) is estimated such that P(Z(s,a)≤Z_i(s,a))=τ_i; S323. Quantile Regression Loss Function: During training, the agent uses the quantile regression loss function to update the quantile Q-value estimate. By optimizing this loss function, the quantile Q network can gradually learn the shape of the reward distribution, thereby capturing the uncertainty of the reward and supporting the agent in learning risk-sensitive policies.

[0044] In this embodiment, the quantile Q-learning algorithm learns a Q-value function Z(s,a), which represents the conditional distribution of the future cumulative reward obtainable by performing action a in state s. This distribution is approximated by a set of discrete quantile points, for example, N_quantiles = 10, with corresponding quantiles τ_i = {0.05, 0.15, ..., 0.95}. For each quantile τ_i, the algorithm estimates a corresponding Q-value Z_i(s,a) such that P(Z(s,a) ≤ Z_i(s,a)) = τ_i. During training, the system uses a quantile regression loss function, such as the Huber loss function or the Pinball loss function, to update these quantile Q-value estimates; by optimizing the loss function, the quantile Q-learning algorithm can progressively learn the shape of the reward distribution, rather than just its mean, thus providing richer reward information. Of course, it is not limited to this; any other algorithm capable of providing richer reward information can be used.

[0045] Specifically, such as Figure 8 As shown, the rendering output in step S4 includes the following steps: S41. Differentiable Rendering System: Employing differentiable rasterization technology or differentiable volume rendering technology based on neural radiation fields, all geometric parameters, material parameters, and lighting parameters in the rendering process are differentiable. The differentiable rendering system, combined with a physically based material model, achieves physically correct lighting interaction and supports real-time calculation of shadows, reflections, and refractions. In this embodiment, differentiable rasterization softens pixel attribution, allowing the gradients of vertex positions, normal vectors, UV coordinates, and material parameters with respect to the rendered image to be calculated. This system, combined with a PBR material model, maps the gloss parameters output by the material simulation module to PBR material properties, achieving physically correct lighting interaction. By analyzing gradient calculations, it ensures that every operation in the rendering process is differentiable; of course, other methods capable of achieving the same effect can also be used. S42. Ambient Lighting Estimation and Fusion: From the background image provided by the user, a deep learning network is used to infer and estimate the ambient lighting information of the scene, generating a high dynamic range ambient lighting probe. During the differentiable rendering process, the ambient lighting probe is fused with the 3D model in the virtual scene to ensure that the virtual try-on clothing visually matches the lighting effects of the background environment. In this embodiment, the system receives a background image provided by the user and uses a deep learning network, such as a CNN-based U-Net architecture or a more advanced Transformer-based network, to infer the ambient lighting information of the scene from the image. This network, through... The system analyzes visual cues such as highlights, shadows, color distribution, and texture gradients in the image to estimate the environment's brightness, color temperature, main light source direction, intensity, color temperature, and more complex global illumination information. This estimated lighting information is used to generate ambient lighting probes, typically represented as high dynamic range cube maps or spherical harmonic function coefficients. Cube maps capture lighting information radiating from the scene center in all directions with high accuracy; while spherical harmonic functions provide a compact, mathematically comprehensible lighting representation, facilitating rapid rendering calculations. During differentiable rendering, these ambient lighting probes are integrated with the 3D models (human body and clothing) in the virtual scene. The material model utilizes this ambient lighting information to calculate diffuse and specular reflections at each point on the clothing surface, ensuring that the virtual try-on clothing visually matches the lighting effects of the background environment, thereby enhancing the overall realism and immersion. Of course, this is not limited to these methods; other methods that maintain consistency between the virtual try-on effect and the lighting conditions of the user's environment can also be used. S43. Quality Assessment and Fine-tuning Mechanism: A deep learning-based no-reference image quality assessment algorithm is used to assess the quality of the final rendered virtual try-on image. When the assessed image quality score is lower than the preset quality threshold, the parameter fine-tuning mechanism is triggered to automatically adjust the lighting parameters, material parameters and / or perform slight geometric fine-tuning in the renderer. In this embodiment, a deep learning-based no-reference image quality assessment algorithm is used to evaluate the quality of the final rendered virtual try-on image. When the image quality score evaluated by the no-reference image quality assessment algorithm model is lower than a preset quality threshold, the system will trigger a parameter fine-tuning mechanism. The fine-tuning process is achieved through gradient descent: automatically adjusting the lighting parameters in the renderer, including the brightness, contrast, and color temperature of the ambient light, as well as the direction and intensity of the main light source; fine-tuning the material parameters of the clothing, such as adjusting roughness, specular reflectivity, or metallicity, to optimize the gloss and texture details of the clothing surface, so that it presents a better visual effect under specific lighting conditions; and, without violating physical constraints, making minor vertex displacements or pose adjustments to the clothing mesh to correct any possible minor punctures or unnatural wrinkles. This quality assessment and fine-tuning mechanism ensures that even under complex and changing user input and environmental conditions, the system can continuously output high-quality virtual try-on results. Of course, it is not limited to this; any other mechanism that can ensure that the image quality of the final rendered output remains at a high level is acceptable.

[0046] Working principle: Body shape recognition and reconstruction: Based on multiple views provided by the user Figure 2 In the 3D image sequence, through image preprocessing, 3D point cloud generation and processing, and parametric human body model regression, the user's 3D body features and posture information are accurately identified and reconstructed, generating the user's 3D point cloud data and key body parameters. Material simulation: Based on the material type of the clothing to be virtually tried on, the corresponding physical property parameters are retrieved and instantiated from the pre-built material parameter database. Using a differentiable physics simulation system, the physical behavior and deformation state of the virtual clothing under the action of gravity, collision force and damping force are simulated, and the quantified physical behavior parameters of the clothing are output. Adaptive Optimization: A quantile Q-learning algorithm based on deep reinforcement learning is constructed. The high-dimensional state space is composed of a 3D human body mesh model, key body shape parameters, deformation state of virtual try-on clothing, physical behavior parameters of clothing, and environmental state. The action space is composed of global rigid body transformation, local non-rigid body deformation, and / or dynamic adjustment of physical material parameters of virtual try-on clothing. The learning process of the quantile Q-learning algorithm is guided by a multi-objective reward function that integrates visual realism scores, physical constraints, and user preference feedback. The actions proposed by the quantile Q-learning algorithm are verified and corrected in real time through a safety filter mechanism so that the quantile Q-learning algorithm can select the optimal action, thereby optimizing the try-on effect of virtual try-on clothing. Rendering output: The virtual try-on clothing model adjusted through adaptive optimization steps and the 3D point cloud data are rendered differently to generate the final virtual try-on image or video. The parameter fine-tuning mechanism is triggered based on the quality assessment results of the no-reference image quality assessment model to optimize the rendering parameters to ensure the quality of the output image.

[0047] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0048] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A virtual try-on system based on deep reinforcement learning, characterized in that: It includes a body shape recognition and reconstruction module, a material simulation module, an adaptive optimizer, and a rendering output module. The body shape recognition and reconstruction module is used to collect human body shape parameters. The material simulation module is used to perform differentiable approximate simulation of virtual try-on clothing fabrics. The adaptive optimizer is used to automatically adjust the optimization strategy based on different user characteristics. The rendering output module is used to render the results of the clothing try-on.

2. A fitting method based on the system described in claim 1, characterized in that: Includes the following steps: S1. Body Shape Recognition and Reconstruction: From the multi-view two-dimensional image sequence provided by the user, through image preprocessing, three-dimensional point cloud generation and processing, and parameterized human body model regression, the user's three-dimensional body shape features and posture information are accurately identified and reconstructed, and the user's three-dimensional point cloud data and key body shape parameters are generated. S2. Material Simulation: Based on the material type of the clothing to be virtually tried on, retrieve and instantiate the corresponding physical property parameters from the pre-built material parameter database, and use a differentiable physical simulation system to simulate the physical behavior and deformation state of the virtual clothing under the action of gravity, collision force and damping force, and output quantified physical behavior parameters of the clothing. S3. Adaptive Optimization: Construct a quantile Q-learning algorithm based on deep reinforcement learning. The state space is the 3D point cloud data, the body shape parameters, the clothing posture of the virtual try-on clothing, the overlap measurement between the virtual try-on clothing and the user, and the material state of the virtual try-on clothing. The action space is the global rigid body transformation and / or local non-rigid body deformation and / or dynamic adjustment of the material parameters of the virtual try-on clothing. The learning process is guided by a multi-objective reward function that integrates visual realism scores, physical constraints, and user preference feedback. The actions proposed by the safety filter mechanism are verified and corrected in real time. The algorithm learns and selects the optimal action to optimize the try-on effect of the virtual try-on clothing. S4. Rendering Output: The virtual try-on clothing model adjusted through the adaptive optimization steps is rendered differently with the 3D point cloud data to generate the final virtual try-on image or video. The parameter fine-tuning mechanism is triggered based on the quality assessment results of the no-reference image quality assessment model to optimize the rendering parameters to ensure the quality of the output image.

3. The fitting method according to claim 2, characterized in that: The body shape recognition and reconstruction in step S1 includes the following steps: S11. Multi-view Image Acquisition: Acquires frontal, side, and multi-angle images or RGB images of the user. S12. 3D Point Cloud Generation: A depth estimation network is used to predict the depth information of each pixel in the input image, generating a corresponding depth map. Combined with camera intrinsic parameters, the depth map is converted into 3D point cloud data. Noise reduction and completion processing are then performed on the 3D point cloud data. S13. SMPL-X Parameter Regression: A neural network regressor is used to extract the shape parameter β and pose parameter θ of SMPL-X from the 3D point cloud data. The shape parameter β controls the body shape of the body shape parameter, and the pose parameter θ controls the joint angles of the body shape parameter. The output body shape parameter includes height, chest circumference, waist circumference, and hip circumference. S14. Calculation of surface contact probability map: Based on the three-dimensional point cloud data, calculate the probability that each surface point may come into contact with the clothing.

4. The fitting method according to claim 2 or 3, characterized in that: The material simulation in step S2 includes the following steps: S21. Fabric property modeling and material parameter database construction: Construct a material parameter database, wherein the material parameters include elastic modulus, Poisson's ratio, shear modulus, bending stiffness, density, coefficient of friction, damping coefficient, and thickness. Input the fabric properties of the virtual try-on garment, and retrieve the material parameters corresponding to the fabric properties of the virtual try-on garment from the material parameter database. S22. Differentiable physical simulation: A differentiable fabric simulation framework based on a mass-spring model is adopted. The three-dimensional mesh of the virtual try-on garment is discretized into a series of mass points and connected by structural springs, shear springs and bending springs. Under the action of external forces such as gravity, human-garment collision force and garment self-collision force, the simulation framework uses the numerical method of implicit Euler integral for time stepping and ensures that all physical parameters and state variables in the simulation process are differentiable in the entire simulation calculation graph. S23. Key Parameter Output: After the differentiable physical simulation, a series of key parameters that quantify the physical behavior and visual characteristics of the garment are output, including drape parameters, stretch parameters, and gloss parameters.

5. The fitting method according to claim 4, characterized in that: The clothing posture includes the grid vertex coordinates and normal vector of the virtual try-on clothing; the overlap metric includes the distance field between the user's body surface and the virtual try-on clothing, and the puncture detection result of the virtual try-on clothing; the material state includes the degree of deformation of the virtual try-on clothing and the stress distribution of the virtual try-on clothing.

6. The fitting method according to claim 5, characterized in that: The multi-objective reward function is: R(s,a)=α·R_visual+β·R_physics+γ·R_preference; Where α, β, and γ are positive weighting coefficients used to balance the importance of different reward items, R_visual is the visual realism score, R_physics is the physical constraint term, and R_preference is the user preference feedback.

7. The fitting method according to claim 6, characterized in that: The security filter mechanism includes real-time verification of the action after the agent selects an action but before the action is actually executed, and correction or rejection as necessary. The verification and correction strategies include: S301 Maximum stretch ratio limit: Evaluate the local stretch ratio of the clothing mesh that may be caused by the action. If the stretch ratio of any edge exceeds the preset material elastic limit, the action is corrected and the relevant vertex displacement vector or deformation gradient parameter is limited so that it does not exceed the safety threshold. S302. Minimum safe distance maintenance: The minimum distance between clothing and human body is monitored using a body surface contact probability map and a signed distance function. If the predicted minimum distance is less than a preset safety threshold, the action is corrected by applying additional repulsive force in the contact area or adjusting the local vertex displacement to ensure that clothing and body are kept above the minimum safe distance. S303. Self-puncture detection and prevention: Real-time monitoring of self-puncture within the virtual try-on clothing mesh; if any intersection or vertex penetration of a clothing mesh surface is detected, the action is corrected, triggering a local optimization process to eliminate puncture. The security filter mechanism feeds back the corrected action to the quantile Q-learning algorithm to update its strategy.

8. The fitting method according to claim 7, characterized in that: The design of the motion space includes the following steps: S311. Global Rigid Body Transformation: Rotate and move the virtual try-on clothing in three-dimensional space. The rotation is parameterized by Euler angles, and the translation represents the movement of the clothing in the X, Y, and Z axis directions. S312. Local Non-rigid Deformation: Fine deformation adjustment is performed on local areas of the virtual try-on clothing mesh, including applying small three-dimensional displacement vectors to key vertices of specific areas. The displacement direction can be the normal direction or the tangent direction of the vertex. The deformation field of the local area of ​​the clothing is controlled by deformation gradient parameters. The deformation gradient parameters affect advanced deformation properties such as fold intensity, fold direction or fabric drape curvature.

9. The fitting method according to claim 8, characterized in that: The quantile Q-learning algorithm works through the following steps: S321. Modeling the reward distribution: The quantile Q-learning algorithm models the entire conditional distribution P(R|s,a) of the future cumulative reward that can be obtained by performing action a in a specific state s by learning a Q-value function Z(s,a), which is approximated by a set of discrete quantile points; S322. Maintaining multiple Q-value estimates: The quantile Q-learning algorithm maintains Q-value estimates at multiple quantile levels. For each quantile τ_i, a corresponding Q-value Z_i(s,a) is estimated such that P(Z(s,a)≤Z_i(s,a))=τ_i; S323. Quantile Regression Loss Function: During training, the agent uses the quantile regression loss function to update the quantile Q-value estimate. By optimizing this loss function, the quantile Q network can gradually learn the shape of the reward distribution, thereby capturing the uncertainty of the reward and supporting the agent in learning risk-sensitive policies.

10. The fitting method according to any one of claims 5-9, characterized in that: The rendering output in step S4 includes the following steps: S41. Differentiable rendering system: It adopts differentiable rasterization technology or differentiable volume rendering technology based on neural radiation field, so that all geometric parameters, material parameters and lighting parameters in the rendering process can be differentiated. The differentiable rendering system combines physical material model to realize physically correct lighting interaction and supports real-time calculation of shadow, reflection and refraction effects. S42. Ambient lighting estimation and fusion: From the background image provided by the user, the ambient lighting information of the scene is inferred and estimated using a deep learning network to generate a high dynamic range ambient lighting probe. During the differentiable rendering process, the ambient lighting probe is fused with the 3D model in the virtual scene to ensure that the virtual try-on clothing is visually highly matched with the lighting effect of the background environment. S43. Quality assessment and fine-tuning mechanism: A deep learning-based no-reference image quality assessment algorithm is used to assess the quality of the final rendered virtual try-on image. When the assessed image quality score is lower than a preset quality threshold, a parameter fine-tuning mechanism is triggered to automatically adjust the lighting parameters, material parameters and / or perform slight geometric fine-tuning in the renderer.

Images