A virtual try-on method for e-commerce products based on augmented reality

Abstract

This invention relates to the field of augmented reality technology, specifically to a virtual try-on method for e-commerce products based on augmented reality. The method includes the following steps: acquiring real-time images of the user through an augmented reality device's camera, identifying the user's skeletal joints and constructing a three-dimensional body model of the user; simultaneously retrieving the three-dimensional model and resting state mesh data of the target garment from an e-commerce product database; and overlaying the garment's three-dimensional model onto the user's three-dimensional body model. This invention captures user images in real-time through a camera, extracts information about the user's skeletal joints, and accurately constructs the user's three-dimensional body model based on this information, thereby achieving precise alignment with the garment model. By establishing an initial three-dimensional model of the wearing state, it reflects the actual fit between the garment and the human body in real time. Simultaneously, it traverses the mesh structure of the garment model, analyzing the tightness, gap distance, and alignment deviation of key semantic lines of the garment point by point, quantifying these deviations into an intuitive global fit index.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, and in particular to a virtual try-on method for e-commerce products based on augmented reality. Background Technology

[0002] Augmented reality (AR) is a technology that calculates and integrates virtual information with the real environment in real time. Through cameras, display devices, and spatial tracking technology, virtual objects are accurately superimposed onto the real world, forming a new mode of human-computer interaction.

[0003] Current technologies typically only capture user images using camera devices and overlay virtual clothing onto them. This fails to fully consider the user's specific body shape and real-time posture changes, and lacks quantitative analysis and intuitive feedback on clothing fit. Consequently, it's difficult to provide accurate assessments and suggestions regarding clothing suitability. Because it doesn't delve into the actual interaction between clothing and the human body, relying solely on visual overlay makes it difficult for users to accurately judge the clothing's effect, easily leading to errors in size judgment. This, in turn, results in problems such as unsuitable clothing after purchase, increased return rates, and a deteriorating user experience. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a virtual try-on method for e-commerce products based on augmented reality.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a virtual try-on method for e-commerce products based on augmented reality, comprising the following steps:

[0006] The system acquires real-time images of the user through the camera of the augmented reality device, identifies the user's skeletal joints and constructs a three-dimensional body model of the user. At the same time, it retrieves the three-dimensional model of the target clothing and the resting state mesh data from the e-commerce product database, and overlays the clothing three-dimensional model onto the user's three-dimensional body model to generate an initial three-dimensional model of the wearing state.

[0007] Based on the initial three-dimensional model of the wearing state, the clothing mesh is traversed to establish multi-dimensional geometric deviation values. The tightness value, gap value and alignment value in the multi-dimensional geometric deviation values ​​are calculated to obtain the global quantitative fit index.

[0008] Calculate the local fit index and set a color gradient from positive to negative values. Establish a visual mapping rule set. Based on the visual mapping rule set, convert the local fit index to the color, transparency and highlight attributes of each vertex on the clothing 3D model and establish a vertex rendering instruction set.

[0009] The vertex rendering instruction set is read, and the defined color and transparency information are assigned to the corresponding vertices of the 3D model in the initial wearing state. Streamlines and highlight effects are drawn on the target surface area to obtain a dynamic fit texture. In the augmented reality view, the clothing model with the dynamic fit texture is superimposed on the user image in real time. The global quantized fit index, local fit index, and vertex rendering instruction set are updated cyclically as the user's posture changes. Wearing suggestions are given based on the global quantized fit index, and an augmented reality fit heatmap is generated.

[0010] Preferably, the steps for obtaining the initial three-dimensional model of the wearing state are as follows:

[0011] The augmented reality device's camera collects user image information in real time, and uses image feature detection technology to identify skeletal key points in the user image information to obtain three-dimensional coordinate data of the head, torso, and limbs. The three-dimensional coordinate data of the skeletal key points are then used to reconstruct the user's three-dimensional body model, generating the user's three-dimensional body model.

[0012] Based on the user's 3D body model, the clothing 3D model and clothing resting state mesh data corresponding to the target clothing are retrieved and extracted from the e-commerce product database. The clothing 3D model and the user's 3D body model are superimposed through coordinate system transformation, and the relative positions of the clothing 3D model and the user's 3D body model are adjusted to form a preliminary superimposed model.

[0013] Based on the preliminary overlay model, the current pose of each joint of the user's three-dimensional body model is obtained, and the pose of each key positioning point of the clothing three-dimensional model is synchronized to the pose of the corresponding skeletal key points of the user's three-dimensional body model to form an initial wearing state three-dimensional model.

[0014] Preferably, the step of obtaining the multidimensional geometric deviation value is as follows:

[0015] Based on the initial wearing state 3D model, extract the node coordinates of all mesh edges in the clothing 3D model, traverse each mesh edge and calculate the Euclidean distance between two nodes in the current frame state and the distance in the corresponding resting state, obtain the relative change rate of each mesh edge, and generate a set of clothing mesh edge length change ratios.

[0016] Based on the initial wearing state 3D model, the normal vector direction of each mesh surface is obtained, and a ray is emitted from the opposite direction of the normal vector to intersect with the surface of the user's 3D body model. The distance from the ray to the intersection point is recorded as the gap distance value. Combined with the set of clothing mesh side length change ratios, a multidimensional geometric difference set is formed.

[0017] Preferably, the step of obtaining the global quantitative fit index is as follows:

[0018] The global quantized fit index is calculated based on the multidimensional geometric difference set.

[0019] Preferably, the step of obtaining the visualization mapping rule set is as follows:

[0020] Based on the initial wearable state 3D model, extract all mesh edge indices connected to each vertex and the current length and resting length of the corresponding edge, calculate the relative stretch value of each edge, and extract the gap distance and ideal gap distance of the mesh surface adjacent to each vertex to construct the stretch set and gap set of each vertex.

[0021] Based on the stretch set and the gap set, calculate the local fit index of each vertex;

[0022] Based on the local fit index, the local fit index is mapped to the final visual color value and assigned to the corresponding vertex to generate a visual mapping rule set.

[0023] Preferably, the step of obtaining the vertex rendering instruction set is as follows:

[0024] Based on the aforementioned visualization mapping rule set, the local fit index of each vertex of the clothing 3D model is read one by one, and the local fit index is mapped to the corresponding color value, transparency value and highlight attribute label respectively. The color value, transparency value and highlight attribute label are recorded one by one in vertex order to generate a vertex rendering instruction set.

[0025] Preferably, the step of obtaining the dynamic fit texture is as follows:

[0026] According to the vertex rendering instruction set, all vertex records in the vertex rendering instruction set are traversed, and the color value and transparency value in each vertex record are assigned to the corresponding vertex in the initial wearing state 3D model. The target number of adjacent vertices are extracted around the vertex with the highlight attribute label as highlight, and continuous streamlines and highlight areas are drawn using the coordinate information of the adjacent vertices to form a dynamic fit texture.

[0027] Preferably, the step of obtaining the augmented reality fit heatmap is as follows:

[0028] Based on the dynamic fit texture, the augmented reality device captures the user's image in real time and calculates the user's real-time posture. The spatial position of the dynamic fit texture in the augmented reality view is adjusted in real time. When the user's posture changes, the quantized fit index, local fit index and vertex rendering instruction set are repeatedly acquired to update the dynamic fit texture in real time, generate an augmented reality fit heatmap, and provide wearing suggestions based on the global quantized fit index.

[0029] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0030] This invention captures user images in real time using a camera, extracts information about the user's skeletal joints, and accurately constructs a 3D body model of the user based on this data. This model then achieves precise alignment with the clothing model, establishing an initial 3D model of the wearing state to reflect the actual fit between the clothing and the human body in real time. Simultaneously, it traverses the mesh structure of the clothing model, analyzing the tightness, gap distance, and alignment deviation of key semantic lines of the clothing point by point, quantifying these into an intuitive global fit index. Further refining to each vertex of the model, it calculates local fit indices and uses color gradients to intuitively represent the differences in wearing effects in different areas. Real-time vertex rendering instructions are generated and dynamic fit textures are overlaid on the user's image. When the user's posture changes, this texture automatically updates, providing clear wearing suggestions in real time and forming an intuitive augmented reality heatmap. This improves the realism and accuracy of the interaction between the human body and virtual clothing during online try-on, effectively reducing size judgment errors and return / exchange risks during online shopping, and enhancing the accuracy of users' online shopping decisions and their confidence in purchasing. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the steps of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0033] Please see Figure 1 This invention provides a technical solution: a virtual try-on method for e-commerce products based on augmented reality, comprising the following steps:

[0034] The system acquires real-time images of the user through the camera of the augmented reality device, identifies the user's skeletal joints and constructs a three-dimensional body model of the user. At the same time, it retrieves the three-dimensional model of the target clothing and the resting state mesh data from the e-commerce product database, and overlays the clothing three-dimensional model onto the user's three-dimensional body model to generate an initial three-dimensional model of the wearing state.

[0035] Based on the initial three-dimensional model of the wearing state, the clothing mesh is traversed to establish multi-dimensional geometric deviation values. The tightness value, gap value and alignment value in the multi-dimensional geometric deviation values ​​are calculated to obtain the global quantitative fit index.

[0036] Calculate the local fit index and set a color gradient from positive to negative values. Establish a visual mapping rule set. Based on the visual mapping rule set, convert the local fit index to the color, transparency and highlight attributes of each vertex on the clothing 3D model and establish a vertex rendering instruction set.

[0037] The system reads the vertex rendering instruction set, assigns the defined color and transparency information to the corresponding vertices of the 3D model in the initial wearing state, and draws the streamlines and highlights on the target surface area to obtain the dynamic fit texture. In the augmented reality view, the clothing model with the dynamic fit texture is superimposed on the user image in real time, and the global quantized fit index, local fit index and vertex rendering instruction set are updated cyclically as the user's posture changes. Wearing suggestions are given based on the global quantized fit index, and an augmented reality fit heatmap is generated.

[0038] The steps for obtaining the initial 3D model of the wearable state are as follows:

[0039] The augmented reality device's camera collects user image information in real time, and uses image feature detection technology to identify skeletal key points in the user image information to obtain three-dimensional coordinate data of the head, torso, and limbs. The three-dimensional coordinate data of the skeletal key points are then used to reconstruct the user's three-dimensional body model, generating the user's three-dimensional body model.

[0040] Based on the user's 3D body model, the 3D model of the target clothing and the resting state mesh data of the clothing are retrieved and extracted from the e-commerce product database. The 3D model of the clothing is superimposed on the user's 3D body model through coordinate system transformation, and the relative position of the 3D model of the clothing and the user's 3D body model is adjusted to form a preliminary superimposed model.

[0041] Based on the initial overlay model, the current pose of each joint of the user's 3D body model is obtained, and the pose of each key positioning point of the clothing 3D model is synchronized to the pose of the corresponding skeletal key points of the user's 3D body model to form the initial wearing state 3D model.

[0042] Specifically, the augmented reality device's camera captures user image information in real time at a frequency of 30 frames per second. Each frame of RGB image data is processed using image feature detection technology employing a convolutional neural network. Specifically, a cascaded pose regression network is used to identify skeletal key points in the user image information. This network consists of a backbone feature extraction network and three cascaded pose estimation subnetworks. The backbone feature extraction network uses a ResNet-50 structure and is responsible for extracting deep visual feature maps from the input 224x224 pixel single-frame RGB image. Each pose estimation subnetwork receives the feature map from the previous stage as input. The network takes a single frame image as input and outputs 3D coordinate data containing 33 skeletal keypoints. These keypoints cover the major joints of the head, torso, and limbs, such as the tip of the nose, left and right eyes, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles. During the network training phase, a hybrid dataset of the publicly available human pose dataset Human3.6M and COCO is used. Data augmentation techniques, such as random rotation, scaling, cropping, and color jitter, are employed to expand the training samples. The loss function is the mean squared error between the predicted keypoint coordinates and the ground truth label coordinates. The model employs a difference loss method combined with a bone length consistency loss. This consistency loss penalizes deviations in the length of the line connecting adjacent keypoints from the standard human skeleton proportions, ensuring the physiological plausibility of the predicted skeleton. Training utilizes the Adam optimizer with an initial learning rate of 0.001 and a batch size of 64, undergoing 100 iterations. After training, the model is permanently installed and deployed locally on the augmented reality device. During inference, each frame captured by the camera is input into the trained model, directly outputting the three (x, y, z) coordinates of 33 keypoints. Subsequently, the acquired 3D coordinate data of the skeletal keypoints is used... The parametric human body model SMPL is used to reconstruct the user's 3D body model. This process is achieved through an optimization algorithm. The goal is to find a set of pose parameters (representing the relative rotation of 24 body joints) and body shape parameters (representing the first 10 principal component coefficients of body height, weight, and build) of the SMPL model such that the sum of the Euclidean distances between the 33 corresponding keypoint positions of the SMPL model generated by this set of parameters and the 33 3D coordinate data predicted by the previous step network is minimized. This optimization process uses the L-BFGS algorithm and converges within 20 iterations, thereby generating a user 3D body model that highly matches the user's current pose and body shape.

[0043] Based on the user's 3D body model, the system retrieves and extracts the corresponding 3D model and resting state mesh data of the target garment from the e-commerce product database according to the product ID currently viewed or selected by the user. The 3D model is stored in FBX format and includes vertex coordinates, normals, UV coordinates, and mesh topology. The resting state mesh data specifically refers to the original set of vertex coordinates of the garment on a standard T-pose or A-pose mannequin without any deformation. This data serves as the basis for subsequent deformation calculations. Next, the 3D model of the garment is superimposed on the user's 3D body model through coordinate system transformation. First, coordinate system alignment is performed, setting the origin of the user's 3D body model's coordinate system at the center of its pelvis, with the Y-axis pointing upwards and the Z-axis pointing forwards. Then, an initial rigidity transformation is performed on the 3D model of the garment. This transformation is achieved by aligning several predefined anchor points on the garment model with the corresponding skeletal key points on the user's body model. For example, the center points of the left and right shoulders, the center point of the neckline, and the center point of the waist of the garment model are aligned with the user's body... The model's four key points—left and right shoulder joints, the base of the neck, and the center of the pelvis—are aligned. The optimal rotation matrix and translation vector are calculated by solving a least-squares problem. Specifically, singular value decomposition (SVD) is used to decompose the covariance matrix of two sets of corresponding points, resulting in a rigid transformation matrix that minimizes the sum of squared distances between all corresponding anchor points. This matrix is ​​applied to all vertices of the clothing 3D model to complete the initial alignment and adjust the relative position of the clothing 3D model and the user's 3D body model. To address the issue of mismatch between different clothing sizes and user body shapes, a non-uniform scaling adjustment is performed in this step. The user's body model's dimensions in height, shoulder width, and waist circumference are calculated and compared with the default model size of the clothing model to obtain the scaling ratio in the three axes. For example, if the user's shoulder width is 45cm, while the default shoulder width of the clothing is 42cm, the scaling ratio in the X-axis direction is 1.07. This non-uniform scaling transformation is applied to the clothing model to make its overall dimensions closer to the user's body shape, forming a preliminary superimposed model.

[0044] Based on the initial overlay model, the current pose of all 24 joints in the user's 3D body model is first obtained. This pose information is represented in the form of a series of local rotation matrices or quaternions, defining the relative orientation from parent joint to child joint. For example, the rotation matrix of the elbow joint defines the bending and twisting state of the forearm relative to the upper arm. These pose parameters have been optimized and determined in the step of generating the user's 3D body model. Next, the pose of each key positioning point of the clothing 3D model is synchronized to the pose of the corresponding skeletal key points of the user's 3D body model. This process is implemented using linear hybrid skinning technology. When the clothing 3D model is created, a set of skinning weights is pre-calculated and stored for each vertex. This set of weights defines the degree of influence of each skeletal joint of the user's body model on the position of that vertex. The calculation of skinning weights is based on generalized centroid coordinates or heat diffusion methods to ensure that the weight distribution is smooth and intuitive. For example, for a vertex at the cuff of a T-shirt, its skinning weight will be mainly allocated to the wrist and elbow joints, while for a vertex at the shoulder, its weight is mainly determined by the shoulder joint and clavicle. The specific weights are set as a vector W. v =[w v,1 ,w v,2 ,...,w v,24 ], where w v,j Let be the weight of the influence of the j-th joint on vertex v, and The synchronization process iterates through each vertex of the clothing 3D model, calculating its new spatial position based on its skin weights and the joint pose of the user's body model. For any vertex v, its coordinates in the resting state are p. v Its new coordinates p′ after skinning v Calculated using the following formula: Among them, T j It is the global transformation matrix from the root node of the model to the j-th joint. This matrix is ​​obtained by multiplying the local rotation matrices of each joint in the user's current pose. By performing this calculation on all vertices of the clothing model, the clothing mesh as a whole deforms along with the movement of the user's body skeleton, thus forming the initial three-dimensional model of the wearing state.

[0045] The steps for obtaining the multidimensional geometric deviation value are as follows:

[0046] Based on the initial wearing state 3D model, extract the node coordinates of all mesh edges in the clothing 3D model, traverse each mesh edge and calculate the Euclidean distance between two nodes in the current frame state and the distance in the corresponding resting state, obtain the relative change rate of each mesh edge, and generate a set of clothing mesh edge length change ratios.

[0047] Based on the initial wearing state 3D model, the normal vector direction of each mesh surface is obtained, and a ray is emitted in the opposite direction of the normal vector to intersect with the surface of the user's 3D body model. The distance from the ray to the intersection point is recorded as the gap distance value. Combined with the set of clothing mesh side length change ratios, a multidimensional geometric difference set is formed.

[0048] Specifically, based on the initial 3D model of the clothing's wearing state, a complete list of edges is first extracted from the model's topological data structure. This list contains all the mesh edges that constitute the clothing's 3D model and records the indices of the two vertices connected to each edge. Subsequently, the system initiates a traversal process, processing each mesh edge in the list one by one. For the i-th edge being processed, the system retrieves the 3D coordinates of the two endpoints in the current frame from the initial 3D model of the clothing's wearing state based on its vertex indices, denoted as V. i1,current and V i2,current And by using the method for calculating the distance between two points in three-dimensional space, that is, calculating the magnitude of the difference vector between the coordinates of the two points, the length L of the edge in the current pose can be obtained. i Meanwhile, using the same vertex index, the system retrieves the three-dimensional coordinates of these two endpoints in their resting state without any deformation from the clothing resting state mesh data extracted during loading, denoted as V. i1,rest and V i2,rest And its resting length R was calculated in exactly the same way. i , obtain L i and R i Then, the system calculates (L) i -R i ) / R i To obtain the relative rate of change of this edge, a positive value indicates that the edge is stretched, a negative value indicates that it is compressed, and a zero value indicates that the length remains unchanged. This calculation process is repeated for all N mesh edges in the clothing 3D model until the relative rate of change of all edges has been calculated. Finally, all the calculated relative rate of change values ​​are stored together with their corresponding edge indices to form a key-value pair set, where the key is the edge index and the value is the relative rate of change of that edge. This set is the set of clothing mesh edge length change ratios.

[0049] Based on the initial 3D model of the clothing in its wearing state, the system begins to traverse each mesh face in the clothing 3D model, usually a triangular face. For the currently processed mesh face, its normal vector direction is first obtained. This calculation is performed by selecting two edge vectors of the triangular face, such as vector E1 = V2 - V1 and vector E2 = V3 - V1, and calculating their cross product N. face=E1×E2, and then the resulting vector is normalized to obtain a unit normal vector, which points to the outer side of the clothing surface. Next, the system emits a virtual ray from the geometric center point of the mesh surface in the opposite direction of the normal vector, that is, towards the inner side of the user's body. In order to efficiently calculate the intersection point of the ray with the surface of the user's 3D body model, a bounding box hierarchical tree acceleration structure has been pre-built when the user's 3D body model is loaded. The ray intersection calculation first performs an intersection test with the root node of the BVH tree, and then recursively tests with the child nodes until the nearest intersection point with the mesh surface of the body model is found. The system records the distance from the ray emission point to this intersection point and uses this distance value as the corresponding value of the mesh surface. If a ray fails to intersect with the user's 3D body model, such as at a loose cuff or hem, the system marks the gap distance value at that location as a preset maximum value, such as 50 centimeters, to indicate that this is a completely open space. After calculating the gap distance values ​​for all mesh faces, the system integrates these gap distance values ​​with the set of clothing mesh side length change ratios generated in the previous step to create a unified data structure. This structure associates each mesh face with its own gap distance value and the side length change rate of all the edges that make up that face, thus forming a comprehensive dataset containing multi-dimensional geometric information such as stretching, compression, and gaps, i.e., a multi-dimensional geometric difference set.

[0050] The steps to obtain the global quantitative fit index are as follows:

[0051] Based on the multidimensional geometric difference set, the global quantitative fit index is calculated using the following formula:

[0052]

[0053] Among them, F fit To quantify the fit index globally, To normalize the tension index, The normalized porosity index, The normalized alignment index; where N is the total number of grid edges, L i Let R be the current length of the i-th edge. i Let G be the resting length of the i-th edge, P be the number of sampling points for the gap distance, and G be the distance between the i-th edge and the i-th edge. j S is the gap distance measured at the j-th point. j Let D be the ideal gap distance at the j-th point. ref For a fixed reference length constant, M is the total number of semantic line-skeletal line pairs, and θ k Let be the angle in radians between the k-th vector pairs, λ be the alignment sensitivity constant, and w be the angle between the vector pairs. Gt ,w Gg ,w A The normalized weighting coefficients for each sub-index satisfy w Gt +wGg +w A =1.

[0054] Specifically, the formula: The advantage of the formula is that it assesses three core physical dimensions of clothing fit: tightness, gaps, and alignment, avoiding the one-sidedness of a single indicator. Tightness E T This reflects the stretching and compression of the clothing material, directly related to the feeling of restriction when wearing it; porosity E G It quantifies the gap between clothing and the body, which relates to the looseness and silhouette of the garment; Alignment E A This assesses whether the key structural lines of the garment are aligned with corresponding parts of the human body, affecting visual regularity and harmony. The formula uses a weighted square root form, structurally similar to a Euclidean distance in the "fit feature space," which makes the final global quantitative fit index F... fit It becomes an intuitive, non-negative scalar; the smaller its value, the better the overall fit. In addition, a weighting coefficient w is introduced. Gt w Gg and w A This gives the model great flexibility, allowing it to dynamically adjust the importance of various indicators based on the characteristics of different clothing categories (such as bodysuits and casual jackets), thereby achieving accurate and personalized assessments of the fit of various types of clothing.

[0055] N represents the total number of mesh edges. This parameter is directly read from the topology of the 3D model of the target garment and represents the total number of edges that make up the garment model. It defines the overall sample size for the tightness calculation, and its value is determined by the complexity and detail of the garment model. A simple T-shirt model contains 5,000 edges, while a complex jacket may have more than 20,000 edges. This parameter does not need to be calculated. During the model loading stage, the number of elements can be directly obtained by accessing the edge list data structure of the 3D model data file (such as FBX or OBJ file). For example, in a fitting simulation, a shirt model with medium complexity of 12,345 vertices and 24,690 faces is loaded. According to Euler's formula (V-E+F=2-2g, for closed surfaces), the number of edges can be estimated, or the total number of edges N=37035 can be obtained directly by parsing the model file.

[0056] L i The current length of the i-th edge is the actual spatial length of the i-th edge on the clothing mesh after the clothing 3D model is superimposed on the user's 3D body model and deformed by skinning. It is obtained by first locating the two endpoints of the i-th edge and then reading the 3D coordinates p of these two endpoints in the world coordinate system from the initial wearing state 3D model. i1 =(x1,y1,z1) and pi2 = (x2, y2, z2), and then the Euclidean distance between the two points is calculated. The calculation process is to calculate the square root of the sum of the squares of the differences between the coordinate components of the two points. This process is recalculated for all edges after each frame of user pose update to reflect the real-time clothing deformation state. For example, for an edge of the shoulder of the clothing, the coordinates of its two endpoints in the current frame are (15.5, 140.2, -5.1) cm and (16.0, 140.8, -5.3) cm, respectively. Then its current length L i for

[0057] R i Let L be the resting length of the i-th edge. This parameter represents the length of the garment in its original, natural state without any wearing deformation, i.e., the design length on a standard T-pose mannequin or in a flat lay state. This value is obtained from the loaded garment resting state mesh data. For the i-th edge, its acquisition method is the same as calculating L. i They are exactly the same, the difference being that the vertex coordinates used are derived from the resting state coordinate data p′. i1 and p′ i2 This value is fixed for a given garment and serves as the benchmark for all deformation calculations. For example, for the aforementioned shoulder edge, its resting coordinates might be (18.2, 138.5, -4.0) cm and (18.9, 138.6, -4.1) cm, then its resting length R i for

[0058]

[0059] P represents the number of gap distance sampling points. This parameter represents the total number of points on the inner surface of the garment used to measure the gap between the garment and the user's body. In this method, the center of each mesh face (usually a triangle) is considered as a sampling point. Therefore, the value of P is equal to the total number of faces in the garment's 3D model. This value is also read directly from the topology data of the garment's 3D model. Its magnitude reflects the spatial resolution of the porosity measurement. The more faces there are, the denser the sampling is, and the more accurate the porosity assessment is. For example, for the aforementioned shirt model with 24,690 faces, the number of gap distance sampling points P is 24,690.

[0060] G jLet G be the gap distance measured at the j-th point. This parameter is the distance traveled by ray emanating from the center point of the j-th clothing mesh surface in the initial wearing state 3D model, along the opposite direction of the surface's normal vector, until it intersects with the surface of the user's 3D body model. It directly reflects the actual gap size between the clothing and the body at a specific local location. Obtaining this value relies on an efficient ray-mesh intersection detection algorithm. For example, in the user's abdominal region, if the distance measured from the center point of a certain mesh surface of a shirt to the skin below is 1.2 cm, then the corresponding G... j The value is 1.2 centimeters.

[0061] S j The ideal gap distance at point j is a parameter that is preset by the fashion designer for a specific garment at a specific location, representing the ideal gap value that best matches the design intent. It is stored as metadata along with the garment's 3D model, existing as a 2D texture map (i.e., the "ideal gap map"). Each pixel in the map is color-coded with the ideal gap value corresponding to its location on the garment surface. During calculation, the system maps the UV coordinates of the j-th sampling point onto this map, queries and decodes it to obtain the corresponding S... j For example, for a slim-fit shirt, the ideal gap in the torso area is set to 0.5 cm, while the elbow movement area might be set to 1.5 cm. If the j-th sampling point is located in the torso, then its S... j The value is 0.5 cm.

[0062] D ref To maintain a fixed reference length, this parameter is a constant used to normalize the porosity index. Its purpose is to eliminate the dimensional influence of porosity distance values ​​at different scales, making the porosity index a dimensionless relative value. This allows for meaningful comparisons and combinations with other indices. D ref The settings are based on statistical analysis of standard human body dimensions and are usually set to values ​​equivalent to the main horizontal dimensions of the human body. A reasonable setting method is to take the average shoulder width of a standard adult. For example, according to anthropometry data, the average shoulder width of an adult male is about 40 cm and that of a female is about 36 cm. The system can set a general reference value D. ref = 40 centimeters.

[0063] M represents the total number of semantic line-skeletal line pairs. This parameter indicates the number of key feature lines used to evaluate alignment. Semantic lines are key structural lines pre-annotated by the designer on the 3D model of the garment, such as shoulder lines, side seams, and waistlines. Skeletal lines are line segments on the user's 3D body model formed by connecting specific skeletal joints, such as the line connecting the two shoulder joints or the line connecting the shoulder joint to the hip joint. The system pairs the two according to preset matching rules, and M is the total number of successfully paired pairs. For example, for a top, the designer may define 4 key semantic lines: left and right shoulder lines and left and right side seams, which are paired with the user's left and right clavicle lines and the lines connecting the left and right shoulder joints to the hip joints, respectively. In this case, M = 4.

[0064] θ k V represents the angle in radians between the k-th pair of vectors. This parameter is the angle between the matched semantic line vector and the skeletal line vector, used to quantify the directional deviation between them. The calculation process is as follows: First, obtain the three-dimensional coordinates of the two endpoints of the semantic line in the initial wearing state and calculate the direction vector V. semantic Then, obtain the three-dimensional coordinates of the two ends of the corresponding skeletal line and calculate the direction vector V. bone Finally, the cosine of the included angle is calculated using the dot product formula. Then, the included angle θ is obtained using the inverse cosine function. k The radian value, for example, if the angle between a lateral suture vector and the corresponding torso vector is 10 degrees, then

[0065] λ is an alignment sensitivity constant, which acts as a regulating factor in the calculation of the alignment index, controlling the degree of alignment index change with the included angle θ. k The rate at which the alignment index decreases as the angle increases represents the severity of the penalty for deviation. The larger the value of λ, the more significant the decrease in the alignment index will be, even for small angular deviations. Conversely, a smaller value indicates greater tolerance for deviations. This value is calibrated through subjective fit evaluation experiments on a large number of users. Data on users' tolerance for different alignment deviations is collected, and an optimal value is obtained through fitting. In practice, setting λ = 5.0 can achieve a good balance between the perceived accuracy of alignment deviations and numerical stability. When the angle is 0, the index is 0, and when the angle is close to π / 2 (90 degrees), the index is close to 1.

[0066] w Gt w Gg w AThese are the normalized weight coefficients corresponding to each sub-index. These three coefficients represent the proportion of importance of tightness, gaps, and alignment in the final global quantitative fit index, and their sum must be 1. The setting of these weights is not fixed but strongly correlated with the category and style attributes of the clothing. The system maintains a "clothing category-weight" mapping table, which is jointly developed by fashion experts and data analysts. For example, for a "tight-fitting stretch sports top," the key to a good fit is a close fit and minimal wrinkles; therefore, the weight is set to w. Gt =0.6,w Gg =0.3,w A =0.1, while for a "loose-fitting trench coat", its design itself requires a larger gap, and the alignment of structural lines is particularly important, so the weight may be set to w. Gt =0.1,w Gg =0.4,w A =0.5. During the fitting process, the system first reads the clothing category from the product information and then queries the table to obtain the corresponding weight coefficient value.

[0067] Calculation process:

[0068] Taking trying on a "slim-fit shirt" as an example, the system retrieves its category attribute from the product database and obtains the corresponding weight coefficient from the weight mapping table: tightness weight w Gt =0.5, porosity weight w Gg =0.3, alignment weight w A =0.2.

[0069] Calculate the normalized tension index E T :

[0070] After traversing all N = 37035 grid edges, the average absolute value of the relative rate of change of all edge lengths was calculated, resulting in...

[0071] Therefore, E T =0.08.

[0072] Calculate the normalized porosity index E G :

[0073] After traversing all P = 24690 sampling points, the square of the difference between the gap distance and the ideal gap distance is calculated, divided by the square of the reference length constant, and then the mean is obtained. Where D ref = 40 centimeters.

[0074] therefore,

[0075] Calculate the normalized alignment index EA :

[0076] For M = 4 pairs of semantic line-skeleton lines, calculate the included angle penalty term for each pair and then average it. For example, if the four pairs of included angles are 5, 8, 6, and 12 degrees, convert them to radians and substitute them into the formula. (where λ = 5.0), the four calculated values ​​are 0.0038, 0.0098, 0.0055, and 0.0215.

[0077] Their average value is

[0078] Calculate the final global quantized fit index F fit :

[0079] Substitute the sub-indices and their weights obtained from the above calculations into the final formula:

[0080]

[0081] The result indicates that the global quantitative fit index of this slim-fit shirt in the user's current posture is 0.0656. This value is a comprehensive and quantitative fit score. The lower the value, the better the fit. The system interprets the value according to preset rating standards. For example, [0,0.05) is "perfect", [0.05,0.1) is "good", [0.1,0.2) is "average", and ≥0.2 is "not good". The current calculation result of 0.0656 falls in the "good" range, indicating that the shirt fits relatively well overall.

[0082] The steps for obtaining the visualization mapping rule set are as follows:

[0083] Based on the initial wearing state 3D model, extract all mesh edge indices connected to each vertex and the current length and resting length of the corresponding edge, calculate the relative stretch value of each edge, and extract the gap distance and ideal gap distance of the mesh face adjacent to each vertex to construct the stretch set and gap set of each vertex.

[0084] Based on the stretched set and the gap set, the local fit index of each vertex is calculated using the following formula:

[0085]

[0086] Among them, F v E is the local fit index of vertex v. v Let |E| be the set of edges connected to vertex v. v | is the size of the set, L i R is the current length of the i-th edge. i Let G be the resting length of the i-th edge.f(v) S is the actual gap distance corresponding to the face adjacent to vertex v. f(v) D is the ideal gap distance corresponding to this face. ref As a reference distance constant, w Lt w Lg This is the local combination coefficient of the tension term and the porosity term;

[0087] Based on the local fit index, the local fit index is mapped to the final visualized color value and assigned to the corresponding vertex, generating a visual mapping rule set. The formula for calculating the final visualized color value is as follows:

[0088]

[0089] Among them, C v C represents the final visual color value of vertex v. min With C max The color mapping range boundary is γ, the color perception adjustment factor is F. min F max These are the minimum and maximum values ​​of the local fit index for all vertices.

[0090] Specifically, based on the initial 3D model of the clothing in its wearing state, the system initiates a parallel computing task, traversing each vertex in the clothing 3D model. For the currently processed vertex v, it first queries the model's topological adjacency table to obtain a list containing the indices of all mesh edges directly connected to that vertex. Subsequently, for each edge index in this list, the system performs a data extraction operation. Specifically, for the edge with index i, the system directly searches for and reads the length L of that edge in the current frame from the already calculated and cached set of clothing mesh edge length change ratios. i Length R in the resting state i Next, the system calculates the relative stretch value of the side based on these two length values, using the calculation method (L... i -R i ) / R i This calculation is repeated for all edges connected to vertex v, and all the obtained relative stretch values ​​are collected to form a numerical list, which is the stretch set of vertex v. At the same time, the system also needs to extract the void information related to vertex v. The system queries the model's topological adjacency table again to obtain the index list of all adjacent mesh faces shared with vertex v. For each face index j in the list, the system searches for and reads the actual void distance G corresponding to that face from the multidimensional geometric difference set generated in the previous step. j and ideal gap distance S jTo assign a single gap metric to vertex v, the system performs a weighted average of the actual gap distance and the ideal gap distance of all adjacent faces. The weights are determined based on the proportion of each adjacent face's area to the total adjacent face area. This yields an average actual gap distance and an average ideal gap distance representing the local region of vertex v. These two averages are used as the gap set of vertex v. After traversing all vertices, each vertex will be associated with a stretch set containing its surrounding deformation information and a gap set representing its local spatial relationships.

[0091] formula: The advantage of this formula lies in its comprehensive reflection of the local fit by integrating two orthogonal physical dimensions: local stretching and local voids. The stretching term, expressed as root mean square, effectively amplifies and penalizes areas of excessive stretching or compression, while the void term directly measures the deviation from the ideal design profile by introducing a variable local combination coefficient w. Lt and w Lg This allows the formula to adapt to clothing with different fabric properties. For example, for elastic fabrics, the weight of the stretch term can be reduced, while for stiff fabrics, its weight can be increased. This is a refined, configurable local evaluation mechanism.

[0092] E v The set of edges connected to vertex v is dynamically constructed by querying the topological data structure (e.g., adjacency list or half-edge data structure) of the garment's 3D model when processing each vertex. It contains unique identifiers or indices for all edges with vertex v as one endpoint. The size and content of this set directly reflect the local connectivity of the vertex in the mesh and are the basis for calculating local stretching deformation. For example, in a mesh region composed of regular quadrilaterals, an interior vertex typically connects to four edges, while in a mesh composed of triangles, the number of edges (i.e., degrees) connected to an interior vertex is typically 6. In this calculation, a vertex v located at the shoulder of the garment is selected. By querying its adjacency information, it is found to be connected to 6 edges; therefore, its edge set E is... v It contains the indices of these 6 edges.

[0093] |E v | represents the size of the set, this parameter is set E. v The number of elements in the equation, i.e., the total number of edges connected to vertex v, also known as the degree of the vertex, is directly obtained by querying E. v The number of elements is then calculated, and this value is used as the denominator in the formula to average the degree of tensile deformation of all connected edges. This yields a standardized tensile index that represents the average stress on the vertex, avoiding calculation errors caused by different vertex connectivity. For the previously selected shoulder vertex v, which is connected to 6 edges, |Ev |=6.

[0094] L i is the current length of the i-th edge. This parameter value comes from the analysis results of the initial wearing state 3D model in the previous step. It is the actual spatial length of the mesh edge i under the user's current posture. The system has already calculated and cached this value for all edges when calculating the global quantized fit index. In this step, for set E v For each edge i in the algorithm, its corresponding L is read directly from the cache. i The values ​​are sufficient and there is no need to calculate them repeatedly. For example, for the 6 edges connected to the vertex v of interest, the current lengths read from the cache are: L1 = 0.85, L2 = 0.88, L3 = 0.92, L4 = 0.86, L5 = 0.89, L6 = 0.91 (unit: centimeters).

[0095] R i Let be the resting length of the i-th edge. This parameter also comes from the previous steps and represents the original design length of the grid edge i in the natural state of the garment when not worn. It is extracted and cached from the garment's resting state grid data. When calculating the local fit index, for set E... v For each edge i in the system, the system reads its corresponding fixed value R from the cache. i For example, the resting lengths of the six edges connected to vertex v are: R1 = 0.80, R2 = 0.82, R3 = 0.90, R4 = 0.81, R5 = 0.84, and R6 = 0.85 (unit: centimeters).

[0096] G f(v) This represents the actual gap distance between the face adjacent to vertex v. Since a vertex is usually adjacent to multiple faces, G here... f(v) Defined as the weighted average of the actual void distances of all mesh faces adjacent to vertex v, with the weights being the area of ​​each face. This is intended to more accurately reflect the average void situation around the vertex, and its calculation formula is as follows: Where F v G is the set of faces adjacent to vertex v. j and A j These are the actual gap distance and area of ​​face j, respectively. These values ​​have been calculated and cached in previous steps. For example, vertex v is adjacent to 6 triangular faces, and its corresponding actual gap distance and area are (0.8cm, 0.3cm, ..., 0.3cm). 2 (0.9cm, 0.32cm) 2 (0.85cm, 0.31cm) 2 (0.95cm, 0.33cm) 2 (1.0cm, 0.34cm) 2(0.92cm, 0.32cm) 2 ), calculate G f(v) ≈0.906 cm.

[0097] S f(v) For the ideal gap distance corresponding to this face, and G f(v) The calculation method is similar, S f(v) Defined as the weighted average of the ideal gap distances of all mesh faces adjacent to vertex v, with the weights also being the area of ​​each face, calculated using the following formula: Where S j It is obtained from the ideal gap pattern texture of the garment. For example, for the 6 faces adjacent to vertex v, the ideal gap distance obtained is 1.0 cm (the shoulder area design is more fitted), therefore its weighted average S f(v) = 1.0 cm.

[0098] D ref As a reference distance constant, this parameter is completely consistent with the definition in the global quantized fit index. It is used to normalize the gap term to eliminate the influence of dimensions. Its value is set according to standard anthropometry data, and here we continue to use the value set in the previous steps, i.e., D. ref = 40 centimeters.

[0099] w Lt With w Lg This is a local combination coefficient for the stretch and gap terms. The sum of these two coefficients is 1, used to balance the importance of local stretch and gap in calculating the local fit index. Their values ​​are dynamically determined based on the fabric properties of that local area of ​​the garment. Specifically, the system reads a "material property map" bound to the garment model. The map uses colors to encode the elasticity coefficient of the fabric (between 0 and 1, where 1 is perfectly rigid and 0 is infinitely elastic). For vertex v, the system queries the elasticity coefficient value e at its corresponding position on the material map. v Then set w Lt =e v And w Lg =1-e v For example, if the fabric in the shoulder area of ​​interest has a certain degree of elasticity, its elastic coefficient value e can be obtained by querying the database. v =0.4, therefore set w Lt =0.4, w Lg =0.6.

[0100] Calculation process:

[0101] Calculate the stretch term:

[0102] First, calculate the square of the stretch ratio for each edge:

[0103]

[0104] Find their average value:

[0105] Calculate the void term:

[0106]

[0107] Calculate the final local fit index F v :

[0108] The results show that at the selected shoulder vertex v, the local fit index is 0.0384. This value quantifies the micro-fit at that point. The low value indicates that the clothing around this point is neither too tight nor too far from the ideal design gap, and belongs to the area of ​​good fit. This calculation will be performed on all vertices of the clothing model to obtain a complete local fit index field.

[0109] formula: The advantage of the formula is that the introduced color perception adjustment factor γ makes the mapping process nonlinear, which is consistent with the human visual system's perception of color changes. By adjusting γ, the areas where the user is most concerned about fit can be highlighted (for example, setting γ to a value greater than 1 can amplify the red display of high-pressure areas), thereby transforming complex data analysis results into a fit heatmap that users can understand at a glance, improving the practicality of virtual try-on and user experience.

[0110] C min With C max These two parameters define the two ends of the color gradient range of the fit heatmap, representing the boundaries of the color mapping range. They are RGB color values ​​pre-set in the system configuration, and their selection follows general principles of color psychology and information visualization. Typically, C... min Represents the most ideal fit (i.e., F) v The smallest value (often set to cool colors, such as blue, indicating comfort and stress-free, while C...) max Represents the least ideal fit (i.e., F) v The maximum value (often set to a warm color, such as red, to indicate tension and stress) is used in this application. In this case, C is set to... min It is pure blue, with an RGB value of (0.0, 0.0, 1.0), and C is set to... max It is pure red, and its RGB value is (1.0, 0.0, 0.0).

[0111] F vF is the local fit index of vertex v. This parameter is a direct output of the previous step and represents the microscopic fit of vertex v. In this formula, it serves as input, determining the specific color of the vertex on the final heatmap. For the shoulder vertex v calculated in the previous step, its calculation result is substituted into the calculation here, i.e., F. v =0.0384.

[0112] F min With F max These are the minimum and maximum values ​​of the local fit index for all vertices. These two parameters represent the local fit index (F) of the clothing model for all vertices within the current frame. v After calculation, obtained through a single global traversal, the system iterates over the F values ​​of all vertices. v The system records and updates the minimum and maximum values ​​encountered. These two values ​​define the actual dynamic range of the fit index under the current pose and are key to normalization. For example, after calculating all tens of thousands of vertices of the entire shirt, the system finds the F value of the current frame... v The value range is [0.012, 0.155], therefore, F is set to... min =0.012, F max =0.155.

[0113] γ is a color perception adjustment factor. This parameter is used to adjust the non-linearity of color interpolation to better match human visual perception or highlight specific information. When γ = 1.0, the color interpolation is linear; when γ > 1.0, it makes the midtones more biased towards C. min Thus, getting closer to C max The color areas appear more prominent and concentrated, and vice versa. This value is set based on experience in improving visual readability and is optimized through user testing and designer feedback. Usually, 2.2 is a commonly used value because it is close to the gamma value of the standard sRGB color space and can produce a more natural visual transition. In this application, γ is set to 2.2.

[0114] Calculation process:

[0115] Calculate the normalization factor:

[0116] First, calculate the normalized position of the local fit index of vertex v in its global domain:

[0117]

[0118] Apply color perception adjustment factor:

[0119] Apply gamma correction to the normalization factor: gamma_corrected_factor = (norm_factor)γ =(0.1846) 2.2 ≈0.0268;

[0120] Perform linear color interpolation:

[0121] Using the corrected factor in C min and C max Interpolation is performed between the values ​​to calculate the final RGB color value C. v This calculation is performed independently for the R, G, and B channels: C v =C min +(C max -C min )·gamma_corrected_factorC v (R)=C min (R)+(C max (R)-C min (R))·0.0268=0.0+(1.0-0.0)·0.0268=0.0268, C v (G)=C min (G)+(C max (G)-C min (G))·0.0268=0.0+(0.0-0.0)·0.0268=0.0, C v (B) = C min (B)+(C max (B)-C min (B))·0.0268=1.0+(0.0-1.0)·0.0268=1.0-0.0268=0.9732, therefore, the final visual color value C of vertex v is... v It is RGB(0.0268,0.0,0.9732).

[0122] The result shows that the final visual color value of vertex v is a color very close to pure blue, with a slight red component, which intuitively reflects its local fit index F. v =0.0384 is very close to the minimum value in the global range [0.012, 0.155], which is a region with very good fit. This calculation process will be applied to all vertices, and each vertex will get a unique RGB color value. The set of these (vertex index, color value) pairs eventually constitutes the visualization mapping rule set for the next stage of rendering.

[0123] The steps to obtain the vertex rendering instruction set are as follows:

[0124] Based on the set of visual mapping rules, the local fit index of each vertex of the clothing 3D model is read one by one, and the local fit index is mapped to the corresponding color value, transparency value and highlight attribute label. The color value, transparency value and highlight attribute label are recorded one by one in vertex order to generate a vertex rendering instruction set.

[0125] Specifically, based on the visualization mapping rule set, the system initiates a data transformation process. This process reads the records generated for each vertex of the clothing 3D model from the rule set one by one. Each record contains the vertex index, its corresponding final visualized color value, and the original local fit index used to calculate that color value. According to a preset mapping function, the system further converts the local fit index of each vertex into transparency and highlight attribute labels. For the mapping of transparency values, the system sets a "focus threshold" for the local fit index. This threshold is dynamically determined by statistically analyzing the distribution characteristics of all local fit indices in the current frame. Specifically, it is set to the average of all index values ​​plus 1.5 times the standard deviation. When the local fit index of a vertex is lower than this focus threshold, it indicates that the fit of that area is good, and the system sets its transparency value to 1.0, meaning it is completely opaque. Transparency: When the index exceeds the attention threshold, it indicates a potential fit problem in that area. The system sets its transparency to 0.5, making it semi-transparent, allowing users to see the body outline beneath the clothing. For the mapping of highlight attribute labels, the system sets a "severity threshold," which is the critical value of the top 5% of all local fit index values, i.e., the 95th percentile. If a vertex's local fit index exceeds this severity threshold, its highlight attribute label is set to "highlighted," otherwise it is set to "normal." Subsequently, the system stores the newly generated color value (directly from the visualization mapping rule set), transparency value, and highlight attribute label of each vertex in the original vertex index order of the clothing 3D model into a dedicated data structure, forming a compact and ordered instruction list. This list is the vertex rendering instruction set.

[0126] The steps to obtain the dynamic fit texture are as follows:

[0127] Based on the vertex rendering instruction set, all vertex records in the vertex rendering instruction set are traversed. The color value and transparency value in each vertex record are assigned to the corresponding vertex in the initial wearing state 3D model. The target number of adjacent vertices are extracted around the vertex with the highlight attribute label as highlight. The coordinate information of the adjacent vertices is used to draw continuous streamlines and highlight areas to form a dynamic fit texture.

[0128] Specifically, based on the vertex rendering instruction set, the graphics rendering pipeline begins processing the rendering task for the current frame. First, it initiates a traversal process, which sequentially reads each vertex record in the vertex rendering instruction set. For the currently read record, the system extracts the color and opacity values ​​and directly assigns these values ​​to the attribute buffer of the vertex at the corresponding index in the initial dressed-up 3D model. Specifically, the color value (a 3D or 4D vector) is written to the vertex color buffer, and the opacity value is written to the alpha channel of the color buffer. After assigning all vertex attributes, the rendering pipeline then processes the highlighting effect. It again traverses the vertex rendering instruction set, filtering out all vertices with the highlight attribute label "highlighted." For each vertex marked as highlighted, the system... Centered on the model, the system extracts adjacent vertices within two concentric rings by querying the model's adjacency topology information. The target number is typically set to 12 to 18 adjacent vertices. After collecting the 3D coordinate information of these adjacent vertices, the system uses the Catmull-Rom spline interpolation algorithm to generate a smooth, closed curve between these coordinate points. This curve is the outline of the highlighted area. At the same time, the system uses dynamic textures to map a preset, animated flowing light texture onto the clothing surface area surrounded by this outline. This flowing light texture moves along a specific direction at a certain speed, thereby creating a visually flowing light effect. By combining vertex colors, transparency, and the drawn flowing lines and highlighted areas, a dynamic fit texture is formed on the surface of the clothing model.

[0129] The steps to obtain an augmented reality fit heatmap are as follows:

[0130] Based on dynamic fit texture, the system captures user images in real time and calculates the user's real-time posture through augmented reality devices. It then adjusts the spatial position of the dynamic fit texture in the augmented reality view in real time. When the user's posture changes, it repeatedly acquires the quantized fit index, local fit index, and vertex rendering instruction set to update the dynamic fit texture in real time, generate an augmented reality fit heatmap, and provide wearing suggestions based on the global quantized fit index.

[0131] Specifically, based on the dynamic fit texture, the augmented reality system enters a real-time display and interaction loop. The system continuously captures real-world images of the user at a frequency of 30 frames per second using the augmented reality device's camera. Utilizing the device's built-in inertial measurement unit (IMU) and simultaneous localization and mapping (SLAM) technology, it accurately calculates the user's real-time position and posture in three-dimensional space. This real-time posture information is used to update a view transformation matrix, which is applied to the clothing model with the dynamic fit texture. This adjusts the spatial position of the dynamic fit texture in the augmented reality view in real time, ensuring it is accurately and stably superimposed on the user image captured by the camera. When the system detects a significant change in the user's posture, an update event is triggered. The trigger condition for this event is set to the average displacement of the three-dimensional coordinates of the user's major joints (such as shoulder, elbow, hip, and knee) over five consecutive frames. If the distance exceeds 2 centimeters, once the event is triggered, the system will repeat all the previous core calculation steps, namely, re-acquiring the global quantized fit index, the local fit index of all vertices, and generating a brand-new vertex rendering instruction set, thereby updating the dynamic fit texture in real time. This cyclical update process ensures that the fit heatmap displayed on the screen can respond instantly to every action of the user, ultimately generating an augmented reality fit heatmap that interacts with the user in real time. At the same time, the system will match and provide corresponding wearing suggestions from a preset suggestion library based on the latest calculated global quantized fit index. This suggestion library stores corresponding text suggestions according to different ranges of the global quantized fit index. For example, when the global quantized fit index is in the range of [0.1, 0.2), the system will display "The overall fit is average. It is recommended to try a smaller size or choose a more elastic fabric style for a better fit."

[0132] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A virtual try-on method for e-commerce products based on augmented reality, characterized in that, Includes the following steps: The system acquires real-time images of the user through the camera of the augmented reality device, identifies the user's skeletal joints and constructs a three-dimensional body model of the user. At the same time, it retrieves the three-dimensional model of the target clothing and the resting state mesh data from the e-commerce product database, and overlays the clothing three-dimensional model onto the user's three-dimensional body model to generate an initial three-dimensional model of the wearing state. Based on the initial three-dimensional model of the wearing state, the clothing mesh is traversed to establish multi-dimensional geometric deviation values. The tightness value, gap value and alignment value in the multi-dimensional geometric deviation values ​​are calculated to obtain the global quantitative fit index. Calculate the local fit index and set a color gradient from positive to negative values. Establish a visual mapping rule set. Based on the visual mapping rule set, convert the local fit index to the color, transparency and highlight attributes of each vertex on the clothing 3D model and establish a vertex rendering instruction set. The vertex rendering instruction set is read, and the defined color and transparency information are assigned to the corresponding vertices of the 3D model in the initial wearing state. Streamlines and highlight effects are drawn on the target surface area to obtain a dynamic fit texture. In the augmented reality view, the clothing model with the dynamic fit texture is superimposed on the user image in real time. The global quantized fit index, local fit index, and vertex rendering instruction set are updated cyclically as the user's posture changes. Wearing suggestions are given based on the global quantized fit index, and an augmented reality fit heatmap is generated.

2. The augmented reality-based virtual try-on method for e-commerce products according to claim 1, characterized in that, The steps for obtaining the initial three-dimensional model of the wearing state are as follows: The augmented reality device's camera collects user image information in real time, and uses image feature detection technology to identify skeletal key points in the user image information to obtain three-dimensional coordinate data of the head, torso, and limbs. The three-dimensional coordinate data of the skeletal key points are then used to reconstruct the user's three-dimensional body model, generating the user's three-dimensional body model. Based on the user's 3D body model, the clothing 3D model and clothing resting state mesh data corresponding to the target clothing are retrieved and extracted from the e-commerce product database. The clothing 3D model and the user's 3D body model are superimposed through coordinate system transformation, and the relative positions of the clothing 3D model and the user's 3D body model are adjusted to form a preliminary superimposed model. Based on the preliminary overlay model, the current pose of each joint of the user's three-dimensional body model is obtained, and the pose of each key positioning point of the clothing three-dimensional model is synchronized to the pose of the corresponding skeletal key points of the user's three-dimensional body model to form an initial wearing state three-dimensional model.

3. The virtual try-on method for e-commerce products based on augmented reality according to claim 1, characterized in that, The steps for obtaining the multidimensional geometric deviation value are as follows: Based on the initial wearing state 3D model, extract the node coordinates of all mesh edges in the clothing 3D model, traverse each mesh edge and calculate the Euclidean distance between two nodes in the current frame state and the distance in the corresponding resting state, obtain the relative change rate of each mesh edge, and generate a set of clothing mesh edge length change ratios. Based on the initial wearing state 3D model, the normal vector direction of each mesh surface is obtained, and a ray is emitted from the opposite direction of the normal vector to intersect with the surface of the user's 3D body model. The distance from the ray to the intersection point is recorded as the gap distance value. Combined with the set of clothing mesh side length change ratios, a multidimensional geometric difference set is formed.

4. The augmented reality-based virtual try-on method for e-commerce products according to claim 1, characterized in that, The steps for obtaining the global quantitative fit index are as follows: The global quantized fit index is calculated based on the multidimensional geometric difference set.

5. The virtual try-on method for e-commerce products based on augmented reality according to claim 1, characterized in that, The steps for obtaining the visualization mapping rule set are as follows: Based on the initial wearable state 3D model, extract all mesh edge indices connected to each vertex and the current length and resting length of the corresponding edge, calculate the relative stretch value of each edge, and extract the gap distance and ideal gap distance of the mesh surface adjacent to each vertex to construct the stretch set and gap set of each vertex. Based on the stretch set and the gap set, calculate the local fit index of each vertex; Based on the local fit index, the local fit index is mapped to the final visual color value and assigned to the corresponding vertex to generate a visual mapping rule set.

6. The virtual try-on method for e-commerce products based on augmented reality according to claim 1, characterized in that, The steps for obtaining the vertex rendering instruction set are as follows: Based on the aforementioned visualization mapping rule set, the local fit index of each vertex of the clothing 3D model is read one by one, and the local fit index is mapped to the corresponding color value, transparency value and highlight attribute label respectively. The color value, transparency value and highlight attribute label are recorded one by one in vertex order to generate a vertex rendering instruction set.

7. The virtual try-on method for e-commerce products based on augmented reality according to claim 1, characterized in that, The steps for obtaining the dynamic fit texture are as follows: According to the vertex rendering instruction set, all vertex records in the vertex rendering instruction set are traversed, and the color value and transparency value in each vertex record are assigned to the corresponding vertex in the initial wearing state 3D model. The target number of adjacent vertices are extracted around the vertex with the highlight attribute label as highlight, and continuous streamlines and highlight areas are drawn using the coordinate information of the adjacent vertices to form a dynamic fit texture.

8. The virtual try-on method for e-commerce products based on augmented reality according to claim 1, characterized in that, The steps for obtaining the augmented reality fit heatmap are as follows: Based on the dynamic fit texture, the augmented reality device captures the user's image in real time and calculates the user's real-time posture. The spatial position of the dynamic fit texture in the augmented reality view is adjusted in real time. When the user's posture changes, the quantized fit index, local fit index and vertex rendering instruction set are repeatedly acquired to update the dynamic fit texture in real time, generate an augmented reality fit heatmap, and provide wearing suggestions based on the global quantized fit index.

Images