Jewelry virtual try-on system based on multi-modal interaction and real-time dynamic tracking

Abstract

The application discloses a jewelry virtual try-on system based on multi-modal interaction and real-time dynamic tracking, and relates to the technical field of jewelry.The technical scheme points of the application include: acquiring multi-modal interaction data and article three-dimensional model data of a virtual try-on full link; constructing an article ontology adaptive model according to the article three-dimensional model data, and obtaining an article wearing fit coefficient by quantitatively evaluating the article ontology adaptive model; constructing a human body dynamic motion tracking model according to dynamic tracking data of the multi-modal interaction data, and obtaining a dynamic following stability coefficient by stably evaluating the human body dynamic motion tracking model; obtaining a try-on comprehensive management coefficient according to the article ontology comprehensive adaptive coefficient and a scene operation matching coefficient, outputting an operation control scheme according to the try-on comprehensive management coefficient, and achieving automatic intelligent scheduling of goods promotion strategies.

Description

Technical Field

[0001] This invention relates to the field of jewelry technology, and more specifically, to a virtual jewelry try-on system based on multimodal interaction and real-time dynamic tracking. Background Technology

[0002] The jewelry industry currently widely adopts online virtual try-on terminals and offline smart experience devices. However, existing virtual try-on technologies generally suffer from multiple technical shortcomings that are difficult to address simultaneously, failing to meet the three core requirements of simulation realism, dynamic tracking stability, and scene operation adaptability. Existing devices simply load a 3D jewelry model and overlay it with a static human body surface, without establishing a standardized jewelry adaptation evaluation process. This makes it impossible to quantify the fit between the jewelry and the body part being worn. Different styles of rings, bracelets, and necklaces have significant differences in surface contours, material optical parameters, and structural deformation characteristics. A uniform rendering standard would result in many pieces of jewelry exhibiting problems such as excessive gaps, surface misalignment, and distorted gemstone light and shadow. The static try-on images lack realism, and there is a lack of objective numerical evaluation to assess the quality of the jewelry's fit.

[0003] Meanwhile, traditional dynamic tracking modules only collect basic displacement data of human skeletal points, ignoring interference factors such as head micro-movements, limb rotation speed, and ambient light occlusion. They cannot build a complete human dynamic motion tracking model. When the human body makes large movements such as turning the head, raising the hand, or swinging the limbs, virtual jewelry is prone to drifting, delay, and distortion due to detachment from the skin. There are also no quantitative indicators such as dynamic tracking stability coefficients, making it impossible to distinguish the quality of tracking performance under different motion amplitudes. Static fit evaluation is completely separated from dynamic motion interference, and it is impossible to comprehensively evaluate the performance of jewelry in all scenarios. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a virtual jewelry try-on system based on multimodal interaction and real-time dynamic tracking.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A virtual jewelry try-on system based on multimodal interaction and real-time dynamic tracking includes: Acquisition module: Acquires multimodal interaction data and 3D model data of the items throughout the entire virtual try-on process; First evaluation module: Construct an item body adaptation model based on the item's 3D model data, and perform quantitative evaluation of the item body adaptation model to obtain the item's wearing fit coefficient; The second evaluation module: Construct a human dynamic motion tracking model based on the dynamic tracking data of multimodal interaction data, and evaluate the stability of the human dynamic motion tracking model to obtain the dynamic following stability coefficient; First processing module: Calculates the overall fit coefficient of the item based on the item's wearing fit coefficient and dynamic following stability coefficient; Generation module: Based on the object body adaptation model and the human body dynamic motion tracking model, a virtual try-on full-domain simulation model is generated through spatial coupling; The second processing module: acquires management requirement data from the operation side, and combines it with the virtual try-on full-domain simulation model to output rendering performance parameters to obtain the scene operation matching coefficient; Output module: Based on the overall compatibility coefficient of the item itself and the scene operation matching coefficient, the overall management coefficient of the trial wearing is obtained, and the operation control plan is output based on the overall management coefficient of the trial wearing.

[0006] Preferably, constructing an adaptation model of the object based on the object's 3D model data specifically includes the following steps: Establish a standard benchmark model for the material of the item in advance; Extract the contour dimensional features, material physical characteristics, and structural deformation characteristics of the three-dimensional model data of the product; A material adaptation model for jewelry is constructed by combining contour size features, material physical features, structural deformation features, and standard benchmark models of object materials.

[0007] Preferably, the fitting coefficient of the item is obtained by quantitatively evaluating the item's fit model, specifically including the following steps: Extract the wearing contact surface morphology characteristics and human wearing reference surface morphology characteristics corresponding to the product body adaptation model; By comparing the fit gap difference between the wear contact surface morphology and the human wear reference surface morphology and the degree of overlap of the curved surface, basic fit fit feature data are obtained. Based on the basic fit and fitting feature data, the critical fit area and the fully fit area are selected, and the fit difference attributes of different areas are distinguished based on the critical fit area and the fully fit area. The evaluation weight is obtained by weighting each fitting region according to the regional adaptation difference attribute. The overall wearing fit is evaluated based on the evaluation weights to obtain the item wearing fit coefficient.

[0008] Preferably, the physical characteristics of the material include the characteristics of precious metals and the optical refraction characteristics of gemstones.

[0009] Preferably, a human dynamic motion tracking model is constructed based on the dynamic tracking data of multimodal interaction data, specifically including the following steps: Establish a standard benchmark model for human body tracking in advance; Extract displacement features, micro-expression features, limb rotational angular velocity features, and environmental interference features from key points of the human skeleton in dynamic tracking data; By combining the displacement characteristics of key points of the human skeleton, micro-expression characteristics, limb rotational angular velocity characteristics, environmental interference characteristics, and human body tracking standard benchmark models, a human dynamic motion tracking model is constructed.

[0010] Preferably, the dynamic tracking stability coefficient is obtained by evaluating the stability of the human body dynamic motion tracking model, specifically including the following steps: Collect the continuous posture change trajectory generated by continuous human activity, extract the posture offset fluctuation information during the continuous posture change trajectory, and obtain the original characteristics of posture fluctuation. Generate interval fluctuation differentiation results based on the original characteristics of attitude fluctuations; By using interval fluctuation differentiation results to match the stability evaluation criteria corresponding to different activity amplitudes, a dynamic following stability coefficient is obtained.

[0011] Preferably, the overall fit coefficient of the item is obtained based on the item's wearing fit coefficient and dynamic following stability coefficient, specifically including the following steps: The dynamic deformation interference factor corresponding to the wearing fit coefficient is determined based on the dynamic tracking stability coefficient. The dynamic fit influence coefficient is obtained based on the fit coefficient and the dynamic deformation interference factor. The overall fit coefficient of the item is obtained based on the wearing fit coefficient and the dynamic fit influence coefficient.

[0012] Preferably, the management requirements data from the operations side are obtained, and the scene operation matching coefficient is obtained by combining the rendering performance parameters output by the virtual try-on full-domain simulation model. Specifically, this includes the following steps: Extract total passenger flow and interaction response rate from operational demand data; The time-period control weights for each time period are obtained based on the total passenger flow and the interaction response rate. The virtual try-on full-domain simulation model is used to output real-time rendering rate, total rendering volume, and dynamically corrected rendering rate. The scene operation matching coefficient is obtained based on the real-time rendering rate, the total amount of rendering, and the dynamic correction rendering rate.

[0013] Preferably, the comprehensive management coefficient for try-on is obtained based on the overall compatibility coefficient of the item itself and the scenario operation matching coefficient, specifically including the following steps: Pre-configure item adaptation weights and operational matching weights; The overall management coefficient for try-on is obtained by multiplying the overall compatibility coefficient of the item itself, the item compatibility weight, the scenario operation matching coefficient, and the operation matching weight.

[0014] Preferably, the operation and control plan is output based on the comprehensive management coefficient of trial wearing, specifically including the following steps: The product category with the highest comprehensive management coefficient value will be the primary target product. Output a multimodal interactive, dynamic tracking and linkage virtual try-on operation and management solution for the main target products.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention constructs a standardized jewelry body adaptation model based on a 3D model and quantifies the item's fit coefficient. It unifies the evaluation standards for the materials, surfaces, and deformations of various types of jewelry, thereby quantifying the static fit between the jewelry and the wearer's body. This fundamentally reduces problems such as gaps in the virtual image, surface misalignment, and gemstone light and shadow distortion, significantly improving the realism of static try-on simulation. It also integrates multiple dynamic features to construct a human dynamic motion tracking model, quantifying and generating a dynamic tracking stability coefficient. This automatically compensates for tracking deviations caused by light occlusion, image jitter, and rapid limb movements, distinguishing tracking stability levels under different motion amplitudes. This effectively improves the distortion problems of jewelry drift, tracking delay, and detachment from the skin during human activity in traditional try-on systems, ensuring continuous stability of the dynamic try-on image. By merging the static fit and dynamic stability coefficients to calculate the comprehensive fit coefficient of the item body, it uniformly considers the jewelry's try-on performance in both static and dynamic scenarios, overcoming the shortcomings of traditional technologies where static and dynamic evaluations are separate and unable to comprehensively assess product fit capabilities. It automatically selects key products and outputs corresponding interactive and tracking control solutions, eliminating the need for operations personnel to rely on manual experience to allocate equipment and promotion strategies. This achieves automated and intelligent scheduling of product promotion strategies, thereby significantly reducing manual operation costs. Attached Figure Description

[0016] Figure 1 A schematic diagram of a virtual jewelry try-on system based on multimodal interaction and real-time dynamic tracking is provided for embodiments of the present invention. Figure 2 This invention provides a schematic diagram of the process for obtaining the fitting coefficient of an item in a virtual jewelry try-on system based on multimodal interaction and real-time dynamic tracking, as part of an embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0018] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0019] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0020] Reference Figures 1-2 As shown.

[0021] The embodiments further illustrate the jewelry virtual try-on system based on multimodal interaction and real-time dynamic tracking proposed in this invention.

[0022] A virtual jewelry try-on system based on multimodal interaction and real-time dynamic tracking includes: Acquisition module: Acquires multimodal interaction data and 3D model data of the items throughout the entire virtual try-on process; First evaluation module: Construct an item body adaptation model based on the item's 3D model data, and perform quantitative evaluation of the item body adaptation model to obtain the item's wearing fit coefficient; The second evaluation module: Construct a human dynamic motion tracking model based on the dynamic tracking data of multimodal interaction data, and evaluate the stability of the human dynamic motion tracking model to obtain the dynamic following stability coefficient; First processing module: Calculates the overall fit coefficient of the item based on the item's wearing fit coefficient and dynamic following stability coefficient; Generation module: Based on the object body adaptation model and the human body dynamic motion tracking model, a virtual try-on full-domain simulation model is generated through spatial coupling; The second processing module: acquires management requirement data from the operation side, and combines it with the virtual try-on full-domain simulation model to output rendering performance parameters to obtain the scene operation matching coefficient; Output module: Based on the overall compatibility coefficient of the item itself and the scene operation matching coefficient, the overall management coefficient of the trial wearing is obtained, and the operation control plan is output based on the overall management coefficient of the trial wearing.

[0023] The process of constructing an adaptation model of an item based on its 3D model data includes the following steps: Establish a standard benchmark model for the material of the item in advance; Extract the contour dimensional features, material physical characteristics, and structural deformation characteristics of the three-dimensional model data of the product; Material physical characteristics include the material characteristics of precious metals and the optical refraction characteristics of gemstones.

[0024] A material adaptation model for jewelry is constructed by combining contour size features, material physical features, structural deformation features, and standard benchmark models of object materials.

[0025] The raw materials for jewelry are entered with standardized and fixed parameters, covering two major categories: precious metals and gemstones. Taking pure gold as an example, the benchmark model is entered with fixed density values, basic surface reflectance values ​​at room temperature, standard hardness values, and basic light diffuse reflection coefficients. Taking diamond as an example, the benchmark model is entered with fixed refractive index values, dispersion coefficients, and basic values ​​for light transmission loss.

[0026] The outline size features record all the shape parameters that come into contact between the jewelry and the parts of the human body that are worn. Taking the 3D model of a ring as an example, the outline size features include the inner diameter of the ring, the curvature of the inner surface, the width of the ring wall, and the circumference of the ring setting. Taking the 3D model of a necklace pendant as an example, the outline size features include the curvature of the back surface of the pendant and the length and width of the bottom contact area of ​​the pendant. All size parameters are uniformly converted to millimeters for storage, and the curvature of the surface is uniformly recorded in radians for use in calculating the overlap of the reference surface for human wear. The physical characteristics of the material are broken down into the material characteristics of precious metals and the optical refraction characteristics of gemstones. The material characteristics of precious metals record the density of the precious metal substrate, the surface specular reflectivity, and the deformation threshold under external pressure. For example, platinum is recorded with a set of exclusive density values ​​and critical deformation pressure values. The optical refraction characteristics of gemstones record the refractive index, dispersion value, and light absorption loss ratio of gemstones in a standard white light environment. For example, sapphire is recorded with the refractive index and light transmission attenuation values. The structural deformation characteristics record the shape change law of the jewelry splicing structure under external force. Taking a clasp bracelet as an example, the structural deformation characteristics include the maximum rotation angle of the bracelet hinge, the maximum displacement length of the bracelet body under tension, and the deformation rebound difference after the clasp is closed. Taking a multi-diamond set ring as an example, the structural deformation characteristics include the critical displacement value of the ring prongs under force bending and the small displacement range of the set gemstones after being compressed. These are used to simulate the shape changes of jewelry caused by limb movements during wearing.

[0027] The physical characteristics of the material are clearly defined, consisting only of two parts: the physical characteristics of the precious metal and the optical refraction characteristics of the gemstone. No other auxiliary parameters are included. The physical characteristics of the precious metal correspond only to the physical parameters of the base metal of the jewelry and do not include data related to gemstone optics. The optical refraction characteristics of the gemstone correspond only to the optical propagation parameters of the set gemstone and do not include data related to metal density and hardness. The total deviation of the precious metal is calculated as: (extracted precious metal density difference + extracted precious metal reflectivity difference + extracted precious metal deformation threshold difference). The total deviation of the gemstone is calculated as: (extracted gemstone refractive index difference + extracted gemstone dispersion difference + extracted gemstone light transmission loss difference).

[0028] A jewelry material adaptation model is constructed by integrating three types of extracted features with the standard benchmark model of the object's material. Matching scores between individual features and the benchmark model are calculated: contour size matching score = total value of contour benchmark standard - total difference between contour size features and the benchmark; precious metal material matching score = total coefficient of precious metal benchmark standard - total deviation of precious metal material; gemstone optical matching score = total coefficient of gemstone benchmark standard - total deviation of gemstone optical; structural deformation matching score = total value of deformation benchmark standard - total difference between structural deformation features and the benchmark. Fixed weights are set to calculate the comprehensive adaptation degree: comprehensive adaptation base value = contour size matching score × contour weight value + precious metal material matching score × metal weight value + gemstone optical matching score × gemstone weight value + structural deformation matching score × deformation weight value. Based on the comprehensive adaptation base value, the parameters of the 3D model are corrected, mapping the contour surface, metal optical parameters, gemstone refraction parameters, and deformation range to the standardized framework of the benchmark model, eliminating parameter offsets caused by scanning the physical 3D model, and finally generating the jewelry material adaptation model.

[0029] The following steps are involved in quantitatively evaluating the fit model of the item to obtain the item's wearing fit coefficient: Extract the wearing contact surface morphology characteristics and human wearing reference surface morphology characteristics corresponding to the product body adaptation model; By comparing the fit gap difference between the wear contact surface morphology and the human wear reference surface morphology and the degree of overlap of the curved surface, basic fit fit feature data are obtained. Based on the basic fit and fitting feature data, the critical fit area and the fully fit area are selected, and the fit difference attributes of different areas are distinguished based on the critical fit area and the fully fit area. The evaluation weight is obtained by weighting each fitting region according to the regional adaptation difference attribute. The overall wearing fit is evaluated based on the evaluation weights to obtain the item wearing fit coefficient.

[0030] The system adapts the wear contact surface morphology features stored within the model of the object itself. These features completely record all surface parameters of the area where the jewelry directly contacts the human skin. Taking a closed ring as an example, the wear contact surface morphology features include the curvature of the complete inner ring surface, the radial depth of each arc segment of the inner ring, the edge contour length of the contact area, and the height difference of local concave and convex areas. Taking a necklace pendant that fits the neck as an example, the wear contact surface morphology features include the arc range of the back surface of the pendant, the length and width of the back plane, and the edge chamfer offset. All values ​​are directly read from the jewelry material adaptation model built in the early stage. These are the human wear reference surface morphology features pre-stored by the system. These features correspond to the standard physiological surfaces of the wearing parts of people with different body shapes. They are divided into finger reference surfaces, wrist reference surfaces, and neck reference surfaces. Each type of reference surface is divided into multiple continuous sampling areas, recording the standard curvature, standard contour perimeter, and standard concavity and convexity difference of each surface segment. For example, the finger reference surface will distinguish the three independent surface parameters of the finger root, finger middle, and fingertip.

[0031] To compare the quantitative indicators of two sets of morphological features in both directions, basic fitting and matching feature data is generated. The comparison process includes the fitting gap difference and the degree of surface overlap. The fitting gap difference refers to the vertical distance between the jewelry contact surface and the human body reference surface. An equal number of sampling points are simultaneously divided on both sets of curved surfaces, and the vertical gap distance between each point is calculated. The average value of all gap values ​​is taken as the overall fitting gap mean. The gap matching score = 1 - overall fitting gap mean ÷ system preset maximum allowable gap value. The larger the gap mean, the lower the gap matching score. The degree of surface overlap refers to the matching ratio of the curvature change trends of the two sets of curved surfaces. The curvature of the jewelry contact surface is compared segment by segment with the curvature of the human body reference surface. The surface overlap score = 1 - the sum of curvature deviations of all sampling segments ÷ the total number of sampling segments. The greater the curvature deviation, the lower the surface overlap score. The two scores are added together to obtain the basic adaptation score for a single region. The basic adaptation score for a single region = gap adaptation score + surface coincidence score. The basic adaptation scores for all sampled regions are then summed to form the basic fit adaptation feature dataset.

[0032] Based on the basic fit and matching feature data, the fitting areas are classified, and the fitting difference attributes corresponding to the two types of areas are defined. A pre-set score threshold of 0.7 is used to divide the fully fitted area into the critical fitting area. For any sampling area, if the basic fit score of a single area is greater than or equal to 0.7, it is determined to be a fully fitted area. The fitting difference attributes of this type of area are: surface height matching, gap value close to 0, slight human movement will not change the fit, and there is no visual problem of jewelry floating or misalignment. For example, the center area where the inner circle of a ring touches the fingertip is a fully fitted area. If the basic fit score of a single area is less than 0.7, it is determined to be a critical fitting area. The fitting difference attributes of this type of area are: low surface matching, observable gaps, and small human movements such as raising the hand or turning the head will amplify the gap displacement. In virtual try-on, the jewelry may appear to float and detach from the skin. For example, the two ends of the bracelet opening and the edges of the pendant are mostly critical fitting areas. All sampling areas were marked and classified. The total length of the fully fitting area and the total length of the critical fitting area were calculated separately. At the same time, the basic adaptation score ranges corresponding to the two types of areas were recorded to distinguish the adaptation differences between the two types of areas.

[0033] The system performs weighted grading based on the adaptation differences between the two types of regions, generating an evaluation weight value for each sampling region. Two basic weight intervals are set: a fixed basic weight of 0.3 for the fully fitted region and a fixed basic weight of 0.7 for the borderline fitted region. This weight allocation logic aligns with user visual perception patterns; fitting defects in the borderline fitted region are more easily detected by the naked eye, thus assigning a higher weight to amplify the impact of this region on the overall evaluation result. The system also adds a category correction coefficient based on the jewelry category: 1.0 for rings, 1.1 for bracelets, and 0.9 for necklaces and pendants. The evaluation weight for a single region = basic weight of the region × corresponding jewelry category correction coefficient.

[0034] Based on the evaluation weights corresponding to each sampling area, a comprehensive calculation of the overall wearing fit status is completed, and the final output is the item wearing fit coefficient with a value range between 0 and 1. The weighted fit component value of a single area = the basic fit component value of that single area × the evaluation weight of that single area. The item wearing fit coefficient is obtained by summing the weighted fit component values ​​of all sampling areas. The item wearing fit coefficient = the sum of the weighted fit component values ​​of all single areas in the sampling area. For example, a ring consists of two fully fitting areas and one borderline fitting area. The basic fit scores for the two fully fitting areas are 0.85 and 0.82, respectively, with evaluation weights of 0.3 and 0.3. The basic fit score for the borderline fitting area is 0.62, with an evaluation weight of 0.4. Calculating the individual component values, the first fully fitting component value is 0.85 × 0.3 = 0.255, the second fully fitting component value is 0.82 × 0.3 = 0.246, and the borderline fitting component value is 0.62 × 0.4 = 0.248. Adding these three values ​​together, 0.255 + 0.246 + 0.248 = 0.749, the ring's fit coefficient is 0.749. The closer the value is to 1, the better the overall static fit of the jewelry. The closer the value is to 0, the larger the gap between the jewelry and the wearer's surface, the poorer the surface matching, and the weaker the realism of the static try-on.

[0035] The human dynamic motion tracking model is constructed based on the dynamic tracking data of multimodal interaction data, specifically including the following steps: Establish a standard benchmark model for human body tracking in advance; Extract displacement features, micro-expression features, limb rotational angular velocity features, and environmental interference features from key points of the human skeleton in dynamic tracking data; By combining the displacement characteristics of key points of the human skeleton, micro-expression characteristics, limb rotational angular velocity characteristics, environmental interference characteristics, and human body tracking standard benchmark models, a human dynamic motion tracking model is constructed.

[0036] A pre-established human tracking benchmark model is constructed, and standardized motion parameters are entered into a general ergonomic database. Standard motion ranges are defined for key parts of the body, such as the head, wrist, fingers, and neck, where jewelry is worn. The benchmark model distinguishes the basic skeletal dimensions of different body types and defines the standard range of motion for each joint to track without distortion. For example, the standard rotation angle range of the wrist joint is 0 to 180 degrees, the standard maximum displacement distance of the finger joint is 20 millimeters, and the upper limit of the standard angular velocity of head deflection is 60 degrees per second. The benchmark tracking error values ​​under standard uniform lighting conditions without environmental obstruction are also entered. The same set of benchmark parameters will be used when any user performs dynamic try-on tracking, ensuring that the evaluation and calibration standards of the tracking model remain consistent for different users and in different scenarios.

[0037] Four types of human skeleton spatial position, subtle head movements, limb movement speed, and external environmental interference are extracted from the dynamic tracking raw data stream. Each type of feature is decomposed into multiple sets of continuously sampled quantized values. The human skeleton key point displacement feature records the real-time coordinate changes of all joint bone points in three-dimensional space. Taking the wrist of a bracelet as an example, the bone key point displacement feature includes the displacement distance of the three spatial coordinate axes of the wrist joint per frame, the offset length of the displacement trajectory in multiple consecutive frames, and the interval span of the round-trip movement of the bone point. All displacement values ​​are uniformly converted to millimeters. The system records the coordinate difference of each bone point frame by frame for comparison with the standard bone displacement interval in the benchmark model. The bone displacement deviation value = real-time bone displacement value - benchmark model standard bone displacement value. Micro-expression features focus on the offset of the wearing area caused by subtle changes in head posture. They primarily record minute displacements of the neck bones caused by slight head tilting, lateral tilting, and lifting. Simultaneously, they capture the overall spatial offset of the head caused by subtle facial muscle movements. For example, the offset value of the neck contact point corresponding to a 1-degree head tilt is synchronously recorded in the feature dataset. The micro-movement offset deviation value = real-time head micro-displacement value - baseline model standard head micro-movement threshold. Limb rotational angular velocity features statistically analyze the rotation amplitude of the wearing areas such as the wrist, neck, and fingers per unit time, distinguishing between three angular velocity ranges: slow micro-movement, medium-speed swinging, and rapid large-amplitude rotation. For example, the wrist angular velocity corresponding to a rapid arm-swinging motion can reach 90 degrees per second, while the wrist angular velocity corresponding to a slow arm-raising motion is only 10 degrees per second. The angular velocity deviation value = real-time limb angular velocity value - baseline model standard safe angular velocity upper limit. The fourth category is environmental interference features. These features record external variables that affect tracking accuracy in the trial wearing scenario, including the light attenuation coefficient caused by changes in screen brightness, the amplitude of image shake, and the duration of occlusion of skeletal points by the human arm or foreign objects. The total interference impact = real-time environmental interference coefficient × system preset interference weight value.

[0038] A human dynamic motion tracking model is constructed by integrating four types of extracted dynamic features with a human body tracking standard benchmark model. The deviation values ​​corresponding to each of the four feature types are standardized and converted: displacement deviations are uniformly mapped to the 0-1 range, angular velocity deviations are uniformly mapped to the 0-1 range, and the total environmental interference is directly converted to a coefficient within the 0-1 range. After unifying the units of all deviation values, a comprehensive correction calculation is performed. The scores are as follows: Skeletal displacement correction score = 1 - Skeletal displacement deviation value ÷ Maximum allowable skeletal displacement value of the benchmark model; Micro-expression correction score = 1 - Micro-movement offset deviation value ÷ Maximum allowable head micro-movement value of the benchmark model; Limb angular velocity correction score = 1 - Angular velocity deviation value ÷ Upper limit of standard safe angular velocity of the benchmark model; Environmental anti-interference correction score = 1 - Total interference impact. The weights for skeletal keypoint displacement are set to 0.4, micro-expression weight to 0.1, limb rotational angular velocity weight to 0.3, and environmental interference weight to 0.2. The comprehensive dynamic correction baseline value is calculated as: Skeletal displacement correction score × 0.4 + Micro-expression correction score × 0.1 + Limb rotational velocity correction score × 0.3 + Environmental interference correction score × 0.2. Based on the comprehensive dynamic correction baseline value, the original dynamic tracking coordinate data is offset-corrected. The corrected skeletal point displacement, head micro-motion changes, limb rotational velocity, and environmental interference compensation parameters are all mapped to the standardized skeletal framework of the human tracking standard benchmark model. This eliminates tracking and positioning errors caused by camera shake, excessively fast limb movements, and light occlusion, ultimately generating a human dynamic motion tracking model.

[0039] The stability evaluation of the human dynamic motion tracking model yields the dynamic following stability coefficient, which specifically includes the following steps: Collect the continuous posture change trajectory generated by continuous human activity, extract the posture offset fluctuation information during the continuous posture change trajectory, and obtain the original characteristics of posture fluctuation. Generate interval fluctuation differentiation results based on the original characteristics of attitude fluctuations; By using interval fluctuation differentiation results to match the stability evaluation criteria corresponding to different activity amplitudes, a dynamic following stability coefficient is obtained.

[0040] The system continuously collects the posture change trajectory generated by continuous human activity, extracts posture offset fluctuation information throughout the trajectory, and generates original posture fluctuation features. Based on a human dynamic motion tracking model, it collects the three-dimensional motion coordinates of the body parts wearing the bracelet frame by frame. Taking the wrist wearing a bracelet as an example, the system records the spatial coordinates of key skeletal points in each frame throughout the entire process of the wrist moving from stillness to a small swing and then to a rapid swing. Multiple consecutive frames are concatenated to form a complete and continuous posture change trajectory. The system separates two types of core fluctuation information from the trajectory data: the first is the point offset distance between single frames, representing the spatial jump amplitude of human skeletal points in adjacent frames; the second is the posture rotation angle difference between consecutive frames, representing the amplitude of angle changes during limb rotation. These two types of information are arranged and summarized in chronological order to form the original posture fluctuation feature dataset. The dataset contains four basic values ​​within each second sampling period: average offset distance, maximum offset distance, average angle difference, and maximum angle difference. The single-frame point offset = total distance of skeletal three-dimensional coordinates in the next frame - total distance of skeletal three-dimensional coordinates in the previous frame; the single-frame angle fluctuation difference = limb rotation angle value in the next frame - limb rotation angle value in the previous frame.

[0041] Based on the original characteristics of posture fluctuations, standardized interval fluctuation distinction results are generated. Two sets of pre-set boundary thresholds are used to divide the intervals: the first set of thresholds is the offset distance boundary value of 2 mm, and the second set of thresholds is the angle difference boundary value of 5 degrees. The average offset distance and average angle difference of all sampling periods in the original characteristics of posture fluctuations are compared to divide the intervals into three independent fluctuation categories. The first category is the stable fluctuation interval, corresponding to an average offset distance of less than or equal to 2 mm and an average angle difference of less than or equal to 5 degrees per period. This corresponds to low-amplitude micro-movement scenarios such as a person being still or slowly raising their hand. For example, a user slowly turning their neck to look at a necklace belongs to the stable fluctuation interval. The second category is the small-amplitude fluctuation interval, corresponding to an average offset distance of more than 2 mm and less than 8 mm per period, or an average angle difference of more than 5 degrees and less than 20 degrees per period. This corresponds to moderate-amplitude limb movements, such as the wrist movement of raising a hand to flip through goods. The third category is the large fluctuation range, corresponding to an average offset distance of 8 mm or more per cycle, or an average angle difference of 20 degrees or more. This corresponds to rapid, large-amplitude limb movements, such as rapid wrist swings or large head turns. The sampling duration of each of the three types of ranges within the entire posture trajectory is statistically analyzed to generate the range fluctuation differentiation results. These results are labeled with the fluctuation range type, duration, maximum offset value, and maximum angle difference for each time segment, thus distinguishing the fluctuation distribution pattern of the entire motion trajectory.

[0042] The system calls upon the interval fluctuation differentiation results to match the stability evaluation criteria corresponding to the activity amplitude, and obtains the dynamic following stability coefficient with a value between 0 and 1 through weighted calculation. Independent basic stability scores are pre-configured for the three types of fluctuation intervals: 0.9 for the stable fluctuation interval, 0.6 for the small fluctuation interval, and 0.3 for the large fluctuation interval. The weighted stability component value for a single interval = interval basic stability score × duration of continuous sampling for that interval ÷ total sampling duration of the trajectory. The three weighted stability component values ​​corresponding to the stable fluctuation interval, small fluctuation interval, and large fluctuation interval are summed, and the sum is the dynamic following stability coefficient: Dynamic following stability coefficient = weighted component value for stable interval + weighted component value for small fluctuation interval + weighted component value for large fluctuation interval. Taking wrist movement trajectory as an example, the total sampling time of the entire trajectory is 100 frames, of which the stable fluctuation range lasts for 60 frames, the small fluctuation range lasts for 30 frames, and the large fluctuation range lasts for 10 frames. The weighted component value of the stable range = 0.9 × 60 ÷ 100 = 0.54, the weighted component value of the small fluctuation range = 0.6 × 30 ÷ 100 = 0.18, and the weighted component value of the large fluctuation range = 0.3 × 10 ÷ 100 = 0.03. The sum of the three values ​​is 0.54 + 0.18 + 0.03 = 0.75. The dynamic tracking stability coefficient corresponding to this segment of human movement is 0.75. The closer the value is to 1, the smaller the deviation fluctuation of the virtual jewelry following the posture throughout the human activity, and the stronger the tracking synchronization. The closer the value is to 0, the more serious the tracking deviation is during limb movement, and the jewelry is prone to misalignment and drift.

[0043] The overall fit coefficient of the item is obtained based on the item's wearing fit coefficient and dynamic tracking stability coefficient, specifically including the following steps: The dynamic deformation interference factor corresponding to the wearing fit coefficient is determined based on the dynamic tracking stability coefficient. The dynamic fit influence coefficient is obtained based on the fit coefficient and the dynamic deformation interference factor. The overall fit coefficient of the item is obtained based on the wearing fit coefficient and the dynamic fit influence coefficient.

[0044] The dynamic deformation interference factor is calculated based on the dynamic tracking stability coefficient. This factor quantifies the negative impact of human limb movements on the static fit of jewelry. This parameter exhibits an inverse relationship with the dynamic tracking stability coefficient. A higher dynamic tracking stability coefficient indicates better synchronization of human movement tracking, smaller dynamic drift of the jewelry, and a lower corresponding dynamic deformation interference factor. Conversely, a lower dynamic tracking stability coefficient indicates more pronounced tracking drift and a higher dynamic deformation interference factor. The dynamic deformation interference factor is calculated as 1 - dynamic tracking stability coefficient. When a user slowly rotates their neck to try on a necklace, the dynamic tracking stability coefficient is 0.85. Substituting this into the formula, the dynamic deformation interference factor is calculated as 1 - 0.85 = 0.15, indicating that this movement only slightly interferes with the jewelry's fit. When a user quickly swings their wrist to try on a bracelet, the dynamic tracking stability coefficient is 0.32. Substituting this into the formula, the dynamic deformation interference factor is calculated as 1 - 0.32 = 0.68, indicating that large limb movements cause significant interference to the jewelry's fit.

[0045] The dynamic fit influence coefficient is calculated by combining the fit coefficient of the item and the dynamic deformation interference factor. This coefficient quantifies the loss caused by dynamic motion interference superimposed on the static fit effect, visually reflecting the decrease in jewelry fit after human activity. The dynamic fit influence coefficient = item fit coefficient × dynamic deformation interference factor. In the first necklace try-on scenario, the item fit coefficient was 0.88, the dynamic deformation interference factor was 0.15, and the dynamic fit influence coefficient = 0.88 × 0.15 = 0.132, meaning that dynamic movements cause a loss of 0.132 in the static fit effect. In the second bracelet try-on scenario, the item fit coefficient was 0.76, the dynamic deformation interference factor was 0.68, and the dynamic fit influence coefficient = 0.76 × 0.68 = 0.5168, meaning that large-amplitude body movements cause a significant loss of 0.5168 in the static fit effect. These two sets of values ​​clearly distinguish the difference in fit loss caused by different amplitudes of movement.

[0046] The final overall fit coefficient of the jewelry is obtained by subtracting the dynamic fit influence coefficient from the original fit coefficient. This coefficient combines the basic level of static fit with the fit loss caused by dynamic movement, and can objectively reflect the overall fit quality of the jewelry in all scenarios of human stillness and activity. The overall fit coefficient = fit coefficient - dynamic fit influence coefficient. In the first necklace try-on scenario, the overall fit coefficient = 0.88 - 0.132 = 0.748; in the second necklace try-on scenario, the overall fit coefficient = 0.76 - 0.5168 = 0.2432. The closer the overall fit coefficient is to 1, the better the static fit of the jewelry and the less fit loss caused by human movement, resulting in a stable and realistic overall virtual try-on experience. The closer the value is to 0, the more likely there are defects in the static fit of the jewelry or serious fit loss caused by human movement, making it prone to distortion problems such as jewelry floating and misalignment, and edges detaching from the skin during dynamic try-on. This coefficient will serve as the core input parameter for subsequent calculations of the overall fit management coefficient.

[0047] Obtain management requirement data from the operations side, and combine it with the rendering performance parameters output by the virtual try-on full-domain simulation model to obtain the scene operation matching coefficient. This specifically includes the following steps: Extract total passenger flow and interaction response rate from operational demand data; The time-period control weights for each time period are obtained based on the total passenger flow and the interaction response rate. The virtual try-on full-domain simulation model is used to output real-time rendering rate, total rendering volume, and dynamically corrected rendering rate. The scene operation matching coefficient is obtained based on the real-time rendering rate, the total amount of rendering, and the dynamic correction rendering rate.

[0048] The extracted operational requirements data includes corresponding scene traffic load and user experience response standards. Total traffic volume records the total number of users entering the virtual try-on scene within a single operational period. For example, the total traffic volume during peak evening hours is 300 people per hour, while during midday off-peak hours it's 60 people per hour. A higher total traffic volume indicates more users requesting try-on at the same time, requiring more simultaneous processing of 3D rendering tasks. Interaction response rate defines the maximum allowable latency for user operation feedback on the operational side. A higher standard value indicates stricter requirements for the smoothness of the try-on screen. Live-streaming e-commerce scenarios typically require an interaction response rate of 60 frames per second, while offline store self-service terminals can relax this to 30 frames per second. This indicator directly defines the minimum baseline that the system rendering performance must meet. Both types of indicators are stored separately by hour, forming a time-segmented operational requirements dataset.

[0049] The time-period control weight is calculated based on two indicators: total passenger flow and interaction response rate. This weight quantifies the pressure level on the system rendering performance during the current time period; a higher weight indicates greater system load and a higher evaluation weight in the matching calculation. The passenger flow normalized score is calculated as: total passenger flow in the current time period ÷ system preset peak passenger flow. The interaction normalized score is calculated as: interaction response rate in the current time period ÷ system preset maximum response rate. Both normalized scores are converted to dimensionless values ​​between 0 and 1. The fixed weight for passenger flow is set to 0.6, and the fixed weight for interaction is set to 0.4. The time-period control weight is calculated as: passenger flow normalized score × 0.6 + interaction normalized score × 0.4. For example, if the peak audience for a live stream is set to 400 people per hour, and the current audience is 300 people per hour, with a preset maximum response rate of 80 frames per second and a current required interaction response rate of 60 frames per second, then the audience normalization score is 300 ÷ 400 = 0.75, the interaction normalization score is 60 ÷ 80 = 0.75, and the time period control weight is 0.75 × 0.6 + 0.75 × 0.4 = 0.75. This value indicates that the system load during this time period is at a medium to high level.

[0050] The system calls upon the virtual try-on full-domain simulation model to read the corresponding real-time screen output, total concurrent load, and dynamic tracking correction. The real-time rendering rate represents the number of complete try-on frames per second that a single terminal can output, directly reflecting the upper limit of screen smoothness during a single user's try-on. High-performance devices can reach 90 frames per second, while low-end terminals can only stably output 30 frames per second. The total rendering rate represents the total number of frames per second that the model can process when multiple users are simultaneously trying on the device, corresponding to the concurrent load capacity in multi-customer scenarios. For example, a total rendering rate of 1200 frames per second means the system can simultaneously support 20 users trying on the device at a standard of 60 frames per second. The dynamic correction rendering rate parameter represents the number of screen output frames that the model simultaneously completes human body dynamic tracking offset correction and jewelry fit surface correction. This parameter determines the smoothness of the virtual jewelry without drifting when the human body moves significantly. The higher the dynamic correction rendering rate value, the lower the probability of screen distortion in dynamic try-on scenarios.

[0051] The scene operation matching coefficient is obtained by combining three types of rendering performance parameters with time period control weights. This coefficient is used to measure the degree of adaptation between the current simulation rendering capability and the time period passenger flow interaction needs. The closer the value is to 1, the more the system performance can fully bear the current operation load. The closer the value is to 0, the more the rendering performance cannot meet the passenger flow and response speed requirements. Real-time rendering adaptation score = real-time rendering rate ÷ operational requirement interaction response rate; total concurrency adaptation score = total rendering volume ÷ total concurrency of single frame demand in the current time period; dynamic correction adaptation score = dynamic correction rendering rate ÷ operational requirement interaction response rate. All three adaptation scores are uniformly limited to the range of 0 to 1. Fixed performance weights are then assigned to each of the three scores: real-time rendering weight 0.3, total rendering volume weight 0.4, and dynamic correction rendering weight 0.3. The comprehensive performance base score is calculated as follows: Comprehensive performance base score = real-time rendering adaptation score × 0.3 + total concurrency adaptation score × 0.4 + dynamic correction adaptation score × 0.3. Finally, a weighted conversion is performed using the time period control weight to obtain the scene operation matching coefficient: Scene operation matching coefficient = comprehensive performance base score × (1 - time period control weight) + comprehensive performance base score × time period control weight × comprehensive performance base score. Using the example of a time-period control weight of 0.75 from the previous text, the operational requirements for this time period are an interactive response rate of 60 frames per second, a real-time rendering rate of 72 frames per second, a total rendering rate of 1200 frames per second, and a dynamic correction rendering rate of 66 frames per second. The total concurrent demand for a single frame in the current time period is 1000 frames per second. The real-time rendering adaptation score = 72 ÷ 60 = 1, the total concurrent adaptation score = 1200 ÷ 1000 = 1, and the dynamic correction adaptation score = 66 ÷ 60 = 1.1. The system truncates adaptation scores exceeding 1 to 1. The overall performance base score = 1 × 0.3 + 1 × 0.4 + 1 × 0.3 = 1, and the scene operation matching coefficient = 1 × (1 - 0.75) + 1 × 0.75 × 1 = 1. This means that the system's rendering performance in this time period fully matches the operational passenger flow and interaction requirements, and there will be no operational failures such as screen stuttering or tracking delays.

[0052] The overall management coefficient for try-on is obtained based on the overall compatibility coefficient of the item itself and the matching coefficient of the scenario operation. The specific steps include: Pre-configure item adaptation weights and operational matching weights; The overall management coefficient for try-on is obtained by multiplying the overall compatibility coefficient of the item itself, the item compatibility weight, the scenario operation matching coefficient, and the operation matching weight.

[0053] Pre-configure item compatibility weights and operational matching weights. These two weights are used to differentiate the proportions of product experience and operational load dimensions in the overall evaluation. The numerical ratio of these two weights can be freely adjusted according to different business scenarios to switch the evaluation focus. When the operational scenario is a high-end offline experience store, operations personnel increase the item compatibility weight and decrease the operational matching weight, prioritizing the fit and dynamic following effect of virtual jewelry try-on. For example, the item compatibility weight is configured to be 0.7 and the operational matching weight to be 0.3. When the operational scenario is a high-concurrency live-streaming e-commerce platform, operations personnel increase the operational matching weight and decrease the item compatibility weight, prioritizing the system's rendering performance and ability to handle customer traffic. For example, the item compatibility weight is configured to be 0.4 and the operational matching weight to be 0.6. Both weights are pre-entered into the system parameter library and directly called during each round of comprehensive coefficient calculation, maintaining fixed values ​​throughout until manually reconfigured.

[0054] The overall management coefficient for try-on is calculated as follows: Item-specific overall compatibility coefficient × Item compatibility weight + Scenario operation matching coefficient × Operation matching weight. The final value remains within the range of 0 to 1. A value closer to 1 indicates that the jewelry product has an excellent try-on effect and the system performance is well-suited to the current customer traffic, making it suitable as a key product for promotion. A value closer to 0 indicates that the jewelry's fit to dynamic effects is flawed or the system cannot handle the current operational load, making it unsuitable for concentrated traffic promotion.

[0055] The first scenario is a high-end offline jewelry experience store. The preset item compatibility weight is 0.7, and the operation matching weight is 0.3. The overall compatibility coefficient of a certain diamond necklace is calculated to be 0.748, and the operation matching coefficient for the current time period is 0.62. Substituting these values ​​into the formula, the weighted component of product compatibility is 0.748 × 0.7 = 0.5236, and the weighted component of operation matching is 0.62 × 0.3 = 0.186. Adding these two values ​​together, the overall management coefficient for trying on the necklace is 0.5236 + 0.186 = 0.7096. This value indicates that the necklace performs at a slightly above-average level in the store setting. The second scenario is during peak evening live-streaming sales hours. The preset item compatibility weight is 0.4, and the operational matching weight is 0.6. The overall compatibility coefficient of a certain plain bracelet is calculated to be 0.2432, while the operational matching coefficient for the current time period is 0.91. Step-by-step calculations yield the following weighted product compatibility value: 0.2432 × 0.4 = 0.09728; the weighted operational matching value: 0.91 × 0.6 = 0.546; and the overall trial-wearing management coefficient: 0.09728 + 0.546 = 0.64328. This value indicates that the bracelet's dynamic trial-wearing fit is relatively poor.

[0056] Based on the comprehensive management coefficient of trial wear, an operational control plan is generated, which includes the following steps: The product category with the highest comprehensive management coefficient value will be the primary target product. Output a multimodal interactive, dynamic tracking and linkage virtual try-on operation and management solution for the main target products.

[0057] The system iterates through the comprehensive management coefficients for all jewelry categories, sorts the values, and selects the category with the highest value as the primary target item. All category coefficients are then sorted in descending order. The jewelry category corresponding to the first element of the sorted sequence is extracted; this category is the primary target item for the current operating period. Based on the primary target item, a multimodal interaction and dynamic tracking-linked virtual try-on operation management solution is output. The management solution consists of two core control modules. The first module is the multimodal interaction control content. It reads the comprehensive adaptation coefficient of the primary item and adjusts the user interaction trigger logic according to the coefficient value. When the comprehensive adaptation coefficient is higher than 0.7, full-dimensional multimodal interaction functions are enabled, supporting hand gestures to switch styles, head posture rotation to adjust the try-on perspective, and voice commands to switch lighting and shadow environments. When the comprehensive adaptation coefficient is lower than 0.4, the interaction mode is simplified, retaining only basic click-to-switch functions to reduce the rendering load caused by multimodal synchronous computation. The second module involves dynamic tracking and linkage control. It retrieves the dynamic tracking stability coefficient of the featured item and the scene operation matching coefficient for the current time period. Based on this, it adjusts two core rendering parameters: the human body dynamic tracking sampling frame rate and the dynamic correction rendering rate. Taking a diamond necklace as the featured item, the dynamic tracking stability coefficient for this category is 0.85, and the scene operation matching coefficient is 0.62. The system automatically increases the dynamic tracking sampling frame rate to 60 frames per second and matches the dynamic correction rendering rate to 60 frames per second to ensure that the necklace does not float or misalign when the user turns their head or looks down. If the featured item is a bracelet with a dynamic tracking stability coefficient of 0.32 and it is during a high-traffic live broadcast period, the dynamic tracking sampling frame rate is appropriately reduced to 30 frames per second to balance the smoothness of the screen and the system's concurrent load.

[0058] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0059] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual jewelry try-on system based on multimodal interaction and real-time dynamic tracking, characterized in that, include: Acquisition module: Acquires multimodal interaction data and 3D model data of the items throughout the entire virtual try-on process; First evaluation module: Construct an item body adaptation model based on the item's 3D model data, and perform quantitative evaluation of the item body adaptation model to obtain the item's wearing fit coefficient; The second evaluation module: Construct a human dynamic motion tracking model based on the dynamic tracking data of multimodal interaction data, and evaluate the stability of the human dynamic motion tracking model to obtain the dynamic following stability coefficient; First processing module: Calculates the overall fit coefficient of the item based on the item's wearing fit coefficient and dynamic following stability coefficient; Generation module: Based on the object body adaptation model and the human body dynamic motion tracking model, a virtual try-on full-domain simulation model is generated through spatial coupling; The second processing module: acquires management requirement data from the operation side, and combines it with the virtual try-on full-domain simulation model to output rendering performance parameters to obtain the scene operation matching coefficient; Output module: Based on the overall compatibility coefficient of the item itself and the scene operation matching coefficient, the overall management coefficient of the trial wearing is obtained, and the operation control plan is output based on the overall management coefficient of the trial wearing.

2. The jewelry virtual try-on system based on multimodal interaction and real-time dynamic tracking according to claim 1, characterized in that, The process of constructing an adaptation model of an item based on its 3D model data includes the following steps: Establish a standard benchmark model for the material of the item in advance; Extract the contour dimensional features, material physical characteristics, and structural deformation characteristics of the three-dimensional model data of the product; A material adaptation model for jewelry is constructed by combining contour size features, material physical features, structural deformation features, and standard benchmark models of object materials.

3. The jewelry virtual try-on system based on multimodal interaction and real-time dynamic tracking according to claim 2, characterized in that, The following steps are involved in quantitatively evaluating the fit model of the item to obtain the item's wearing fit coefficient: Extract the wearing contact surface morphology characteristics and human wearing reference surface morphology characteristics corresponding to the product body adaptation model; By comparing the fit gap difference between the wear contact surface morphology and the human wear reference surface morphology and the degree of overlap of the curved surface, basic fit fit feature data are obtained. Based on the basic fit and fitting feature data, the critical fit area and the fully fit area are selected, and the fit difference attributes of different areas are distinguished based on the critical fit area and the fully fit area. The evaluation weight is obtained by weighting each fitting region according to the regional adaptation difference attribute. The overall wearing fit is evaluated based on the evaluation weights to obtain the item wearing fit coefficient.

4. The jewelry virtual try-on system based on multimodal interaction and real-time dynamic tracking according to claim 2, characterized in that, The physical characteristics of the material include the characteristics of precious metals and the optical refraction characteristics of gemstones.

5. The jewelry virtual try-on system based on multimodal interaction and real-time dynamic tracking according to claim 4, characterized in that, The human dynamic motion tracking model is constructed based on the dynamic tracking data of multimodal interaction data, specifically including the following steps: Establish a standard benchmark model for human body tracking in advance; Extract displacement features, micro-expression features, limb rotational angular velocity features, and environmental interference features from key points of the human skeleton in dynamic tracking data; By combining the displacement characteristics of key points of the human skeleton, micro-expression characteristics, limb rotational angular velocity characteristics, environmental interference characteristics, and human body tracking standard benchmark models, a human dynamic motion tracking model is constructed.

6. The jewelry virtual try-on system based on multimodal interaction and real-time dynamic tracking according to claim 5, characterized in that, The stability evaluation of the human dynamic motion tracking model yields the dynamic following stability coefficient, which specifically includes the following steps: Collect the continuous posture change trajectory generated by continuous human activity, extract the posture offset fluctuation information during the continuous posture change trajectory, and obtain the original characteristics of posture fluctuation. Generate interval fluctuation differentiation results based on the original characteristics of attitude fluctuations; By using interval fluctuation differentiation results to match the stability evaluation criteria corresponding to different activity amplitudes, a dynamic following stability coefficient is obtained.

7. The jewelry virtual try-on system based on multimodal interaction and real-time dynamic tracking according to claim 1, characterized in that, The overall fit coefficient of the item is obtained based on the item's wearing fit coefficient and dynamic tracking stability coefficient, specifically including the following steps: The dynamic deformation interference factor corresponding to the wearing fit coefficient is determined based on the dynamic tracking stability coefficient. The dynamic fit influence coefficient is obtained based on the fit coefficient and the dynamic deformation interference factor. The overall fit coefficient of the item is obtained based on the wearing fit coefficient and the dynamic fit influence coefficient.

8. The jewelry virtual try-on system based on multimodal interaction and real-time dynamic tracking according to claim 7, characterized in that, Obtain management requirement data from the operations side, and combine it with the rendering performance parameters output by the virtual try-on full-domain simulation model to obtain the scene operation matching coefficient. This specifically includes the following steps: Extract total passenger flow and interaction response rate from operational demand data; The time-period control weights for each time period are obtained based on the total passenger flow and the interaction response rate. The virtual try-on full-domain simulation model is used to output real-time rendering rate, total rendering volume, and dynamically corrected rendering rate. The scene operation matching coefficient is obtained based on the real-time rendering rate, the total amount of rendering, and the dynamic correction rendering rate.

9. The jewelry virtual try-on system based on multimodal interaction and real-time dynamic tracking according to claim 8, characterized in that, The overall management coefficient for try-on is obtained based on the overall compatibility coefficient of the item itself and the matching coefficient of the scenario operation. The specific steps include: Pre-configure item adaptation weights and operational matching weights; The overall management coefficient for try-on is obtained by multiplying the overall compatibility coefficient of the item itself, the item compatibility weight, the scenario operation matching coefficient, and the operation matching weight.

10. The virtual jewelry try-on system based on multimodal interaction and real-time dynamic tracking according to claim 1, characterized in that, Based on the comprehensive management coefficient of trial wear, an operational control plan is generated, which includes the following steps: The product category with the highest comprehensive management coefficient value will be the primary target product. Output a multimodal interactive, dynamic tracking and linkage virtual try-on operation and management solution for the main target products.

Images