Virtual character clothing changing method and system

Abstract

The application discloses a virtual image dressing method and system, which comprises the following steps: preprocessing a character image input by a user to obtain standardized image data; recognizing a face region in the standardized image data and constructing a three-dimensional face model; determining body parameters corresponding to the three-dimensional face model, generating a three-dimensional human body model based on the body parameters; deforming a clothing template based on a matching result between a measurement size of each key part in the three-dimensional human body model and a design size of a corresponding part in the clothing template, obtaining a three-dimensional clothing model, and simulating virtual dressing effect data when the three-dimensional human body model wears the three-dimensional clothing model; updating the three-dimensional human body model according to a user face feature and a user body posture, updating the virtual dressing effect data according to the updated three-dimensional human body model, and obtaining a virtual image preview picture corresponding to a dynamic performance behavior of the user.

Description

TECHNICAL FIELD

[0001] The present application relates to the virtual reality technical field, and particularly relates to a virtual image dressing method and system. BACKGROUND

[0002] With the rise of the metaverse concept and the rapid development of virtual reality technology, digital human technology has become an important part of the digital economy. The application demand of virtual image in live broadcast, game, social media and other fields is increasing. However, the existing virtual dressing technology has the problems of high threshold for creating virtual image, unrealistic dressing effect of virtual image, lack of dynamic performance of virtual image, low platform integration, insufficient individuality, and unbalanced rendering quality and efficiency, which leads to low user experience of virtual dressing. Therefore, how to improve the user experience of virtual dressing has become a problem to be solved in the field. SUMMARY

[0003] The present application provides a virtual image dressing method and system, which aims to improve the user experience of virtual dressing.

[0004] In order to achieve the above purpose, the present application provides the following technical solutions:

[0005] A virtual image dressing method, comprising:

[0006] preprocessing a character image input by a user to obtain standardized image data;

[0007] identifying a face region in the standardized image data, and constructing a three-dimensional face model;

[0008] determining a body parameter corresponding to the three-dimensional face model, and generating a three-dimensional human body model based on the body parameter;

[0009] deforming a clothing template based on a matching result between a measurement size of each key part in the three-dimensional human body model and a design size of a corresponding part in the clothing template, to obtain a three-dimensional clothing model, and simulating a virtual dressing effect data when the three-dimensional human body model wears the three-dimensional clothing model;

[0010] updating the three-dimensional human body model by using real-time acquired user facial features and user body posture, and updating the virtual dressing effect data according to the updated three-dimensional human body model, to obtain a virtual image preview picture corresponding to a dynamic performance behavior of the user.

[0011] A virtual image dressing system, comprising:

[0012] a character image processing unit, configured to preprocess a character image input by a user to obtain standardized image data;

[0013] a face model construction unit configured to identify a face region in the standardized image data, and construct a three-dimensional face model;

[0014] a digital human generation unit configured to determine body parameters corresponding to the three-dimensional face model, and generate a three-dimensional human body model based on the body parameters;

[0015] a dressing simulation unit configured to deform a preset garment template based on a matching result between a measured size of each key part in the three-dimensional human body model and a designed size of a corresponding part in the garment template, to obtain a three-dimensional garment model, and simulate virtual dressing effect data of the three-dimensional human body model wearing the three-dimensional garment model;

[0016] a performance behavior preview unit configured to update the three-dimensional human body model by using real-time acquired user facial features and user body postures, and update the virtual dressing effect data according to the updated three-dimensional human body model, to obtain a virtual avatar preview screen corresponding to a dynamic performance behavior of the user.

[0017] A storage medium, the storage medium comprising a stored program, wherein the program is executed by a processor to perform the virtual avatar dressing method.

[0018] An electronic device, comprising: a processor, a memory and a bus; the processor is connected with the memory through the bus;

[0019] The memory is configured to store a program, and the processor is configured to execute the program, wherein the program is executed by the processor to perform the virtual avatar dressing method.

[0020] The technical scheme provided in the application pre-processes a character image input by a user to obtain standardized image data. A face region in the standardized image data is recognized, and a three-dimensional face model is constructed. Body parameters corresponding to the three-dimensional face model are determined, and a three-dimensional human body model is generated based on the body parameters. Based on a matching result between a measurement size of each key part in the three-dimensional human body model and a design size of a corresponding part in a preset clothing template, the clothing template is deformed to obtain a three-dimensional clothing model, and virtual dressing effect data when the three-dimensional human body model wears the three-dimensional clothing model is simulated. Real-time acquired user facial features and user body postures are used to update the three-dimensional human body model, and the virtual dressing effect data is updated according to the updated three-dimensional human body model to obtain a virtual image preview screen corresponding to dynamic performance behaviors of the user. According to the character image input by the user, the three-dimensional human body model is generated, and the virtual dressing effect data is used to simulate that the three-dimensional human body model wears the three-dimensional clothing model, virtual image dressing is realized, and the virtual image preview screen corresponding to the dynamic performance behaviors of the user is displayed by updating the virtual dressing effect data, thereby effectively improving user experience. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 A flowchart of a virtual image dressing method provided by an embodiment of the present application;

[0023] Figure 2 An architecture diagram of a virtual image dressing system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0025] In this application, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of additional same elements in the process, method, article or equipment including the element.

[0026] Embodiment one is shown as Figure 1 A virtual image dressing method provided by the embodiment of the application can be applied to a virtual image dressing system, and includes the following steps.

[0027] S101: Preprocessing the user-inputted character image to obtain standardized image data.

[0028] When the user starts the virtual image dressing system, a character image needs to be provided as the basis for generating a virtual image (i.e., a 3D digital person). The preprocessing of the character image is the starting point of the entire virtual image dressing work, and the processing quality of the standardized image data directly affects the effect of all subsequent links.

[0029] Optionally, the implementation process of preprocessing the user-inputted character image to obtain standardized image data can refer to the steps shown in Embodiment Two and the corresponding explanation.

[0030] S102: Identifying the face region in the standardized image data and constructing a three-dimensional face model.

[0031] After obtaining the standardized image data, the face position needs to be accurately identified and the key feature points need to be extracted. This step is a key link for converting a 2D photo into a 3D digital person, and the accuracy directly determines the similarity between the finally generated digital person and the user. After obtaining the key point data, the planar face information is converted into a three-dimensional face model, that is, the three-dimensional shape, depth information and three-dimensional structure of the face need to be inferred from a planar photo.

[0032] Optionally, the process of identifying the face region in the standardized image data and constructing a three-dimensional face model can be: identifying the face region in the standardized image data; generating key point data based on a plurality of key points in the face region; and constructing a three-dimensional face model based on the key point data.

[0033] In some examples, the key points can be facial key points.

[0034] Optionally, the implementation process of identifying the face region in the standardized image data and generating the key point data based on the plurality of key points in the face region can be referred to the implementation process shown in Embodiment 5.

[0035] Optionally, the implementation process of constructing the three-dimensional face model based on the key point data can be referred to the implementation process shown in Embodiment 6.

[0036] S103: determining a body parameter corresponding to the three-dimensional face model, and generating a three-dimensional human body model based on the body parameter.

[0037] After obtaining the three-dimensional face model, the body parameter of the user needs to be inferred to construct the three-dimensional human body model. Since the user only provides one image of the character, the association rule between the facial features and the body features needs to be utilized to generate a reasonable three-dimensional human body model in combination with the knowledge of anthropometry.

[0038] Optionally, the implementation process of determining the body parameter corresponding to the three-dimensional face model and generating the three-dimensional human body model based on the body parameter can be referred to the steps shown in Embodiment 3 and the corresponding explanation.

[0039] S104: deforming the clothing template based on the matching result between the measurement size of each key part in the three-dimensional human body model and the design size of the corresponding part in the clothing template, to obtain a three-dimensional clothing model, and simulating a virtual dressing effect data when the three-dimensional human body model wears the three-dimensional clothing model.

[0040] The clothing template is obtained based on the three-dimensional mesh modeling of the clothing.

[0041] Optionally, the implementation process of deforming the clothing template based on the matching result between the measurement size of each key part in the three-dimensional human body model and the design size of the corresponding part in the clothing template, to obtain a three-dimensional clothing model, and simulating a virtual dressing effect data when the three-dimensional human body model wears the three-dimensional clothing model can be referred to the steps shown in Embodiment 4 and the corresponding explanation.

[0042] S105: updating the three-dimensional human body model by using the real-time acquired facial features and body posture of the user, and updating the virtual dressing effect data according to the updated three-dimensional human body model, to obtain a virtual image preview picture corresponding to the dynamic performance behavior of the user.

[0043] The virtual image preview picture can facilitate the user to intuitively view and improve the user experience.

[0044] Optionally, by using the facial features of the user and the body posture of the user, the three-dimensional human body model is updated, and the virtual clothes changing effect data is updated, to obtain an implementation process of a virtual image preview picture corresponding to the dynamic performance behavior of the user. For details, please refer to the embodiment seven.

[0045] It should be emphasized that the virtual image clothes changing method based on the embodiments of the present application can solve the following core technical problems existing in the existing virtual clothes changing image changing system and digital human technology:

[0046] (1) 3D digital human rapid generation technical problem: the existing technology needs multiple photos or complex equipment to create a 3D digital human, which cannot meet the needs of ordinary users to quickly generate high-quality digital clones from a single photo. The present application realizes the rapid reconstruction of a complete 3D digital human model containing accurate facial features and body parameters from a single front photo through deep learning technology.

[0047] (2) Virtual clothes realistic rendering problem: traditional clothes changing systems can only perform simple mapping and lack realistic performance of clothing materials, lighting, shadows, etc. The present application establishes a physically-based clothing rendering model that can realistically simulate the optical properties, wrinkle effects and wearing state of different material clothes.

[0048] (3) Dynamic expression and clothes changing linkage problem: existing technologies cannot realize the coordinated display of digital human expressions, movements and virtual clothes. The present application designs an expression and movement perception-based clothes adaptive adjustment algorithm to ensure that the clothes remain natural and realistic in various expressions and postures.

[0049] (4) Diversified image preview technology problem: users cannot preview their dressing effects at different ages and body types. The present application develops an image change prediction technology based on physiological parameter modeling to support users to view the clothes changing effects of future images.

[0050] The steps S101-S105 generate a three-dimensional human body model from the user input image, and simulate the three-dimensional human body model wearing a three-dimensional clothes model through virtual clothes changing effect data, to realize virtual image clothes changing, and display a virtual image preview picture corresponding to the dynamic performance behavior of the user by updating the virtual clothes changing effect data, thereby effectively improving the user experience.

[0051] The implementation process of another virtual image clothes changing method according to the embodiment two can be seen from the following steps.

[0052] S201: Obtain the user input image.

[0053] The user can take a photo of a person image through a mobile phone camera or select a person image from a system album as an input of the virtual image clothes changing system. To ensure the accuracy of subsequent processing, the input person image needs to be standardized.

[0054] In some examples, the format of the person image includes, but is not limited to, JPG, PNG, and HEIC.

[0055] S202: Standardizing the person image to obtain a standardized image matrix.

[0056] The standardized processing of the person image can convert different formats of the person image into image data in RGB format (i.e., the standardized image matrix).

[0057] In some examples, the pixel resolution of the standardized image matrix is 512x512.

[0058] In a possible implementation process, after obtaining the person image input by the user, the basic information of the person image is first read, including resolution, color space, shooting parameters, etc. Then the person image is uniformly converted into RGB three-channel format, and the pixel value range of each channel is standardized to 0-255. The purpose of this is to eliminate the differences caused by different devices and different shooting conditions, and to provide a unified input format for subsequent processing.

[0059] S203: Using a preset image quality detection mechanism to detect the standardized image matrix to obtain an image quality detection result.

[0060] The image quality detection result includes a definition detection result, a lighting quality inspection result, and a face integrity inspection result.

[0061] It can be understood that before complex 3D digital human modeling is performed, it is necessary to ensure that the input image data meets the basic quality requirements. Therefore, the standardized image matrix needs to be detected using a preset image quality detection mechanism.

[0062] In some examples, the image quality detection mechanism includes definition detection, lighting quality detection, and face integrity detection. Definition detection can be used to determine the definition detection result, lighting quality detection can be used to determine the lighting quality inspection result, and face integrity detection can be used to determine the face integrity inspection result.

[0063] In the possible implementation process, the implementation process of the definition detection can be: the definition is evaluated by calculating the gradient change of the image data, and the specific method is to perform edge detection on the image data, and the number and intensity of the edge pixels are counted. If the image data is too blurred, the edge pixels will be few and the intensity will be weak. The definitional threshold can be set as: the edge pixel ratio is not less than 15%, and the average gradient intensity is not less than 20.

[0064] In the possible implementation process, the implementation process of the light quality detection can be: analyzing the overall brightness distribution and contrast of the image data, calculating the histogram of the image data, detecting whether the image data is overexposed (the area with pixel value close to 255 is too much) or underexposed (the area with pixel value close to 0 is too much), and evaluating whether the contrast of the image data is sufficient to ensure that the facial features in the image data can be clearly identified.

[0065] In the possible implementation process, the implementation process of the face integrity detection can be: using a face detection algorithm to confirm whether the image data contains a complete face, detecting whether the facial features such as eyes, nose, mouth, and face contour can be identified, whether there is serious obstruction (such as wearing a mask, wearing sunglasses, etc.), and whether the head angle is within an acceptable range (such as left and right deflection not more than 30 degrees, and up and down deflection not more than 20 degrees).

[0066] S204: In the case that any one of the definition detection result, the light quality detection result, and the face integrity detection result meets the corresponding optimization trigger condition, the standardized image matrix is optimized to obtain an optimized image matrix.

[0067] In the case that any one of the definition detection result, the light quality detection result, and the face integrity detection result meets the corresponding optimization trigger condition, it can be determined that the picture quality has certain problems but is still within a processable range, and the standardized image matrix needs to be optimized.

[0068] In some examples, the optimization trigger condition corresponding to the definition detection result and the image processing method can be: if the standardized image matrix is overall dark or bright, brightness correction is needed. The brightness correction method is to analyze the brightness histogram of the standardized image matrix, calculate the optimal brightness adjustment parameter, and specifically adjust the average brightness of the standardized image matrix to 128 while maintaining the naturalness of the brightness distribution to avoid unrealistic effects.

[0069] In some examples, the optimization trigger condition and image processing means corresponding to the illumination quality inspection result can be: for the standardized image matrix with insufficient contrast, an adaptive histogram equalization technique is used for enhancement, which can improve the contrast of the standardized image matrix and make the facial features clearer, but at the same time avoid unnatural effects caused by excessive enhancement.

[0070] In some examples, the optimization trigger condition and image processing means corresponding to the face integrity inspection result can be: for the standardized image matrix with noise, an edge-preserving denoising algorithm is used, which can eliminate random noise in the standardized image matrix while preserving important edge information, ensuring the clarity of facial features.

[0071] S205: performing size standardization and pixel value normalization on the optimized image matrix to obtain an image matrix.

[0072] After the quality detection and optimization processing, the optimized image matrix also needs to be subjected to size standardization and pixel value normalization to obtain an image matrix.

[0073] In some examples, the implementation process of size standardization can be: adjusting the optimized image matrix to a pixel size of 512x512, and if the person image is not square, it also needs to be intelligently cropped or filled, cropping around the face to ensure the completeness of facial information, and filling using the color of the edge pixels to avoid obvious boundaries.

[0074] In some examples, the implementation process of pixel value normalization can be: normalizing the pixel value from the range of 0-255 to the range of 0-1, which is the standard input format of subsequent deep learning algorithms, and recording the normalization parameters so that the original color range can be restored in the final output.

[0075] S206: generating standardized image data based on the image matrix and the first information.

[0076] The first information includes the original size of the person image, the image quality inspection result, and the optimization parameters applied when optimizing the standardized image matrix.

[0077] In some examples, after completing the preprocessing process, standardized image data is generated based on the image matrix and the first information. The data structure of the standardized image data can be: standardized picture data={image matrix: 512x512x3 numerical matrix; original size: record the width and height of the original image; image quality inspection result: comprehensive score of clarity, illumination, and integrity; optimization parameters: record the optimization parameters applied}.

[0078] It should be noted that this standardized image data will serve as the data support for 3D digital human modeling, ensuring that subsequent processing can be carried out on a unified and high-quality basis. Through the detailed processing of Embodiment Two, various different quality and format of human image can be processed, laying a solid foundation for subsequent 3D digital human generation. At the same time, the intelligent quality detection and optimization mechanism ensures that even under less than ideal shooting conditions, usable processing results can still be obtained.

[0079] The above-mentioned flow S201-S206 can effectively convert the user input human image into standardized image data, providing data support for subsequent 3D digital human modeling.

[0080] The implementation flow of another virtual image dressing method shown in Embodiment Three can be seen from the following steps.

[0081] S301: Positioning a plurality of landmark points on the three-dimensional face model, and obtaining a plurality of face key sizes based on the Euclidean distance between each landmark point.

[0082] After obtaining the plurality of face key sizes based on the Euclidean distance between each landmark point, each face key size can be standardized according to the true human body proportion.

[0083] In some examples, the data structure of the face key size can be: face key size={face width: the distance from the widest part of the left cheek to the widest part of the right cheek; face length: the distance from the hairline to the lowest point of the chin; chin width: the distance between the chin corners; nose width: the distance between the widest parts of the nose wings; eye distance: the distance between the inner corners of the two eyes; mouth width: the distance between the corners of the mouth; head depth: the estimated distance from the front end of the face to the back end of the head}.

[0084] S302: Using a classification algorithm to determine the type of each face key size to determine a plurality of face features.

[0085] The pre-trained classification algorithm is used to determine the type of the face key size, and the classification algorithm is trained based on a large amount of anthropometric data, which can identify the statistical association between different face key sizes and body types.

[0086] In some examples, the data structure of the face feature can be: face feature={face shape classification: determined according to the face length-width ratio (such as oval, round, square, long, etc.); skeletal structure: assesses the degree of skeletal thickness according to the chin width and zygomatic prominence; muscle development: infers muscle type according to facial lines and cheek fullness; age estimation: estimates the approximate age range according to facial texture and contour features}.

[0087] S303: Determine the basic body parameters corresponding to the plurality of facial features based on a pre-established mathematical correlation model between facial features and body parameters.

[0088] Wherein, after obtaining the plurality of facial features, the corresponding basic body parameters can be estimated by using the mathematical correlation model.

[0089] In some examples, the estimation of the basic body parameters includes height estimation, body weight category inference, and detailed body size calculation.

[0090] In possible implementations, the algorithm principle of height estimation includes: basic height = face length × 7.5 (based on the golden ratio of human body), correction factor = adjustment parameter calculated according to face shape features, for example, long face shape correction factor = 0.95 (face is relatively long, height may be slightly lower than the estimated value), round face shape correction factor = 1.05 (face is relatively short, height may be slightly higher than the estimated value), square face shape correction factor = 1.0 (face shape is uniform, according to the standard ratio), and final height = basic height × correction factor.

[0091] In possible implementations, the algorithm principle of body weight category inference includes: skeletal score = (mandible width + zygomatic width) / face length, muscle score = face line definition + cheek fullness, fat score = face roundness + double chin degree, if skeletal score > 0.7 AND muscle score > 0.6, then body type category = "athletic type" (muscular, low body fat), if fat score > 0.7, then body type category = "full type" (high body fat), otherwise body type category = "standard type" (proportion uniform).

[0092] In possible implementations, the algorithm principle of detailed body size calculation includes: shoulder width = face width × 3.2 + height adjustment coefficient, chest circumference = height × 0.52 + body type adjustment coefficient, waist circumference = chest circumference × 0.75 + fat adjustment coefficient, hip circumference = chest circumference × 1.05 + gender adjustment coefficient, arm length = height × 0.44, leg length = height × 0.55. Wherein, the adjustment coefficients are calculated based on the face feature analysis results: height adjustment coefficient = (face shape correction + skeletal thickness correction) × 10, body type adjustment coefficient = muscle score × 50 + fat score × 30, fat adjustment coefficient = fat score × 80, gender adjustment coefficient = gender feature value inferred from face features × 20.

[0093] S304: Fuse the determined individual basic body parameters with pre-collected user information to obtain a plurality of fused body parameters.

[0094] Wherein, in order to improve the accuracy of the estimation of the basic body parameters, the user is requested to provide some user information, and these user information is intelligently fused.

[0095] In some examples, the data structure of the user information can be: user metadata = {basic information: {gender: male / female / undisclosed (affecting body shape distribution characteristics); age: specific age or age range (affecting muscle distribution and posture); height: accurate height or height range (correcting face estimation results); weight: accurate weight or weight range (correcting body shape estimation results);}, optional information: {exercise habits: regular exercise / occasional exercise / rare exercise; body shape preference: ideal body shape state desired to be presented; health status: health factors affecting body shape}}.

[0096] In some examples, the fusion algorithm used in the process of fusing user information with each basic body parameter can be: for each basic body parameter, face estimation value = estimation result based on facial features, user provided value = corresponding information filled in by the user, if the user information provides accurate numerical value, then the fusion result = user provided value x 0.8 + face estimation value x 0.2, if the user information provides range information, then the fusion result = selected value x 0.6 + face estimation value x 0.4, otherwise the fusion result = face estimation value, and the fused body parameter = the fusion result after applying reasonable constraints.

[0097] S305: Detect each fused body parameter using a preset body parameter detection mechanism to obtain a parameter detection result of each fused body parameter.

[0098] The body parameter detection mechanism includes proportion checking and consistency checking.

[0099] In some examples, the implementation process of the proportion checking includes: judging whether the proportions of each body part conform to the ergonomic standards, for example, the proportion of shoulder width to height is 0.23-0.28, the proportion of waist circumference to height is 0.4-0.6, and the proportion of leg length to height is 0.5-0.6.

[0100] In some examples, the implementation process of the consistency checking includes: judging whether different parameters are coordinated with each other, for example, the BMI index of body weight and height is within the range of 16-35, and the consistency of muscularity and exercise habits.

[0101] S306: According to the parameter detection result of each fused body parameter, identify an abnormal body parameter and correct the abnormal body parameter to obtain each body parameter.

[0102] The correction of the abnormal body parameter to obtain each body parameter can exclude obviously unreasonable extreme values, and any fused body parameter deviating from the normal range by more than 3 standard deviations is replaced by a statistical average value.

[0103] S307: Select a body template matching each body parameter from the body template library, and deform and locally adjust the body template to obtain a three-dimensional body model.

[0104] The body template library includes various body modules, such as a basic male template corresponding to a standard adult male body mesh model, a basic female template corresponding to a standard adult female body mesh model, a muscle type variant corresponding to a body template with more developed muscles, a plump type variant corresponding to a body template with more fat distribution, and a slim type variant corresponding to a body template with more slender bones.

[0105] In some examples, the filtering algorithm of the body template can be: basic template = select male or female basic template according to inferred gender, variant weight = calculate the mixing weight of each variant according to body type classification, body template = basic template x basic weight + each variant template x corresponding weight.

[0106] In some examples, the deformation algorithm of the body model can be: for each part of the body model (such as head, torso, arm, leg), target size = size of the part calculated from body parameters, current size = size of the part in the standard template, scaling ratio = target size / current size, apply non-uniform scaling transformation, such as X-axis scaling for width (such as shoulder width, waist width), Y-axis scaling for height (such as torso length, leg length), and Z-axis scaling for thickness (such as chest thickness, hip thickness).

[0107] In some examples, the algorithm principle of local adjustment includes: muscle definition adjustment: adjust the visibility of muscle lines according to muscle development parameters; fat distribution adjustment: adjust the roundness of waist, hips, thighs, etc. according to fat score; bone prominence adjustment: adjust the prominence of clavicle, rib, knee, etc. according to bone thickness; age-related adjustment: adjust skin tightness and muscle fullness according to age estimation.

[0108] S308: Perform connection processing on the three-dimensional face model and the three-dimensional body model to obtain a complete human body model.

[0109] The connection processing algorithm can be used to perform connection processing on the three-dimensional face model and the three-dimensional body model to obtain a complete human body model.

[0110] In some examples, the implementation of the connection processing algorithm includes: head and neck connection size matching: the diameter of the neck of the facial model is measured from the bottom of the facial model, and the diameter of the neck of the body model is measured from the neck of the body model. If the size difference is >5%, the smaller model is adjusted to match the larger model to maintain the reasonable head-to-body ratio (between 1:7.5 and 1:8); geometric alignment of the connection surface: calculate the geometry at the connection point of the two models, and use spline interpolation algorithm to create a smooth transition region to ensure that there are no visible gaps or abrupt changes at the connection point.

[0111] S309: Based on the texture data of the 3D facial model, fine-tune the complete human body model to obtain the initial human body model.

[0112] Fine-tuning is used to achieve a unified appearance for the complete human body model.

[0113] In some examples, the appearance unification process includes skin color matching, lighting response unification, and detail texture processing.

[0114] In a possible implementation, the skin color matching process is as follows: extract the average skin color from the 3D facial model and apply the skin color to all skin areas of the complete human body model.

[0115] In a possible implementation, the process of unifying the lighting response is as follows: ensuring that the head and body have a consistent visual effect under the same lighting conditions, and unifying material parameters (reflectivity, roughness, transparency, etc.).

[0116] In a possible implementation, the detailed texture processing is carried out by generating a natural skin texture transition in the connecting area to avoid obvious texture boundary lines.

[0117] S310: Use a preset human model quality detection mechanism to detect the initial human model in order to obtain the human model detection results.

[0118] The human model quality detection mechanism includes ergonomic verification and visual realism assessment.

[0119] In some examples, ergonomic verification includes checking body proportions, joint positions, and symmetry.

[0120] In a possible implementation, the principle of body proportion examination is as follows: the head-to-body ratio should be between 1:7 and 1:8.5, the shoulder width to hip ratio should be 1.1-1.3 for men and 0.9-1.1 for women, and the waist-to-hip ratio should be 0.8-1.0 for men and 0.7-0.9 for women.

[0121] In possible implementations, the principle of joint position checking is whether the shoulder joint, elbow joint, wrist position is reasonable, and whether the hip joint, knee joint, ankle position is correct.

[0122] In possible implementations, the principle of symmetry checking is whether the left and right body is basically symmetrical, and the allowable asymmetry range is within 2%.

[0123] In some examples, visual realism evaluation can be achieved by using a realism scoring algorithm. The process of the realism scoring algorithm is: surface smoothness: check whether there are unnatural concave-convex; anatomical correctness: whether the shape of each body part conforms to the anatomical characteristics; overall coordination: whether the head and body form a harmonious and unified whole. The scoring criteria are: excellent: can be used for high-quality rendering and display; good: suitable for general application, may need slight adjustment; needs improvement: there are obvious problems, some parts need to be reprocessed; unqualified: need to generate a new model.

[0124] S311: According to the human body model detection result, the initial human body model is modified to obtain a three-dimensional human body model.

[0125] If the human body model detection result finds a proportion problem, the body parameters are recalculated, the conservative standard proportion is applied, and the abnormal parameters are gradually adjusted until the proportion requirement is met. If the human body model detection result finds a surface problem, a smoothing filter algorithm is applied to the problem area, and important anatomical features are kept from being excessively smoothed. If the human body model detection result finds a connection problem, the head-body connection algorithm is recalculated, the sampling point density of the transition area is increased, and a higher-order interpolation algorithm is used.

[0126] In some examples, after the initial human body model is detected and modified, the data structure of the obtained three-dimensional human body model can be: complete body model={geometric data:{vertex coordinates: three-dimensional coordinate points of the complete body (about 15,000 points); face connection: triangle face connection information constituting the body surface; skeletal structure: internal skeletal point position and connection relationship (for subsequent animation); joint definition: position and rotation range of main joints}, texture data:{body texture map: complete body surface texture image; material parameters: physical and optical properties of skin; normal vector map: normal vector information for expressing skin details}, parameter data:{body size record: all calculated body parameter values; body type classification label: muscle type / standard type / ample type classification; calculation confidence: confidence score of each parameter calculation}, quality assessment:{overall quality score: comprehensive quality score of the model; part quality: head, torso, and limb quality scores; suggested optimization: if there are problems, the system suggests the improvement direction}}.

[0127] It should be noted that the geometric data in the three-dimensional human body model provides accurate body shape and structure information, the texture data ensures real visual performance, the parameter data supports intelligent selection of clothing sizes, and the quality assessment helps the subsequent system to select the best processing strategy. The three-dimensional human body model is not only consistent with the user's facial features, but also has reasonable body proportions and realistic appearance effects, providing a high-quality basic model for subsequent virtual dressing functions.

[0128] The above-mentioned flow S301-S311 can infer a three-dimensional human body model from a three-dimensional face model, and realize 3D digital human modeling of a user.

[0129] The implementation flow of another virtual avatar dressing method shown in Embodiment Four can be seen from the following steps.

[0130] S401: Obtain the measurement size of each key part in the three-dimensional human body model.

[0131] Among them, the body measurement algorithm can be used to calculate the measurement size of each key part in the three-dimensional human body model.

[0132] In some examples, the body measurement algorithm includes positioning the measurement plane for each key part in the three-dimensional human body model, extracting the cross-sectional contour, calculating the circumference size, and calculating other sizes.

[0133] In possible implementations, the principle of positioning the measurement plane is as follows: chest circumference: through the horizontal plane of the highest point of the chest; waist circumference: through the horizontal plane of the thinnest part of the waist; hip circumference: through the horizontal plane of the widest part of the hips.

[0134] In possible implementations, the principle of extracting the cross-sectional contour is as follows: the cross section of the body model is cut on the measurement plane to obtain the closed contour curve of the part.

[0135] In possible implementations, the principle of calculating the circumference size is as follows: the total length along the contour curve is calculated, which is the girth size of the part.

[0136] In possible implementations, the principle of calculating other sizes is as follows: shoulder width: the straight-line distance between the two shoulder peak points; arm length: the distance along the arm surface from the shoulder peak to the wrist; leg length: the distance along the leg surface from the hip joint to the ankle.

[0137] S402: Determine the clothing template selected by the user in the preset clothing template library.

[0138] Among them, the clothing template library includes clothing templates of various types of clothing, and each clothing template is stored in the form of a 3D grid, containing complete shape, size and attribute information.

[0139] In some examples, the data structure of the garment template includes: garment template data = {geometry information: {mesh vertices: three-dimensional coordinate points constituting the garment surface; face connections: connection relationships between vertices, forming the garment surface; stitching lines: connection boundaries marking parts of the garment; opening positions: definitions of openings such as collar, cuffs, and hem}; size information: {standard sizes: specific sizes corresponding to standard sizes such as S / M / L / XL; key measurement points: measurement positions of key parts such as bust, waist, and hips; ease design: ease settings of each part relative to the human body; version features: digital description of version features such as slim, loose, and straight}; material attributes: {fabric types: fabric classification such as cotton, silk, denim, and knit; elasticity coefficients: stretching ability of the fabric in each direction; weight density: weight of the fabric, affecting drape effect; friction coefficients: smoothness of the fabric surface}}.

[0140] In some examples, the classification system of the garment is: garment classification system = {basic categories: {tops: shirts, T-shirts, sweaters, coats, jackets, etc.; bottoms: pants, shorts, skirts, jeans, etc.; one-pieces: dresses, jumpsuits, suits, etc.; accessories: hats, scarves, ties, etc.}; style labels: {formal: business, formal, professional; casual: daily, sports, home; fashion: trendy, personal, artistic}; applicable scenarios: {work: office, meetings, business negotiations; social: parties, dates, social events; sports: fitness, outdoor, sports activities; daily: home, shopping, rest}}.

[0141] S403: For each key part, determine the matching result of the measurement size of the key part and the plurality of design sizes of the corresponding part in the garment template.

[0142] The matching result includes a matching score.

[0143] In some examples, a size matching algorithm can be used to determine the matching result of the measurement size of the key part and the plurality of design sizes of the corresponding part in the garment template.

[0144] In possible implementations, the algorithm principle of the size matching algorithm is: comparing the measurement size of the key part with the standard size (i.e., the plurality of design sizes, such as XS, S, M, L, XL, and XXL) of the corresponding part in the garment template, specifically, user size = the size of the part measured from the body model, garment size = the design size of the part of the garment under the size, ease = the design ease of the part of the garment, ideal size = user size + ease, size difference = |garment size - ideal size|, if size difference <= 2 cm, then part score = 100 - size difference * 10, otherwise part score = max(0, 80 - size difference * 20), and matching score = part score * importance weight of the part.

[0145] S404: determining the standard size of each key part based on the design size with the highest matching score.

[0146] S405: determining the size adjustment parameter of each part in the garment template based on the standard size of each key part.

[0147] After determining the standard size of each key part, the size adjustment parameter of each part in the garment template needs to be determined for fine-tuning according to the specific body shape of the user.

[0148] In some examples, the principle of obtaining the size adjustment parameter (e.g., adjustment ratio) is as follows: standard size = standard size of the garment in the selected size, user demand size = user body size + ideal ease allowance of the part, adjustment ratio = user demand size / standard size, if the adjustment ratio is between 0.9 and 1.1, linear scaling adjustment is applied, otherwise a non-linear adjustment algorithm is used to maintain the rationality of the garment proportion.

[0149] S406: deforming the garment template based on the size adjustment parameter of each part in the garment template to obtain a three-dimensional garment model.

[0150] By deforming the garment template based on the size adjustment parameter of each part in the garment template, the three-dimensional garment model can be adapted to the body shape of the user.

[0151] In some examples, the garment template can be deformed using a garment deformation algorithm. The garment deformation algorithm includes two steps, namely dividing the garment template into different deformation regions and applying a corresponding deformation operation to each region.

[0152] In possible implementations, the implementation process of dividing the garment template into different deformation regions is as follows: upper garment region = {chest region: needs to adapt to bust size; waist region: needs to adapt to waist size; shoulder region: needs to adapt to shoulder width; sleeve region: needs to adapt to arm length and arm circumference}; lower garment region = {waist region: needs to adapt to waist size; hip region: needs to adapt to hip circumference; thigh region: needs to adapt to thigh circumference; calf region: needs to adapt to calf circumference and length}.

[0153] In possible implementations, the implementation process of applying a corresponding deformation operation to each region is as follows: determining the deformation center point of the region; calculating the required scaling ratio; applying non-uniform scaling transformation; maintaining the continuity of the region boundary.

[0154] In some examples, the deformation process of the garment template includes skeleton-driven deformation, surface detail preservation, proportionality check, suture line and opening processing.

[0155] In possible implementations, the implementation process of the skeleton-driven deformation includes: establishing a virtual skeleton structure inside the garment; the skeleton nodes correspond to key parts of the body; the overall garment deformation is driven by adjusting the skeleton.

[0156] In possible implementations, the implementation process of the surface detail preservation includes: maintaining the design details of the garment during the deformation process; such as pockets, buttons, decorative lines, etc. are not distorted; the deformation amplitude of these areas is limited using constraint conditions.

[0157] In possible implementations, the implementation process of the proportionality check includes: ensuring that the proportions of different parts of the deformed garment are still coordinated, and if a part is deformed too much, the deformation parameters are adjusted or a different size is recommended.

[0158] In possible implementations, a seam adjustment algorithm can be used to process the seams and openings, so that the seams of the garment template (such as side seams, shoulder seams, etc.) need to be adjusted accordingly with the deformation. Specifically, the implementation process of the seam adjustment algorithm is: original length = length of the seam under the standard size, new length = length needed after adjusting according to the body size, if the length change <= 10%, then adjust the positions of all points on the seam in proportion, otherwise increase or decrease the grid density at key positions and recalculate the path of the seam.

[0159] S407: Simulate the physical properties, drape effect, and wrinkles of the cloth of the three-dimensional garment model to obtain physical simulation information of the three-dimensional garment model.

[0160] Among them, the garment is not a rigid body, and the physical properties of real cloth need to be simulated, including drape under the action of gravity, wind influence, contact with the body, etc.

[0161] In some examples, the data structure of the cloth physical properties is: cloth physical properties = {mass properties: {areal density: weight of cloth per square meter (grams); total mass: weight of the entire garment; mass distribution: mass distribution of each part}, elastic properties: {tensile elasticity: elastic coefficient of cloth in the tensile direction; bending elasticity: bending resistance of cloth; shear elasticity: shear deformation resistance of cloth}, damping properties: {internal damping: energy dissipation inside the cloth; air resistance: frictional resistance of cloth and air}}.

[0162] In some examples, the draping effect can be obtained using a draping simulation algorithm, the implementation process of which includes: initializing the simulation environment: setting the gravity direction and size (usually downward, 9.8 m / s²), determining the contact constraints between the clothing and the body, setting the initial state of the cloth (flat or preset shape); force calculation and application: for each cloth vertex, gravity = mass of the vertex x gravity acceleration, elastic force = spring force between adjacent vertices, damping force = resistance proportional to the speed of the vertex, total force = gravity + elastic force + damping force, acceleration = total force / mass, update the position and velocity of the vertex, where new velocity = old velocity + acceleration x time step, new position = old position + new velocity x time step; iterative solution: repeat the above process until the system reaches a stable state, usually 100-500 iterations are needed to obtain a natural draping effect.

[0163] In some examples, the wrinkle is a natural phenomenon of the cloth when it is compressed or bent, and the wrinkle can be obtained using a wrinkle simulation algorithm, the principle of which is: compression detection: for each cloth region, calculate the compression degree of the region, if the compression exceeds the threshold, generate a wrinkle in the region; wrinkle direction determination: determine the direction of the wrinkle according to the direction of compression, usually perpendicular to the main compression direction; wrinkle depth calculation: wrinkle depth = compression degree x cloth thickness coefficient, considering the hardness of the cloth, the harder the cloth, the more obvious the wrinkle; wrinkle shape generation: use a sine wave or other periodic function to generate the geometric shape of the wrinkle to ensure that the wrinkle looks natural and irregular.

[0164] S408: Collision detection and contact processing are performed on the three-dimensional clothing model and the three-dimensional human body model to obtain the wearing simulation information of the three-dimensional clothing model on the three-dimensional human body model.

[0165] Wherein, collision detection needs to be performed between the three-dimensional clothing model and the three-dimensional human body model to ensure that the three-dimensional clothing model does not penetrate the three-dimensional human body model, while simulating the close contact effect of close-fitting clothing.

[0166] In some examples, the principle of the collision detection algorithm is: rough detection stage: use bounding boxes for quick screening, for any body mesh patch in each clothing mesh patch, if the bounding boxes of the two patches intersect, add the candidate list for precise detection; precise detection stage: perform precise geometric intersection test on the candidate patch pairs, use triangle-triangle intersection algorithm to calculate the specific position and depth of intersection; contact point calculation: for each detected collision, calculate the precise position of the contact point, calculate the normal vector of the contact surface, and calculate the penetration depth.

[0167] In some examples, when the clothing is detected to penetrate the body, a correction needs to be made, which includes position correction, velocity correction, and constraint force calculation.

[0168] In possible implementations, the principle of position correction is: for each penetrated clothing vertex, penetration depth = distance from vertex to body surface, correction direction = normal vector of body surface at this point, new position = original position + correction direction x (penetration depth + safety distance).

[0169] In possible implementations, the principle of velocity correction is: stop the motion component of the penetrated vertex in the normal vector direction, retain the tangential motion component, and simulate friction sliding.

[0170] In possible implementations, the principle of constraint force calculation is: calculate the support force of the body on the clothing, which will affect the subsequent motion and deformation of the clothing.

[0171] S409: Material rendering is performed on the three-dimensional clothing model to obtain visual simulation information of the three-dimensional clothing model.

[0172] After the physical simulation is completed, the system needs to perform high-quality material rendering on the three-dimensional clothing model to exhibit the visual characteristics of different fabrics.

[0173] In some examples, the material rendering process includes material property setting, texture mapping application, texture coordinate calculation, light and shadow calculation.

[0174] In possible implementations, the implementation process of material property setting includes: for each fabric type, base color = inherent color of fabric, not affected by light, metallic = 0 (fabric is usually a non-metallic material), roughness = set according to fabric type, such as silk 0.1-0.3 (relatively smooth), cotton 0.4-0.6 (moderate roughness), wool 0.7-0.9 (relatively rough), subsurface scattering = simulate light scattering effect inside the fabric.

[0175] In possible implementations, the implementation process of texture mapping application includes: color map: map the color pattern of the clothing to the 3D surface, handle the repetition, scaling and rotation of the pattern; normal map: simulate the subtle concave-convex texture of the fabric surface, such as the weaving texture of denim, the needle hole texture of knitted goods; detail map: add wear, stain, crease and other detail effects to enhance the realism of the clothing.

[0176] In possible implementations, the implementation process of texture coordinate calculation includes: for each clothing vertex, texture coordinates = corresponding position of the vertex on the 2D texture map, considering the effect of clothing deformation on the texture, ensuring that the texture will not be severely distorted due to deformation.

[0177] In possible implementations, the implementation process of the light and shadow calculation includes: direct light: direct illumination from main light sources (such as the sun, light), calculate the direction, intensity and color of the light; Indirect lighting: ambient light and light reflected by other objects, simulate the complex light propagation in reality; Shadow generation: calculate the shadow cast by the clothing on the body, simulate the shadow effect inside the clothing.

[0178] In some examples, the rendering equation adopted by the material rendering can be: final color = material base color x (direct light + indirect light) + self-luminous, where each term is weighted and calculated considering the physical properties of the material.

[0179] S410: Based on the physical simulation information, the wearing simulation information and the visual simulation information, generate virtual dressing effect data of the three-dimensional human body model wearing the three-dimensional clothing model.

[0180] Among them, the display interface runs the virtual dressing effect data to show the virtual image dressing picture.

[0181] In some examples, the data structure of the virtual dressing effect data is: virtual dressing result = {geometry data: {clothing mesh: the fitted three-dimensional mesh model of the clothing; vertex position: the final three-dimensional coordinates of all vertices; normal vector: the surface normal vector of each vertex; texture coordinate: the texture coordinate corresponding to each vertex;}, physical state: {static position: stable state under the action of gravity; dynamic response: response parameters to body movement; contact relationship: contact point information between clothing and body; constraint condition: numerical value of various physical constraints}, rendering data: {material parameter: complete PBR material property; texture map: all related texture images; lighting setting: lighting parameters suitable for the clothing; shadow information: precomputed shadow data}, fitting information: {size matching: selected size and adjustment parameters; deformation record: all deformation operations applied; quality evaluation: quality score of fitting effect; suggestion optimization: if necessary, system improvement suggestion}}.

[0182] It should be noted that the geometry data in the virtual dressing effect data provides a complete three-dimensional clothing model, the physical state supports subsequent dynamic response calculation, the rendering data ensures high-quality visual effect, and the fitting information helps the system optimize the subsequent processing flow. The virtual dressing effect data successfully fits the virtual clothing to the three-dimensional human body model of the user, realizing a realistic virtual fitting effect. The clothing not only perfectly matches the user's body in size, but also has real physical properties and visual performance, laying a solid foundation for subsequent dynamic performance.

[0183] The above-mentioned S401-S410 process can obtain virtual dressing effect data of the three-dimensional human body model wearing the three-dimensional clothing model.

[0184] The implementation process of another virtual image dressing method shown in Embodiment Five can refer to the following steps.

[0185] S501: Use the trained face detection network to perform face detection on the standardized image data at different scales to obtain a plurality of candidate regions.

[0186] Among them, a multi-scale detection strategy is adopted to ensure that the face region in the picture can be accurately identified regardless of the size of the face in the standardized image data.

[0187] In some examples, the principle of the multi-scale detection strategy is that the goal of face detection is to find a rectangular region containing a face in the picture. Since the face in the user input personal photo may occupy different proportions (some are half-length photos and some are full-length photos), detection at different scales is needed.

[0188] Generally, the entire standardized image data is scanned using the trained face detection network. The working principle of the face detection network is similar to that of an intelligent magnifying glass. It determines whether there are face features at each position of face detection. For each possible position, the face detection network gives a confidence score between 0 and 1. The higher the score, the greater the likelihood that the position contains a face.

[0189] S502: Use a boundary regression algorithm to finely adjust the boundaries of the plurality of candidate regions to obtain a plurality of corresponding face boundary boxes and determine a plurality of corresponding candidate face regions.

[0190] Among them, the process of fine boundary adjustment can be summarized as follows: taking the candidate region obtained by rough positioning as input, finely adjusting the boundary of each candidate region, and using the boundary regression algorithm to optimize the position and size of the face box to obtain an accurate face boundary box (e.g., left upper corner x coordinate, left upper corner y coordinate, width, height).

[0191] Generally, the boundary of the initially detected face region is optimized. This process is similar to accurately framing a painting to ensure that the face is in the best position in the frame, neither cutting off important facial features nor including too much irrelevant background.

[0192] S503: Obtain a comprehensive score for each candidate face region and determine the face region in the standardized image data based on the candidate face region with the highest comprehensive score.

[0193] Among them, when multiple faces are detected in the standardized image data (such as a group photo) or the same face is detected multiple times, the face most suitable for 3D reconstruction needs to be selected. Therefore, the comprehensive score of each candidate face region needs to be obtained.

[0194] In some examples, the comprehensive score comprises a weighted sum of the size score, the sharpness score, and the frontalness score, for example, comprehensive score = size score x 0.3 + sharpness score x 0.4 + frontalness score x 0.3.

[0195] S504: Cropping a face image corresponding to the face region from the standardized image data, and performing key point detection on the face image by using the trained key point detection network to obtain a plurality of candidate key points.

[0196] After determining the face region, accurate key point positioning is further needed, and these key points are basic data for modeling the three-dimensional face model.

[0197] In possible implementation processes, the implementation process of key point detection can be summarized as follows: inputting a face image, adjusting the face image to a standard size of 96x96 pixels, and performing preliminary positioning by using a specially trained key point detection network to obtain the approximate coordinate positions of 68 key points.

[0198] Generally, the key point detection network is used to identify key points, and the key point detection network is trained by hundreds of thousands of labeled face pictures to learn the position rules of key points in different face shapes and different expressions.

[0199] S505: Fine adjustment is performed on each candidate key point to obtain each effective key point.

[0200] The implementation process of fine adjustment can be summarized as follows: inputting the approximate coordinate positions of 68 key points, performing high-precision analysis on a small area around each key point, searching for the most accurate position within a 16x16 pixel area around each point, and thus obtaining the key point coordinates with sub-pixel accuracy.

[0201] Generally, in order to achieve higher accuracy, fine adjustment can be performed on each key point. This process is similar to observing each feature point with a magnifying glass to find the most accurate position in a small range.

[0202] S506: Detecting each effective key point by using a preset key point quality detection mechanism to obtain a quality detection result of each key point.

[0203] After the positioning of the effective key points is completed, the rationality of the effective key points needs to be verified and necessary corrections are needed, and for this purpose, each effective key point is detected.

[0204] In some examples, the key point quality detection mechanism comprises geometric consistency detection and symmetry detection.

[0205] S507: Identifying abnormal key points according to the quality detection results of the key points, and correcting the abnormal key points to obtain each key point.

[0206] Wherein, when obvious unreasonable abnormal key points are found, the abnormal key points can be corrected by correction strategy, and the effective key points with minimum correction error are selected and marked as key points.

[0207] S508: Generate key point data based on each key point and second information.

[0208] Wherein, the second information includes the confidence and quality detection result of each key point, the face boundary box corresponding to the face region, and the geometric feature composed of each key point.

[0209] It should be noted that the structured key point data can have the following benefits: the point coordinate list provides accurate feature position information, the confidence helps the subsequent algorithm to judge which points are more reliable, the geometric feature provides a reference for the size estimation of the three-dimensional face model, and the quality detection result helps the system to adjust the subsequent processing strategy.

[0210] The above-mentioned S501-S508 process obtains the key point data of the user's face, which lays a data foundation for subsequent generation of realistic three-dimensional face models.

[0211] The implementation process of another virtual image dressing method shown in embodiment six can be seen from the following steps.

[0212] S601: Obtain a pre-established standard three-dimensional face template.

[0213] Wherein, the standard three-dimensional face template includes deformation parameters, a plurality of standard points, and corresponding depth information.

[0214] S602: Determine the depth estimation value of each key point based on the relative position relationship between each key point in the key point data.

[0215] Wherein, since the input personal photo is planar and lacks depth information, the front and back position relationship of each facial feature needs to be inferred, so the depth estimation value of each key point is determined based on the relative position relationship between each key point in the key point data.

[0216] In some examples, based on 68 key points, the relative position relationship between each key point is analyzed, and the depth constraint is established in combination with the knowledge of human face anatomy to obtain the depth estimation value of each key point.

[0217] S603: Refine the depth estimation value of each key point by using light and shadow analysis method to obtain the depth information of each key point.

[0218] The depth estimation value is refined by analyzing light and shadow information in the personal photo, so as to improve the accuracy of the depth estimation value. The depth information includes the corrected depth estimation value.

[0219] S604: Based on the matching result between the depth information of each key point and the depth information of each standard point, the parameter value of the deformation parameter is determined, and the standard three-dimensional face template is adjusted and smoothed based on the parameter value to obtain a face deformation model.

[0220] Wherein, after obtaining the depth information of each key point, the depth information of each key point is matched with the depth information of each standard point in the standard three-dimensional face template to obtain the corresponding matching result.

[0221] S605: Fine-tune the face deformation model using the face information of the personal photo to obtain an initial face model.

[0222] Wherein, fine-tuning the face deformation model using the face information of the personal photo can make the initial face model look more realistic.

[0223] In some examples, the face deformation model can be divided into different face texture regions according to the face information, and each face texture region has a different processing strategy, such as sampling correction, illumination correction, etc.

[0224] S606: Detect the initial face model using a preset face model quality detection mechanism to obtain a face model detection result.

[0225] Wherein, the face model quality detection mechanism includes geometric reasonableness check and visual consistency verification.

[0226] S607: According to the face model detection result, the initial face model is corrected to obtain a three-dimensional face model.

[0227] Wherein, when problems are found in the face model detection result, a hierarchical correction strategy can be used to correct the initial face model to obtain a three-dimensional face model.

[0228] It should be noted that the three-dimensional face model is used to provide basic data support for subsequent inference of a three-dimensional human body model (i.e. 3D digital person), wherein the geometric data provides an accurate three-dimensional shape of the face, the texture data ensures a realistic visual effect, the feature data supports subsequent expression animation functions, and the quality evaluation helps the system optimize subsequent processing strategies. That is, converting the user's personal photo into a three-dimensional face model not only preserves the user's facial features, but also has the ability to perform various operations in three-dimensional space, laying an important foundation for subsequent full-body modeling and virtual dressing.

[0229] The flow shown in S601-S607 can realize effective modeling of the three-dimensional face model.

[0230] The implementation flow of another virtual image dressing method shown in Embodiment Seven can be seen from the following steps.

[0231] S701: Obtain user facial features and user body posture corresponding to the dynamic performance behavior based on the visual signal captured by the camera in real time.

[0232] The visual signal can be a video stream.

[0233] In some examples, the user facial features can be user facial expressions, which can be recognized by using a facial expression algorithm.

[0234] In some examples, a full-body posture detection algorithm can be used to recognize the user body posture from the video stream.

[0235] S702: Coordinate and synchronize the user facial features and the user body posture to obtain facial expression parameters and body posture parameters.

[0236] The coordination and synchronization of the user facial features and the user body posture can make the virtual image dressing effect natural and coordinated.

[0237] In some examples, a coordination detection algorithm can be used to analyze the matching degree of the user facial features and the user body posture to avoid uncoordinated combinations. The coordination detection algorithm includes emotion consistency checking and intensity coordination adjustment.

[0238] S703: Update the three-dimensional human body model by using the facial expression parameters and update the three-dimensional human body model by using the body posture parameters to obtain an updated three-dimensional human body model.

[0239] The facial expression parameters can be used to reconstruct the face part of the three-dimensional human body model to realize the expression update of the three-dimensional human body model.

[0240] S704: Re-simulate the new virtual dressing effect data of the updated three-dimensional human body model wearing the three-dimensional clothing model.

[0241] The three-dimensional clothing model also needs to be updated accordingly as the body action (i.e., the posture of the three-dimensional human body model) changes.

[0242] S705: Update the virtual dressing effect data in the display interface based on the new virtual dressing effect data to make the display interface display a virtual image preview picture corresponding to the dynamic performance behavior.

[0243] Among them, real-time visual feedback (i.e. virtual avatar preview picture) needs to be provided to enable users to intuitively see the expression and action of their own on the 3D digital person.

[0244] It should be noted that the virtual clothes changing effect data successfully realizes the real-time synchronization linkage of user expression and action and the 3D digital person. The entire 3D digital person not only has static visual effects, but also dynamically responds to every expression change and body action of the user, creating a highly immersive virtual physical experience. From the initial photo input to the final real-time interaction, the entire scheme forms a complete closed loop.

[0245] The above-mentioned processes S701-S705 can realize the linkage of the three-dimensional human body model and the dynamic performance behavior of the user, and improve the user experience.

[0246] Embodiment eight as Figure 2 The virtual avatar clothes changing system provided by the embodiment of the application includes the following units.

[0247] The character image processing unit 100 is configured to pre-process the character image input by the user to obtain standardized image data.

[0248] Optionally, the character image processing unit 100 is specifically configured to: obtain the character image input by the user; perform standardization processing on the character image to obtain a standardized image matrix; detect the standardized image matrix by using a preset image quality detection mechanism to obtain an image quality detection result; the image quality detection result includes a definition detection result, a lighting quality inspection result, and a face integrity inspection result; in the case that any one of the definition detection result, the lighting quality inspection result, and the face integrity inspection result meets a corresponding optimization trigger condition, perform optimization processing on the standardized image matrix to obtain an optimized image matrix; perform size standardization and pixel value normalization on the optimized image matrix to obtain an image matrix; generate the standardized image data based on the image matrix and first information; the first information includes the original size of the character image, the image quality detection result, and an optimization parameter applied when the standardized image matrix is optimized.

[0249] The face model construction unit 200 is configured to identify the face region in the standardized image data and construct a three-dimensional face model.

[0250] Optionally, the face model construction unit 200 is specifically configured to: identify the face region in the standardized image data; generate key point data based on a plurality of key points in the face region; and construct a three-dimensional face model based on the key point data.

[0251] The digital person generation unit 300 is configured to determine a body parameter corresponding to the three-dimensional face model, and generate a three-dimensional human body model based on the body parameter.

[0252] Optionally, the digital human generation unit 300 is specifically configured to: position a plurality of landmark points on the three-dimensional face model, and obtain a plurality of face key sizes based on the Euclidean distances between the landmark points; determine a plurality of face features by using a classification algorithm to perform type judgment on the face key sizes; and determine the body parameters corresponding to the plurality of face features based on a pre-established mathematical correlation model between the face features and the body parameters.

[0253] Optionally, the digital human generation unit 300 is specifically configured to: determine the basic body parameters corresponding to the plurality of face features based on the pre-established mathematical correlation model between the face features and the body parameters; fuse the determined basic body parameters with pre-collected user information to obtain a plurality of fused body parameters; detect each fused body parameter by using a preset body parameter detection mechanism to obtain a parameter detection result of each fused body parameter; identify an abnormal body parameter according to the parameter detection result of each fused body parameter, and correct the abnormal body parameter to obtain each body parameter.

[0254] Optionally, the digital human generation unit 300 is specifically configured to: select a body template matching each body parameter from a body template library, and deform and locally adjust the body template to obtain a three-dimensional body model; perform connection processing on the three-dimensional face model and the three-dimensional body model to obtain a complete human body model; fine-tune the complete human body model according to the texture data of the three-dimensional face model to obtain an initial human body model; detect the initial human body model by using a preset human body model quality detection mechanism to obtain a human body model detection result; and correct the initial human body model according to the human body model detection result to obtain a three-dimensional human body model.

[0255] The dressing simulation unit 400 is configured to deform a garment template based on a matching result between the measured sizes of each key part in the three-dimensional human body model and the designed sizes of the corresponding parts in the preset garment template, to obtain a three-dimensional garment model, and simulate virtual dressing effect data of the three-dimensional human body model wearing the three-dimensional garment model.

[0256] Optionally, the refitting simulation unit 400 is specifically configured to: acquire the measurement sizes of each key part in the three-dimensional human body model; determine a clothing template selected by the user in a preset clothing template library, the clothing template being obtained by modeling clothing based on a three-dimensional grid; for each key part, determine a matching result of the measurement size of the key part and a plurality of design sizes of a corresponding part in the clothing template; the matching result includes a matching score; determine a standard size of the key part based on the design size with the highest matching score; determine size adjustment parameters of each part in the clothing template based on the standard sizes of each key part; deform the clothing template based on the size adjustment parameters of each part in the clothing template to obtain a three-dimensional clothing model; simulate the physical properties, draping effect and wrinkles of the three-dimensional clothing model to obtain physical simulation information of the three-dimensional clothing model; perform collision detection and contact processing on the three-dimensional clothing model and the three-dimensional human body model to obtain wearing simulation information of the three-dimensional clothing model on the three-dimensional human body model; perform material rendering on the three-dimensional clothing model to obtain visual simulation information of the three-dimensional clothing model; and generate virtual dressing effect data of the three-dimensional human body model wearing the three-dimensional clothing model based on the physical simulation information, the wearing simulation information and the visual simulation information.

[0257] The performance behavior preview unit 500 is configured to update the three-dimensional human body model by using the real-time acquired user facial features and user body posture, and update the virtual dressing effect data according to the updated three-dimensional human body model to obtain a virtual image preview picture corresponding to the dynamic performance behavior of the user.

[0258] The above-mentioned various units generate a three-dimensional human body model according to a user inputted character image, simulate the three-dimensional human body model wearing the three-dimensional clothing model through the virtual dressing effect data to realize virtual image dressing, and display a virtual image preview picture corresponding to the dynamic performance behavior of the user by updating the virtual dressing effect data, thereby effectively improving user experience.

[0259] The present application also provides a computer readable storage medium, which includes a stored program, wherein the program executes the virtual image dressing method provided by the present application.

[0260] The present application also provides an electronic device, which includes a processor, a memory and a bus. The processor is connected with the memory through the bus, the memory is used to store a program, and the processor is used to run the program, wherein the program runs to execute the virtual image dressing method provided by the present application.

[0261] While several inventive embodiments have been described above, it should be appreciated that many modifications can be made of these embodiments in light of the above disclosure. Therefore, the disclosed embodiments are not intended to limit the scope of the application to the particular embodiments disclosed herein but can be practiced with modifications and changes by those having ordinary skill in the art without departing from the scope of the following claims.

[0262] The above description is merely illustrative of the application and the application should not be limited to the specific embodiments that have been described, which are to be regarded as illustrative rather than a limitation on the scope of the application. It will be apparent to those skilled in the art that various modifications can be made to the application as described without departing from the scope of the application. Thus, other embodiments falling within the scope of the application are also contemplated. For example, structural, functional, as well as conceptual equivalents are considered reasonably within the scope of the application.

Claims

1. A virtual character clothes changing method characterized by, The method comprises the following steps: Preprocessing a user-inputted character image to obtain standardized image data; Identifying a face region in the standardized image data and constructing a three-dimensional face model; Positioning a plurality of landmark points on the three-dimensional face model and obtaining a plurality of face key dimensions based on the Euclidean distances between the landmark points; Using a classification algorithm to determine the types of the face key dimensions to determine a plurality of face features; Determining a plurality of body parameters corresponding to the plurality of face features based on a pre-established mathematical correlation model between the face features and the body parameters, wherein the body parameters are obtained by fusing the user information and the basic body parameters corresponding to the plurality of face features determined according to the mathematical correlation model, and the abnormal body parameters are corrected using the parameter detection results of the fused body parameters; Generating a three-dimensional human body model based on the body parameters; Deforming a preset garment template based on the matching results between the measurement dimensions of each key part in the three-dimensional human body model and the design dimensions of the corresponding parts in the garment template to obtain a three-dimensional garment model, and simulating a virtual dressing effect data of the three-dimensional human body model wearing the three-dimensional garment model; Updating the three-dimensional human body model using the real-time obtained user face features and user body posture, and updating the virtual dressing effect data according to the updated three-dimensional human body model to obtain a virtual avatar preview screen corresponding to the dynamic performance behavior of the user.

2. The method of claim 1, wherein, The preprocessing of the user-inputted character image to obtain standardized image data comprises the following steps: Obtaining a user-inputted character image; Standardizing the character image to obtain a standardized image matrix; Detecting the standardized image matrix using a preset image quality detection mechanism to obtain an image quality detection result; the image quality detection result comprises a definition detection result, a lighting quality inspection result, and a face integrity inspection result; Optimizing the standardized image matrix to obtain an optimized image matrix when any one of the definition detection result, the lighting quality inspection result, and the face integrity inspection result satisfies a corresponding optimization trigger condition; Standardizing the size and normalizing the pixel value of the optimized image matrix to obtain an image matrix; Generating standardized image data based on the image matrix and first information; the first information comprises the original size of the character image, the image quality detection result, and the optimization parameters applied when the standardized image matrix is optimized.

3. The method of claim 1, wherein, The identification of the face region in the standardized image data and the construction of the three-dimensional face model comprise the following steps: Identifying a face region in the standardized image data; Generating key point data based on a plurality of key points in the face region; Constructing a three-dimensional face model based on the key point data.

4. The method of claim 1, wherein, The determination of a plurality of body parameters corresponding to a plurality of face features based on a pre-established mathematical correlation model between the face features and the body parameters comprises: Determine a basic body parameter corresponding to each of the facial features based on a pre-established mathematical correlation model between the facial features and the body parameters; Fuse each of the determined basic body parameters with pre-collected user information to obtain a plurality of fused body parameters; Detect each of the fused body parameters using a pre-set body parameter detection mechanism to obtain a parameter detection result of each of the fused body parameters; Identify an abnormal body parameter according to the parameter detection result of each of the fused body parameters, and correct the abnormal body parameter to obtain each of the body parameters.

5. The method of claim 4, wherein, Generate a three-dimensional human body model based on the body parameters, including: Select a body template matching each of the body parameters from a body template library, and deform and locally adjust the body template to obtain a three-dimensional body model; Connect the three-dimensional face model and the three-dimensional body model to obtain a complete human body model; Fine-tune the complete human body model according to the texture data of the three-dimensional face model to obtain an initial human body model; Detect the initial human body model using a pre-set human body model quality detection mechanism to obtain a human body model detection result; Correct the initial human body model according to the human body model detection result to obtain a three-dimensional human body model.

6. The method of claim 1, wherein, Deform a pre-set garment template based on the matching result between the measured size of each key part in the three-dimensional human body model and the design size of the corresponding part in the garment template to obtain a three-dimensional garment model, and simulate the virtual dressing effect data of the three-dimensional human body model wearing the three-dimensional garment model, including: Obtain the measured size of each key part in the three-dimensional human body model; Determine the garment template selected by the user in the pre-set garment template library; the garment template is obtained by modeling the garment based on a three-dimensional grid; For each of the key parts, determine the matching result of the measured size of the key part and a plurality of design sizes of the corresponding part in the garment template; the matching result includes a matching score; Determine the standard size of the key part based on the design size with the highest matching score; Determine the size adjustment parameter of each part in the garment template based on the standard size of each of the key parts; Deform the garment template based on the size adjustment parameter of each part in the garment template to obtain a three-dimensional garment model; Simulate the physical properties, drape effect, and wrinkles of the three-dimensional garment model to obtain physical simulation information of the three-dimensional garment model; Perform collision detection and contact processing on the three-dimensional garment model and the three-dimensional human body model to obtain wearing simulation information of the three-dimensional garment model on the three-dimensional human body model; Render the material of the three-dimensional garment model to obtain visual simulation information of the three-dimensional garment model; Generate the virtual dressing effect data of the three-dimensional human body model wearing the three-dimensional garment model based on the physical simulation information, the wearing simulation information, and the visual simulation information.

7. A virtual character dressing system characterized by, ​ The character image processing unit is configured to pre-process a character image input by a user to obtain standardized image data. The face model construction unit is configured to identify a face region in the standardized image data and construct a three-dimensional face model. The digital human generation unit is configured to locate a plurality of landmark points on the three-dimensional face model and obtain a plurality of face key sizes based on Euclidean distances between the landmark points; determine a plurality of face features by performing type judgment on the face key sizes using a classification algorithm. Based on a pre-established mathematical correlation model between face features and body parameters, body parameters corresponding to the face features are determined, including: fusing a plurality of fusion body parameters obtained by fusing user information and basic body parameters corresponding to the face features determined according to the mathematical correlation model, and correcting each body parameter by using a parameter detection result of each fusion body parameter; generating a three-dimensional human body model based on the body parameters; The dressing simulation unit is configured to deform a preset garment template based on matching results between measured sizes of each key part in the three-dimensional human body model and design sizes of corresponding parts in the garment template to obtain a three-dimensional garment model, and simulate virtual dressing effect data of the three-dimensional human body model wearing the three-dimensional garment model; the garment template is obtained by modeling a garment based on a three-dimensional grid. The performance behavior preview unit is configured to update the three-dimensional human body model using real-time acquired user face features and user body postures, and update the virtual dressing effect data according to the updated three-dimensional human body model to obtain a virtual avatar preview screen corresponding to dynamic performance behaviors of the user.

8. A storage medium, characterized by The storage medium includes a stored program, wherein the program is executed by the processor to perform the virtual avatar dressing method of any one of claims 1-6.

9. An electronic device, comprising: It includes: a processor, a memory and a bus; The processor and the memory are connected through the bus; The memory is used to store programs, and the processor is used to run programs, wherein the programs are executed by the processor to perform the virtual avatar dressing method of any one of claims 1-6.

Images