3D virtual image model training method based on deep learning

Abstract

The present invention discloses a 3D virtual image model training method based on deep learning, which belongs to the field of computer technology. In order to solve the problem of low feature extraction efficiency and inability to accurately measure the quality of the generated 3D model, through the deep learning architecture, the model efficiently extracts key features from 2D images for high-quality 3D model reconstruction, compresses the features into low-dimensional vectors to reduce the calculation cost, and retains key information at the same time. The autoencoder or its variants decode the low-dimensional vectors, and the mapping relationship from 2D to 3D can be learned during the training process. The quality of the generated 3D model can be optimized by adjusting parameters, batch training improves resource utilization and training stability, reduces noise, and a specific loss function accurately measures model performance, guides parameter optimization, and validation set evaluation promptly discovers training problems. The calculation error accurately evaluates the model performance, optimizes the model architecture and parameters, and improves the 3D virtual image reconstruction performance.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a 3D virtual image model training method based on deep learning. Background Art

[0002] 3D virtual images, as an outstanding representative of modern computer graphics and digital art, are an important product of the digital age. In many fields such as entertainment, education, medical care and industrial design, 3D virtual images have shown strong application potential and value. They can not only provide users with an immersive experience, but also simulate the interactions and behaviors of the real world in different scenarios, bringing a more colorful experience to people's digital life.

[0003] With the rapid development of science and technology, deep learning has shown great potential in image processing, computer vision and other fields. In particular, in the generation of 3D virtual images, traditional technical methods can no longer meet the requirements of high quality and high efficiency. Therefore, a 3D virtual image model training method based on deep learning has emerged.

[0004] However, there are still the following problems in actual operation:

[0005] Existing methods cannot efficiently extract key feature information from 2D images, resulting in low quality of reconstructed 3D models. At the same time, traditional training methods may be inefficient and unable to fully utilize computing resources, resulting in a long training process and an inability to accurately measure the quality of the generated 3D models. Summary of the invention

[0006] The purpose of the present invention is to provide a 3D virtual image model training method based on deep learning to solve the problems raised in the above background technology.

[0007] To achieve the above object, the present invention provides the following technical solution: a 3D virtual image model training method based on deep learning, comprising the following steps:

[0008] Data preparation, obtaining training data, the training data including 2D images and corresponding 3D models, wherein the 2D images include close-ups of the face, the whole body, and body parts, and the 3D models include corresponding detailed geometric shapes and textures, preprocessing the 2D images, adjusting the consistency of the input data, converting the 3D models into a unified format, and scaling and rotating the converted 3D models;

[0009] Model design: build a deep learning model that includes a convolutional neural network and an autoencoder. The convolutional neural network is used to extract feature information from 2D images, and the autoencoder is used to learn the encoding and decoding process of data and reconstruct a 3D model. The preprocessed 2D image is input into the deep learning model, and the key feature information of the face is extracted through the convolutional neural network. The key feature information is encoded into a low-dimensional vector representation. Based on the encoded feature vector, the low-dimensional vector is decoded into 3D model data;

[0010] Model training, constructing a loss function to guide the training of the deep learning model, wherein the loss function is used to measure the difference between the generated 3D model and the target 3D model, using the collected training data to train the deep learning model, optimizing the parameters of the model during the training process, and monitoring the performance of the model through the validation set, and saving the parameters of the model when the validation set performs well;

[0011] Model evaluation: After training, the model is evaluated using the test set to measure the model performance. Evaluation indicators include reconstruction error and user satisfaction. The model is adjusted based on the evaluation results. Model adjustments include model architecture adjustment, parameter adjustment, and loss function adjustment.

[0012] Application optimization is to put the trained 3D virtual image model into practical applications, including artistic creation, game design, and virtual fitting. During the actual application process, user feedback and data are collected to optimize and improve the model.

[0013] Furthermore, the data preparation further includes the following steps:

[0014] Acquire 2D images containing different faces, whole bodies, and close-ups of body parts, and collect 3D models corresponding to the 2D images, wherein the 3D models include device scanned models, manually created models, or models generated by modeling software;

[0015] Cropping removes the background part of the 2D image so that the target area becomes the center of the image, and the target area includes the face and the whole body. The image is scaled to 256x256 pixels and normalized to map the pixel values ​​to the range of [0, 1] or [-1, 1].

[0016] Convert all 3D models into a unified file format, set the origin of each 3D model uniformly, and ensure that the coordinate systems of all 3D models are aligned;

[0017] Pair each 2D image with its corresponding 3D model and save the pairing information.

[0018] Furthermore, the data preparation further includes:

[0019] Extracting a region shape of the target region before the scaling operation as first region shape data;

[0020] extracting a region shape of the target region after the scaling operation as second region shape data;

[0021] The shape change parameter is obtained by using the first region shape data and the second region shape data; wherein the shape change parameter is obtained by the following formula:

[0022]

[0023] Among them, R x represents the shape change parameter; R x0 Indicates the reference value of the preset shape change parameter; S y Represents the preset shape similarity threshold; S x represents the shape similarity value between the region shape of the target region before the scaling operation and the region shape of the target region after the scaling operation; x represents the first adjustment coefficient, and the first adjustment coefficient is obtained by the following formula:

[0024]

[0025] Wherein, x represents the first adjustment coefficient; m×m represents the pixel size of the target area before the scaling operation, corresponding to the target pixel size 256x256 corresponding to the scaled target area;

[0026] comparing the shape change parameter with a preset shape change parameter threshold;

[0027] When the shape change parameter exceeds a preset shape change parameter threshold, it is determined that the scaling quality of the target area is abnormal, and an abnormal alarm is issued;

[0028] When the shape change parameter does not exceed the preset shape change parameter threshold, a secondary abnormality determination is performed on the scaling quality of the target area using the gray value change parameters of the target area before the scaling operation and the target area after the scaling operation.

[0029] Further, when the shape change parameter does not exceed a preset shape change parameter threshold, a secondary abnormality determination is performed on the scaling quality of the target area using the gray value change parameters of the target area before the scaling operation and the target area after the scaling operation, including:

[0030] Extracting the grayscale value of the pixel block contained in the target area before the scaling operation as the first area grayscale value data;

[0031] Extracting the grayscale value of the pixel block contained in the target area after the scaling operation as the second area grayscale value data;

[0032] Extracting grayscale values ​​of grayscale values ​​of 0 and 255 in the grayscale value data of the first region to form a first pixel value set;

[0033] Extracting grayscale values ​​of grayscale values ​​of 0 and 255 in the grayscale value data of the second region to form a second pixel value set;

[0034] The grayscale change parameter is obtained by using the first pixel value set and the second pixel value set, wherein the grayscale change parameter is obtained by the following formula:

[0035]

[0036] Among them, R h Represents the grayscale change parameter; R h0 Indicates the preset reference value of the grayscale change parameter; h 01max and h 01min represent the maximum grayscale value and the minimum grayscale value contained in the first pixel value set respectively; h 02max and h 02min They respectively represent the maximum grayscale value and the minimum grayscale value contained in the second pixel value set; r represents the second adjustment coefficient, and the second adjustment coefficient is obtained by the following formula:

[0037]

[0038] Wherein, r represents the second adjustment coefficient; a represents the number of gray value elements included in the first pixel value set; b represents the number of gray value elements included in the second pixel value set; h i represents the corresponding value of the i-th grayscale value in the first pixel value set; h i-1 represents the corresponding value of the i-1th grayscale value in the first pixel value set; h p01 represents the grayscale average value corresponding to the grayscale value elements in the first pixel value set; j represents the corresponding value of the jth grayscale value in the second pixel value set; j-1 represents the corresponding value of the j-1th grayscale value in the second pixel value set; p02 Represents the grayscale average value corresponding to the grayscale value elements in the second pixel value set;

[0039] Comparing the grayscale change parameter with a preset grayscale change parameter threshold;

[0040] When the grayscale change parameter is lower than a preset grayscale change parameter threshold, it is determined that the scaling quality of the target area is abnormal, and an abnormality alarm is issued.

[0041] Furthermore, the data preparation further includes the following steps:

[0042] Divide the preprocessed data into training set, validation set and test set;

[0043] Among them, 80% of the data is used as a training set, which is used to train the deep learning model; 10% of the data is used as a validation set, which is used to verify the model performance and adjust the hyperparameters; and 10% of the data is used as a test set, which is used to finally test the model performance.

[0044] Furthermore, the model design further includes the following steps:

[0045] Determine the model architecture, extract key feature information from the 2D image as a feature vector, compress the feature information into a low-dimensional vector representation, receive the extracted feature vector as input, encode the input data into a low-dimensional vector representation, and output the encoded low-dimensional vector;

[0046] The low-dimensional vector output by the encoder is received as input, 3D model data is decoded from the code, and the 3D model data is output.

[0047] Furthermore, the model training further includes the following steps:

[0048] According to the specific task requirements, add loss functions, load training data through the data loader, including the preprocessed 2D images and corresponding target 3D models in the training set, and divide the data into multiple batches for training, each batch contains a certain number of samples;

[0049] Initialize the deep learning model based on the deep learning framework. For each training batch, input the 2D image into the model, calculate the generated 3D model through forward propagation, and use the loss function to calculate the difference between the generated 3D model and the target 3D model to obtain the loss value.

[0050] Backpropagation is performed based on the loss value to calculate the gradient and update the model parameters.

[0051] Furthermore, the model training further includes the following steps:

[0052] Load the validation set data through the data loader. After each training cycle, use the validation set to evaluate the model. Input the validation set data into the model, calculate the difference between the generated 3D model and the target 3D model, obtain the validation loss, and monitor the change of the validation loss.

[0053] Set the trigger conditions for saving the model, which include the verification loss decreasing for a specific number of consecutive cycles and reaching a preset verification accuracy. When the saving conditions are met, the parameters of the current model are saved to disk;

[0054] Record the training information during the training process, including the loss value, accuracy rate, and learning rate, use visualization tools to visualize the training process, and generate data curves based on the training information.

[0055] Furthermore, the model evaluation further includes the following steps:

[0056] Load the trained deep learning model and input the preprocessed test set 2D images into the model. The model extracts features through a convolutional neural network, encodes the features into low-dimensional vectors using an autoencoder, and decodes them into 3D model data.

[0057] Calculate the error between the generated 3D model and the real 3D model, analyze the distribution of the reconstruction error, obtain the evaluation results, and adjust the model architecture, parameters and loss function based on the evaluation results;

[0058] The process of model evaluation, analysis, and tuning is repeated until the model performance reaches the desired level.

[0059] Furthermore, the application optimization further includes the following steps:

[0060] Integrate the trained 3D avatar model into the corresponding application platform and software, provide users with clear usage guides or tutorials, and set up a feedback mechanism, which includes a rating system, questionnaires, and online chats. During the user's use, collect users' evaluations of model performance and application experience in real time;

[0061] Based on user feedback and data analysis results, the performance and functionality of the model are improved and expanded, the optimized and improved model is redeployed to the application platform, and user usage and feedback are continuously monitored after the application is updated.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. The present invention uses a convolutional neural network or other deep learning architecture, and the model can efficiently extract key feature information from 2D images. This feature information is crucial for reconstructing high-quality 3D models. Compressing the extracted feature information into a low-dimensional vector representation can greatly reduce the dimension of the data while retaining key information, which not only reduces the computational cost but also helps the model learn more efficient representations. Using an autoencoder or its variants as part of the model enables the model to decode 3D model data from a low-dimensional vector, learn the mapping relationship from 2D to 3D during the training process, and optimize the quality of the generated 3D model by adjusting the parameters of the encoder and decoder.

[0064] 2. The present invention divides the data into multiple batches for training, so that the model can use computing resources more effectively and accelerate the training process. Batch training also helps to reduce noise in gradient calculations and improve the stability of model training. Adding a loss function that matches the specific task requirements can accurately measure the difference between the generated 3D model and the target 3D model, thereby guiding the model to perform precise parameter optimization. After each training cycle, the model is evaluated using a validation set, which can promptly identify problems in the model training process.

[0065] 3. The present invention can accurately evaluate the performance of the model by calculating the error between the generated 3D model and the real 3D model, which helps researchers to clarify the performance of the model in the 3D reconstruction task and serve as a basis for model optimization and improvement. Analyzing the distribution of reconstruction errors can help researchers understand the performance of the model on data of different types or difficulty levels, help identify possible weaknesses and problems in the model, and make targeted improvements. Based on the evaluation results, the model's architecture, parameters and loss function can be adjusted to optimize the model's performance in a targeted manner. The model can gradually approach the optimal solution and improve its performance in 3D virtual image reconstruction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a schematic diagram of the process of the 3D virtual image model training method of the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] In order to solve the technical problems that existing methods cannot efficiently extract key feature information from 2D images, resulting in low quality of reconstructed 3D models, and traditional training methods may be inefficient and fail to fully utilize computing resources, resulting in a long training process and an inability to accurately measure the quality of the generated 3D models, please refer to Figure 1 , the present invention provides the following technical solutions:

[0069] The 3D virtual image model training method based on deep learning includes the following steps:

[0070] Data preparation, obtaining training data, the training data including 2D images and corresponding 3D models, wherein the 2D images include close-ups of the face, the whole body, and body parts, and the 3D models include corresponding detailed geometric shapes and textures, preprocessing the 2D images, adjusting the consistency of the input data, converting the 3D models into a unified format, and scaling and rotating the converted 3D models;

[0071] Model design: build a deep learning model that includes a convolutional neural network and an autoencoder. The convolutional neural network is used to extract feature information from 2D images, and the autoencoder is used to learn the encoding and decoding process of data and reconstruct a 3D model. The preprocessed 2D image is input into the deep learning model, and the key feature information of the face is extracted through the convolutional neural network. The key feature information is encoded into a low-dimensional vector representation. Based on the encoded feature vector, the low-dimensional vector is decoded into 3D model data;

[0072] Model training, constructing a loss function to guide the training of the deep learning model, wherein the loss function is used to measure the difference between the generated 3D model and the target 3D model, using the collected training data to train the deep learning model, optimizing the parameters of the model during the training process, and monitoring the performance of the model through the validation set, and saving the parameters of the model when the validation set performs well;

[0073] Model evaluation: After training, the model is evaluated using the test set to measure the model performance. Evaluation indicators include reconstruction error and user satisfaction. The model is adjusted based on the evaluation results. Model adjustments include model architecture adjustment, parameter adjustment, and loss function adjustment.

[0074] Application optimization is to put the trained 3D virtual image model into practical applications, including artistic creation, game design, and virtual fitting. During the actual application process, user feedback and data are collected to optimize and improve the model.

[0075] In the above embodiment, through a large amount of training data including 3D models with detailed geometric shapes and textures and corresponding 2D images, the model can learn the complex mapping relationship from 2D to 3D conversion, thereby generating a highly realistic and accurate 3D virtual image. Since the training data includes close-ups of the face, whole body and body parts, the model can be applied to a variety of scenarios, such as art creation, game design, virtual fitting, etc., and has a wide range of applicability. The model evaluation step can collect user feedback and data in real time, and adjust and optimize the model based on this information, such as model architecture adjustment, parameter adjustment and loss function adjustment, so that the model can continue to evolve and adapt to new needs and challenges.

[0076] In the above embodiment, by preprocessing and converting the 2D image and 3D model into a unified format, the consistency of the input data can be ensured and the efficiency and accuracy of the model training can be improved. At the same time, scaling and rotating the 3D model can further enhance the robustness of the model. The technical solution is based on deep learning and has strong scalability and flexibility. The model architecture, parameters and loss function can be adjusted according to actual needs to adapt to different application scenarios and data sets. Compared with the traditional method of manually creating a 3D model, the 3D virtual image model training method based on deep learning can automatically generate a 3D model from a 2D image, greatly reducing development costs and time.

[0077] Data preparation also includes the following steps:

[0078] Acquire 2D images containing different faces, whole bodies, and close-ups of body parts, and collect 3D models corresponding to the 2D images, wherein the 3D models include device scanned models, manually created models, or models generated by modeling software;

[0079] Cropping removes the background part of the 2D image so that the target area becomes the center of the image, and the target area includes the face and the whole body. The image is scaled to 256x256 pixels and normalized to map the pixel values ​​to the range of [0, 1] or [-1, 1].

[0080] Convert all 3D models into a unified file format, set the origin of each 3D model uniformly, and ensure that the coordinate systems of all 3D models are aligned;

[0081] Pair each 2D image with its corresponding 3D model and save the pairing information.

[0082] Divide the preprocessed data into training set, validation set and test set;

[0083] Among them, 80% of the data is used as a training set, which is used to train the deep learning model; 10% of the data is used as a validation set, which is used to verify the model performance and adjust the hyperparameters; and 10% of the data is used as a test set, which is used to finally test the model performance.

[0084] In the above embodiment, by cropping, scaling and normalizing the 2D image, the consistency of the input data can be ensured and the training effect of the model can be improved. Removing background interference and standardizing the size and pixel value range can help the model focus more on learning the characteristics of the target area. Converting the 3D model to a unified file format and unifying the origin setting to ensure the alignment of the coordinate system greatly reduces the confusion factors in the model training and helps the model learn more accurate and consistent 3D structure information during the training process.

[0085] In the above embodiment, accurately pairing a 2D image with its corresponding 3D model and saving the pairing information is the key to training the 3D virtual image model, which ensures that the model can establish an accurate 2D to 3D mapping relationship during the training process. The preprocessed data is divided into a training set, a validation set, and a test set, and the proportions are reasonably allocated, which helps the stability of model training, the adjustment of hyperparameters, and the evaluation of the final performance, and provides validation and test data to evaluate the performance of the model.

[0086] Specifically, the data preparation further includes:

[0087] Extracting a region shape of the target region before the scaling operation as first region shape data;

[0088] extracting a region shape of the target region after the scaling operation as second region shape data;

[0089] The shape change parameter is obtained by using the first region shape data and the second region shape data; wherein the shape change parameter is obtained by the following formula:

[0090]

[0091] Among them, R x represents the shape change parameter; R x0 Indicates the reference value of the preset shape change parameter; S y Represents the preset shape similarity threshold; S x represents the shape similarity value between the region shape of the target region before the scaling operation and the region shape of the target region after the scaling operation; x represents the first adjustment coefficient, and the first adjustment coefficient is obtained by the following formula:

[0092]

[0093] Wherein, x represents the first adjustment coefficient; m×m represents the pixel size of the target area before the scaling operation, corresponding to the target pixel size 256x256 corresponding to the scaled target area;

[0094] comparing the shape change parameter with a preset shape change parameter threshold;

[0095] When the shape change parameter exceeds a preset shape change parameter threshold, it is determined that the scaling quality of the target area is abnormal, and an abnormal alarm is issued;

[0096] When the shape change parameter does not exceed the preset shape change parameter threshold, a secondary abnormality determination is performed on the scaling quality of the target area using the gray value change parameters of the target area before the scaling operation and the target area after the scaling operation.

[0097] The technical effects of the above technical solution are: Extracting regional shape data: By extracting the target regional shape data (first regional shape data and second regional shape data) before and after the scaling operation, a basis is provided for subsequent shape change analysis. Using these shape data, the shape change parameter (Rx) is calculated by the above formula, which can accurately quantify the impact of the scaling operation on the target regional shape.

[0098] The preset shape change parameter threshold provides a standard for determining whether the scaling quality is abnormal. When the calculated shape change parameter exceeds the preset threshold, the system can automatically determine that the scaling quality of the target area is abnormal and issue an abnormal alarm. This helps to promptly discover and handle possible scaling problems, ensuring the accuracy and reliability of subsequent processing.

[0099] When the shape change parameter does not exceed the preset threshold, the gray value change parameter before and after the scaling operation is further used to perform a secondary abnormality judgment on the scaling quality of the target area. This increases the comprehensiveness and accuracy of the judgment. By combining the judgment results of the shape change parameter and the gray value change parameter, the scaling quality of the target area can be more comprehensively evaluated, reducing the risk of misjudgment and missed judgment.

[0100] The entire technical solution realizes automated processing, reduces manual intervention, and improves processing efficiency. By accurately quantifying shape changes and grayscale value changes, accurate judgment of scaling quality is achieved, improving processing accuracy. At the same time, this technical solution is not only suitable for specific types of image or data processing, but also has certain versatility and can be applied to a variety of scenarios that require evaluation of scaling quality.

[0101] In summary, this technical solution realizes the automatic and accurate judgment of the scaling quality of the target area by accurately quantifying the shape changes and grayscale value changes, and can promptly alarm when abnormalities are found, thereby improving processing efficiency and accuracy, and has broad application prospects.

[0102] Specifically, when the shape change parameter does not exceed a preset shape change parameter threshold, a secondary abnormality determination is performed on the scaling quality of the target area using the gray value change parameters of the target area before the scaling operation and the target area after the scaling operation, including:

[0103] Extracting the grayscale value of the pixel block contained in the target area before the scaling operation as the first area grayscale value data;

[0104] Extracting the grayscale value of the pixel block contained in the target area after the scaling operation as the second area grayscale value data;

[0105] Extracting grayscale values ​​of grayscale values ​​of 0 and 255 in the grayscale value data of the first region to form a first pixel value set;

[0106] Extracting grayscale values ​​of grayscale values ​​of 0 and 255 in the grayscale value data of the second region to form a second pixel value set;

[0107] The grayscale change parameter is obtained by using the first pixel value set and the second pixel value set, wherein the grayscale change parameter is obtained by the following formula:

[0108]

[0109] Among them, R h Represents the grayscale change parameter; R h0 Indicates the preset reference value of the grayscale change parameter; h 01max and h 01min represent the maximum grayscale value and the minimum grayscale value contained in the first pixel value set respectively; h 02max and h 02min They respectively represent the maximum grayscale value and the minimum grayscale value contained in the second pixel value set; r represents the second adjustment coefficient, and the second adjustment coefficient is obtained by the following formula:

[0110]

[0111] Wherein, r represents the second adjustment coefficient; a represents the number of gray value elements included in the first pixel value set; b represents the number of gray value elements included in the second pixel value set; h i represents the corresponding value of the i-th grayscale value in the first pixel value set; h i-1represents the corresponding value of the i-1th grayscale value in the first pixel value set; h p01 represents the grayscale average value corresponding to the grayscale value elements in the first pixel value set; j represents the corresponding value of the jth grayscale value in the second pixel value set; j-1 represents the corresponding value of the j-1th grayscale value in the second pixel value set; p02 Represents the grayscale average value corresponding to the grayscale value elements in the second pixel value set;

[0112] Comparing the grayscale change parameter with a preset grayscale change parameter threshold;

[0113] When the grayscale change parameter is lower than a preset grayscale change parameter threshold, it is determined that the scaling quality of the target area is abnormal, and an abnormality alarm is issued.

[0114] The technical effect of the above technical solution is: Extracting grayscale value data: by respectively extracting the grayscale value data of the target area before and after the zoom operation (the grayscale value data of the first area and the grayscale value data of the second area), a basis is provided for subsequent analysis.

[0115] Forming pixel value sets: Extracting pixel values ​​with grayscale values ​​of 0 and 255 (usually representing pure black and pure white) separately to form sets helps focus on changes in extreme grayscale values, which often have a significant impact on image quality.

[0116] Calculate the grayscale change parameter: Use a specific formula to combine the grayscale maximum value, minimum value, average value and the second adjustment coefficient in the two pixel value sets to calculate the grayscale change parameter (Rh). This parameter can accurately quantify the impact of the scaling operation on the grayscale distribution of the target area.

[0117] First, the shape change parameter is used to make a preliminary abnormality judgment to ensure that significant changes in shape can be discovered in time. If the shape change parameter does not exceed the preset threshold, the grayscale change parameter is further used to make a secondary abnormality judgment. This dual judgment mechanism improves the comprehensiveness and accuracy of abnormality detection.

[0118] The second adjustment coefficient (r) calculated by a complex formula takes into account multiple factors, including the number of gray value elements in the two pixel value sets, the relative change of gray values, and the gray average value, etc. This makes the calculation of gray change parameters more flexible and accurate, and can better reflect the actual impact of the scaling operation on the gray distribution.

[0119] The calculated grayscale change parameter is compared with the preset grayscale change parameter threshold to determine whether the scaling quality of the target area is abnormal. When the grayscale change parameter is lower than the preset threshold, the system can automatically determine that the scaling quality of the target area is abnormal and issue an abnormal alarm. This helps to promptly discover and handle possible scaling problems, ensuring the accuracy and reliability of subsequent processing.

[0120] By comprehensively considering both shape change and grayscale change, this technical solution can more comprehensively evaluate the scaling quality of the target area. Introducing this technical solution into the image processing process can help optimize the scaling operation, improve the quality of the processed image, and reduce image distortion or quality degradation caused by improper scaling operations.

[0121] In summary, this technical solution effectively improves the assessment accuracy and processing efficiency of the scaling quality of the target area by accurately quantifying grayscale changes, introducing a dual anomaly determination mechanism, using flexible adjustment coefficients, and realizing anomaly detection and alarm functions.

[0122] Model design also includes the following steps:

[0123] Determine the model architecture, extract key feature information from the 2D image as a feature vector, compress the feature information into a low-dimensional vector representation, receive the extracted feature vector as input, encode the input data into a low-dimensional vector representation, and output the encoded low-dimensional vector;

[0124] The low-dimensional vector output by the encoder is received as input, 3D model data is decoded from the code, and the 3D model data is output.

[0125] In the above embodiments, through convolutional neural networks or other deep learning architectures, the model can efficiently extract key feature information from 2D images, which is essential for reconstructing high-quality 3D models. Compressing the extracted feature information into a low-dimensional vector representation can greatly reduce the dimension of the data while retaining key information, which not only reduces the computational cost but also helps the model learn more efficient representations. Using an autoencoder or its variants as part of the model enables the model to decode 3D model data from low-dimensional vectors, learn the mapping relationship from 2D to 3D during the training process, and optimize the quality of the generated 3D model by adjusting the parameters of the encoder and decoder.

[0126] Model training also includes the following steps:

[0127] According to the specific task requirements, add loss functions, load training data through the data loader, including the preprocessed 2D images and corresponding target 3D models in the training set, and divide the data into multiple batches for training, each batch contains a certain number of samples;

[0128] Initialize the deep learning model based on the deep learning framework. For each training batch, input the 2D image into the model, calculate the generated 3D model through forward propagation, and use the loss function to calculate the difference between the generated 3D model and the target 3D model to obtain the loss value.

[0129] Backpropagation is performed based on the loss value to calculate the gradient and update the model parameters.

[0130] Load the validation set data through the data loader. After each training cycle, use the validation set to evaluate the model. Input the validation set data into the model, calculate the difference between the generated 3D model and the target 3D model, obtain the validation loss, and monitor the change of the validation loss.

[0131] Set the trigger conditions for saving the model, which include the verification loss decreasing for a specific number of consecutive cycles and reaching a preset verification accuracy. When the saving conditions are met, the parameters of the current model are saved to disk;

[0132] Record the training information during the training process, including loss value, accuracy rate, and learning rate, use visualization tools to visualize the training process, and generate data curves based on the training information.

[0133] In the above embodiment, by dividing the data into multiple batches for training, the model can more efficiently utilize computing resources and accelerate the training process. In addition, batch training also helps to reduce noise in gradient calculations and improve the stability of model training. Adding a loss function that matches the specific task requirements can accurately measure the difference between the generated 3D model and the target 3D model, thereby guiding the model to perform precise parameter optimization. The choice of loss function is crucial to the performance of the model, and reasonable loss calculation can ensure that the model is continuously optimized during training.

[0134] In the above embodiment, the validation set is used to evaluate the model after each training cycle, so that problems in the model training process, such as overfitting, underfitting, etc., can be discovered in a timely manner. At the same time, by monitoring the changes in the validation loss, it can be determined whether the model is gradually improving. When the saving conditions are met, the parameters of the current model are saved to disk, which helps to continue to optimize or deploy the model in subsequent experiments.

[0135] In the above embodiment, recording the training information during the training process, such as loss value, accuracy rate, learning rate, etc., and using visualization tools to generate data curves helps to intuitively understand the process and effect of model training. This not only improves the interpretability of the experiment, but also helps researchers adjust the training strategy or optimize the model architecture based on the training information.

[0136] Model evaluation also includes the following steps:

[0137] Load the trained deep learning model and input the preprocessed test set 2D images into the model. The model extracts features through a convolutional neural network, encodes the features into low-dimensional vectors using an autoencoder, and decodes them into 3D model data.

[0138] Calculate the error between the generated 3D model and the real 3D model, analyze the distribution of the reconstruction error, obtain the evaluation results, and adjust the model architecture, parameters and loss function based on the evaluation results;

[0139] The process of model evaluation, analysis, and tuning is repeated until the model performance reaches the desired level.

[0140] In the above embodiment, by calculating the error between the generated 3D model and the real 3D model, the performance of the model can be accurately evaluated, which helps researchers to clarify the performance of the model in the 3D reconstruction task and serves as a basis for model optimization and improvement. Analyzing the distribution of reconstruction errors can help researchers understand the performance of the model on data of different types or difficulty, help identify possible weaknesses and problems in the model, and make targeted improvements. Based on the evaluation results, the architecture, parameters, and loss function of the model can be adjusted to optimize the performance of the model in a targeted manner. The model can gradually approach the optimal solution and improve its performance in the 3D virtual image reconstruction task.

[0141] In the above embodiment, by evaluating the model through the test set, the performance of the model on unseen data, that is, the generalization ability of the model, can be verified, which is crucial to ensure the stability and reliability of the model in practical applications. Model evaluation is an iterative process. By continuously evaluating, analyzing and adjusting the model, researchers can find problems and find solutions more quickly, which helps to accelerate the model development process and shorten the project cycle. Through clear evaluation steps and quantitative evaluation indicators, the reliability and repeatability of the experimental results can be ensured, which helps other researchers understand and verify the experimental results and promote academic exchanges and cooperation.

[0142] Application optimization also includes the following steps:

[0143] Integrate the trained 3D avatar model into the corresponding application platform and software, provide users with clear usage guides or tutorials, and set up a feedback mechanism, which includes a rating system, questionnaires, and online chats. During the user's use, collect users' evaluations of model performance and application experience in real time;

[0144] Based on user feedback and data analysis results, the performance and functionality of the model are improved and expanded, the optimized and improved model is redeployed to the application platform, and user usage and feedback are continuously monitored after the application is updated.

[0145] In the above embodiments, providing users with clear usage guides or tutorials helps users quickly become familiar with and master the use of the application, thereby enhancing the user experience. Setting a feedback mechanism can collect users' evaluations of model performance and application experience in real time, which helps to promptly discover and solve potential problems and further improve the user experience. Based on user feedback and data analysis results, the performance and functions of the model can be improved and expanded in a targeted manner. This continuous optimization process enables the model to better meet user needs and improve the practicality and competitiveness of the model.

[0146] In the above embodiment, by collecting and analyzing user feedback, we can understand the performance of the model in different user groups and scenarios, which helps to identify the deficiencies of the model and make improvements, thereby improving the adaptability and generalization ability of the model. The setting of the feedback mechanism not only helps to collect user evaluations, but also promotes the interaction between users and developers. Users can make valuable suggestions through ratings, questionnaires or online chats, and developers can respond and improve the model in a timely manner, forming a virtuous interactive cycle.

[0147] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A 3D virtual image model training method based on deep learning, characterized in that: The following steps are involved: Data preparation, obtaining training data, the training data including 2D images and corresponding 3D models, wherein the 2D images include close-ups of the face, the whole body, and body parts, and the 3D models include corresponding detailed geometric shapes and textures, preprocessing the 2D images, adjusting the consistency of the input data, converting the 3D models into a unified format, and scaling and rotating the converted 3D models; Model design: build a deep learning model that includes a convolutional neural network and an autoencoder. The convolutional neural network is used to extract feature information from 2D images, and the autoencoder is used to learn the encoding and decoding process of data and reconstruct a 3D model. The preprocessed 2D image is input into the deep learning model, and the key feature information of the face is extracted through the convolutional neural network. The key feature information is encoded into a low-dimensional vector representation. Based on the encoded feature vector, the low-dimensional vector is decoded into 3D model data; Model training, constructing a loss function to guide the training of the deep learning model, wherein the loss function is used to measure the difference between the generated 3D model and the target 3D model, using the collected training data to train the deep learning model, optimizing the parameters of the model during the training process, and monitoring the performance of the model through the validation set, and saving the parameters of the model when the validation set performs well; Model evaluation: After training, the model is evaluated using the test set to measure the model performance. Evaluation indicators include reconstruction error and user satisfaction. The model is adjusted based on the evaluation results. Model adjustments include model architecture adjustment, parameter adjustment, and loss function adjustment. Application optimization: putting the trained 3D virtual image model into practical applications, such as art creation, game design, and virtual fitting. During the actual application process, user feedback and data are collected to optimize and improve the model. The data preparation further includes the following steps: Acquire 2D images containing different faces, whole bodies, and close-ups of body parts, and collect 3D models corresponding to the 2D images, wherein the 3D models include device scanned models, manually created models, or models generated by modeling software; Cropping removes the background part of the 2D image so that the target area becomes the center of the image, and the target area includes the face and the whole body. The image is scaled to 256x256 pixels and normalized to map the pixel values ​​to the range of [0, 1] or [-1, 1]. Convert all 3D models into a unified file format, set the origin of each 3D model uniformly, and ensure that the coordinate systems of all 3D models are aligned; Pair each 2D image with its corresponding 3D model and save the pairing information.

2. The 3D virtual image model training method based on deep learning according to claim 1, characterized in that: The data preparation further includes: Extracting a region shape of the target region before the scaling operation as first region shape data; extracting a region shape of the target region after the scaling operation as second region shape data; The shape change parameter is obtained by using the first region shape data and the second region shape data; wherein the shape change parameter is obtained by the following formula: Among them, R x represents the shape change parameter; R x0 Indicates the reference value of the preset shape change parameter; S y Represents the preset shape similarity threshold; S x represents the shape similarity value between the region shape of the target region before the scaling operation and the region shape of the target region after the scaling operation; x represents the first adjustment coefficient, and the first adjustment coefficient is obtained by the following formula: Wherein, x represents the first adjustment coefficient; m×m represents the pixel size of the target area before the scaling operation, corresponding to the target pixel size 256x256 corresponding to the scaled target area; comparing the shape change parameter with a preset shape change parameter threshold; When the shape change parameter exceeds a preset shape change parameter threshold, it is determined that the scaling quality of the target area is abnormal, and an abnormal alarm is issued; When the shape change parameter does not exceed the preset shape change parameter threshold, a secondary abnormality determination is performed on the scaling quality of the target area using the gray value change parameters of the target area before the scaling operation and the target area after the scaling operation.

3. The 3D virtual image model training method based on deep learning according to claim 2, characterized in that: When the shape change parameter does not exceed the preset shape change parameter threshold, a secondary abnormality determination is performed on the scaling quality of the target area using the gray value change parameters of the target area before the scaling operation and the target area after the scaling operation, including: Extracting the grayscale value of the pixel block contained in the target area before the scaling operation as the first area grayscale value data; Extracting the grayscale value of the pixel block contained in the target area after the scaling operation as the second area grayscale value data; Extracting grayscale values ​​of grayscale values ​​of 0 and 255 in the grayscale value data of the first region to form a first pixel value set; Extracting grayscale values ​​of grayscale values ​​of 0 and 255 in the grayscale value data of the second region to form a second pixel value set; The grayscale change parameter is obtained by using the first pixel value set and the second pixel value set, wherein the grayscale change parameter is obtained by the following formula: Among them, R h Represents the grayscale change parameter; R h0 Indicates the preset reference value of the grayscale change parameter; h 01max and h 01min represent the maximum grayscale value and the minimum grayscale value contained in the first pixel value set respectively; h 02max and h 02min They respectively represent the maximum grayscale value and the minimum grayscale value contained in the second pixel value set; r represents the second adjustment coefficient, and the second adjustment coefficient is obtained by the following formula: Wherein, r represents the second adjustment coefficient; a represents the number of gray value elements included in the first pixel value set; b represents the number of gray value elements included in the second pixel value set; h i represents the corresponding value of the i-th grayscale value in the first pixel value set; h i-1 represents the corresponding value of the i-1th grayscale value in the first pixel value set; h p01 represents the grayscale average value corresponding to the grayscale value elements in the first pixel value set; j represents the corresponding value of the jth grayscale value in the second pixel value set; j-1 represents the corresponding value of the j-1th grayscale value in the second pixel value set; p02 Represents the grayscale average value corresponding to the grayscale value elements in the second pixel value set; Comparing the grayscale change parameter with a preset grayscale change parameter threshold; When the grayscale change parameter is lower than a preset grayscale change parameter threshold, it is determined that the scaling quality of the target area is abnormal, and an abnormality alarm is issued.

4. The 3D virtual image model training method based on deep learning according to claim 1, characterized in that: The data preparation specifically further includes the following steps: Divide the preprocessed data into training set, validation set and test set; Among them, 80% of the data is used as a training set, which is used to train the deep learning model; 10% of the data is used as a validation set, which is used to verify the model performance and adjust the hyperparameters; and 10% of the data is used as a test set, which is used to finally test the model performance.

5. The 3D virtual image model training method based on deep learning according to claim 1, characterized in that: The model design specifically includes the following steps: Determine the model architecture, extract key feature information from the 2D image as a feature vector, compress the feature information into a low-dimensional vector representation, receive the extracted feature vector as input, encode the input data into a low-dimensional vector representation, and output the encoded low-dimensional vector; The low-dimensional vector output by the encoder is received as input, 3D model data is decoded from the code, and the 3D model data is output.

6. The 3D virtual image model training method based on deep learning according to claim 1, characterized in that: The model training specifically further includes the following steps: According to the specific task requirements, add loss functions, load training data through the data loader, including the preprocessed 2D images and corresponding target 3D models in the training set, and divide the data into multiple batches for training, each batch contains a certain number of samples; Initialize the deep learning model based on the deep learning framework. For each training batch, input the 2D image into the model, calculate the generated 3D model through forward propagation, and use the loss function to calculate the difference between the generated 3D model and the target 3D model to obtain the loss value. Backpropagation is performed based on the loss value to calculate the gradient and update the model parameters.

7. The deep learning-based 3D virtual image model training method according to claim 6, characterized in that: The model training specifically further includes the following steps: Load the validation set data through the data loader. After each training cycle, use the validation set to evaluate the model. Input the validation set data into the model, calculate the difference between the generated 3D model and the target 3D model, obtain the validation loss, and monitor the change of the validation loss. Set the trigger conditions for saving the model, which include the verification loss decreasing for a specific number of consecutive cycles and reaching a preset verification accuracy. When the saving conditions are met, the parameters of the current model are saved to disk; Record the training information during the training process, including loss value, accuracy rate, and learning rate, use visualization tools to visualize the training process, and generate data curves based on the training information.

8. The 3D virtual image model training method based on deep learning according to claim 1, characterized in that: The model evaluation specifically includes the following steps: Load the trained deep learning model and input the preprocessed test set 2D images into the model. The model extracts features through a convolutional neural network, encodes the features into low-dimensional vectors using an autoencoder, and decodes them into 3D model data. Calculate the error between the generated 3D model and the real 3D model, analyze the distribution of the reconstruction error, obtain the evaluation results, and adjust the model architecture, parameters and loss function based on the evaluation results; The process of model evaluation, analysis, and tuning is repeated until the model performance reaches the desired level.

9. The 3D virtual image model training method based on deep learning according to claim 1, characterized in that: The application optimization specifically further includes the following steps: Integrate the trained 3D avatar model into the corresponding application platform and software, provide users with clear usage guides or tutorials, and set up a feedback mechanism, which includes a rating system, questionnaires, and online chats. During the user's use, collect users' evaluations of model performance and application experience in real time; Based on user feedback and data analysis results, the performance and functionality of the model are improved and expanded, the optimized and improved model is redeployed to the application platform, and user usage and feedback are continuously monitored after the application is updated.

Images