Foot scanning method based on differential rendering
By using differentiable rendering method, A4 paper is used to scan foot, establish a coordinate system and iteratively optimize the conversion parameters, the problem of cumbersome and error-free printing of pixel paper in the prior art is solved, and the accurate measurement of foot size is achieved.
Patent Information
- Application Number
- CN202311568647.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-21
- Publication Date
- 2025-07-25
AI Technical Summary
Although the existing foot scanning measurement methods are highly accurate, the process of printing pixel paper is cumbersome and easy to cause errors, causing inconvenience to users.
Using a differentiable rendering method, A4 paper is used to perform foot positioning, coordinate system is established through SfM sparse reconstruction and dense reconstruction, conversion parameters are calculated, and loss function is iteratively optimized to obtain accurate coordinate system conversion to achieve accurate measurement of foot size.
The measurement process is simplified, the steps of printing pixel paper are avoided, the accuracy and efficiency of measurement are improved, and the accurate measurement of foot data is achieved.
Smart Images
Figure CN120374825A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of three-dimensional reconstruction, and specifically to a foot scanning method based on differentiable rendering. Background Art
[0002] With the change of people's lifestyle, there are more and more ways for people to buy shoes, such as buying in physical stores, buying online, customizing, etc. However, no matter which way is adopted, accurate measurement of foot size is required, especially for custom-made shoes and users with high requirements for the accuracy of shoe sizes. During the process of measuring foot size, it is necessary to use foot scanning technology to accurately measure the foot size parameters. The previous positioning technology used in foot scanning requires users to print a pixel paper, which is used to determine the position of the camera in the video taken by the user, and can also be used as a reference for measuring the size of the foot, so as to measure the parameters of the foot.
[0003] However, although the current measurement method has relatively high accuracy, the process of printing the pixel paper itself is a process that brings inconvenience to users, making the entire measurement process very troublesome and prone to errors. Summary of the Invention
[0004] The present invention provides a foot scanning method based on differentiable rendering, which can effectively solve the problem that although the current measurement method has relatively high accuracy, the process of printing the pixel paper itself is a process that brings inconvenience to users, making the entire measurement process very troublesome and prone to errors as mentioned in the above background art.
[0005] To achieve the above object, the present invention provides the following technical solution: A foot scanning method based on differentiable rendering uses an A4 paper to accurately position the scanned foot and analyzes the foot scanning video by using the method of differentiable rendering.
[0006] Specifically, it includes the following steps:
[0007] S1. Scan the foot to obtain the data required for modeling and positioning.
[0008] S2. Use an image mask to show the position of the A4 paper in the image.
[0009] S3. Perform SfM sparse reconstruction on the image to obtain the sparse reconstruction coordinate system of the image.
[0010] S4. Establish a dense reconstruction model of the A4 paper and establish a real coordinate system.
[0011] S5. According to the conversion relationship, transfer the real coordinate system to the reconstruction coordinate system, and then re-project it onto the A4 mask image according to the internal and external parameters of the reconstruction coordinate system, so as to calculate the loss function.
[0012] S6. Find the four vertices of the A4 paper according to the original A4 paper mask, denoted as \(L_i\), \(i = 1, 2, 3, 4\). Denote the four vertices of the projection reprojected into the image mask as \(L_i'\), \(i = 1, 2, 3, 4\), and set the mean square error as the loss function. i , \(i = 1, 2, 3, 4\), and set the mean square error as the loss function. i , \(i = 1, 2, 3, 4\), and set the mean square error as the loss function.
[0013] S7. Feed the calculation result of the loss function back to the parameter calculation in step 4, optimize the transformation parameters based on the loss value, use the optimized transformation parameters to transform the A4 paper in the real coordinate system into the sparse reconstruction coordinate system again, and then map it onto the original A4 image mask. Recalculate the loss value according to the method in step 6, and repeat the above steps until the loss value is less than the threshold.
[0014] S8. Through the iteration in step S7, an accurate transformation parameter \(s\), \(R\), \(t\) from the real coordinate system to the sparse reconstruction coordinate system can be obtained. Obtain the transformation parameter \(s'\), \(R'\), \(t'\) from the sparse reconstruction coordinate system to the real coordinate system through these parameters.
[0015] S9. Transform the model in the sparse reconstruction to the real coordinate system through the transformation parameter \(s'\), \(R'\), \(t'\) to obtain the size of the foot in the real coordinate system, and measure and locate the foot parameters accordingly.
[0016] According to the above technical solution, in S1, mainly use a piece of A4 white paper for calibration to determine the position of the camera and use it to locate the reconstructed foot model.
[0017] Specifically, place a piece of A4 white paper in front of the foot as a reference for calculating foot data, and take a video around the foot, ensuring that the foot and the A4 paper always appear in the picture at the same time.
[0018] According to the above technical solution, in S4, during the process of establishing the dense reconstruction model of the A4 paper, establish a real coordinate system with the center of A4 as the origin, the short side direction as the x-axis, the long side direction as the y-axis, and the vertical direction as the z-axis.
[0019] Let the transformation relationship from the real coordinate system to the sparse reconstruction coordinate system be \(s\), \(R\), \(t\).
[0020] S: Scaling factor, \(R\): Rotation matrix, \(t\): Translation vector, as follows:
[0021] \(R = R_xR_yR_z\),
[0022]
[0023]
[0024]
[0025] According to the above technical solution, in S5, mainly the obtained A4 paper dense reconstruction model is reprojected onto the mask image according to the internal and external parameters of SfM.
[0026] According to the above technical solution, in S6, the loss function is as follows:
[0027]
[0028] The larger the mean square error, the greater the difference between the reprojected image and the original image, and further optimization is required.
[0029] According to the above technical solution, in S6, the chamfer distance can also be used as the loss function. Calculate the average value of the minimum distance from a point in the original image to the reprojected image, then there is:
[0030]
[0031] Where, T is the original image, I is the reprojected image, t is the point in the original image, d(t) is the minimum distance from the point in the original image to the reprojected image. The larger the distance, the greater the difference between the reprojected image and the original image, indicating that further optimization is needed.
[0032] According to the above technical solution, in S8, s, R, t are converted into the conversion parameters s‘, R‘, t‘ from the sparse reconstruction coordinate system to the real coordinate system through the inverse relationship;
[0033] The conversion parameters s‘, R‘, t‘ are as follows:
[0034] R‘ = R x ‘R y ‘R z ‘,
[0035]
[0036]
[0037]
[0038] According to the above technical solution, in steps S4, S5, S6 and S7, the conversion parameters s, R, t between the real coordinate system and the sparse reconstruction coordinate system are obtained by means of differentiable rendering.
[0039] Compared with the prior art, the beneficial effects of the present invention:
[0040] 1. The present invention uses the method of differentiable rendering to locate and measure the feet in a video. By performing SfM sparse reconstruction on the video and regarding the A4 paper in the real coordinate system as a dense reconstruction model, a set of transformation parameters s, R, t from the real coordinate system to the sparse reconstruction coordinates are assumed. Then, the dense reconstruction model is projected onto the SfM sparse reconstruction coordinate system according to the transformation parameters and re-projected onto the image from the sparse reconstruction coordinate system. Next, the loss function is calculated using the A4 image mask, and the transformation parameters s, R, t are optimized according to the feedback of the loss function until the requirements are met, so as to obtain the optimal s, R, t. Then, the feet in the sparse reconstruction coordinate system are transformed into the real coordinate system, and the corresponding relationship between the foot model and the real foot is calculated, which is convenient for more labor-saving calculation of foot parameters and realizes accurate measurement of the real data of the feet.
[0041] 2. The present invention uses an A4 white paper to replace the process of printing pixel paper, realizes accurate positioning of the scanned feet, analyzes and sparsely reconstructs the foot scan video by using the method of differentiable rendering, simplifies the existing measurement process, and calculates the error after mapping the A4 paper in the real coordinate system to the A4 paper image in the video. The transformation parameters are continuously iteratively modified according to the error, and in this way, the transformation parameters from the real coordinate system to the video coordinate system are obtained, so as to know the corresponding relationship between the feet in the video and the feet in reality. Then, the size parameters of the feet in reality can be calculated according to the modeling of the feet in the video, and accurate measurement of the real data of the feet is realized. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The drawings are used to provide further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention.
[0043] In the drawings:
[0044] Figure 1 is a schematic flow chart of the steps of the scanning method of the present invention;
[0045] Figure 2 is a schematic diagram of the feet and the A4 paper of the present invention;
[0046] Figure 3 is a schematic diagram of the image mask of the present invention;
[0047] Figure 4 is a schematic diagram of the transformation from the real coordinate system to the sparse reconstruction coordinate system of the present invention;
[0048] Figure 5 is a schematic diagram of the transformation from the sparse reconstruction coordinate system to the real coordinate system of the present invention;
[0049] Figure 6It is a schematic diagram of the projection in the preliminary stage of the sparse reconstruction of the present invention towards the original mask;
[0050] Figure 7 It is a schematic diagram of the projection in the intermediate stage of the sparse reconstruction of the present invention towards the original mask;
[0051] Figure 8 It is a schematic diagram of the projection in the later stage of the sparse reconstruction of the present invention towards the original mask. Specific Embodiments
[0052] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only for illustrating and explaining the present invention, and are not used to limit the present invention.
[0053] Embodiment: The present invention provides a technical solution, a foot scanning method based on differentiable rendering, which uses an A4 paper to accurately position the scanned foot and analyzes the foot scanning video by using the method of differentiable rendering;
[0054] As Figure 1 shown, it specifically includes the following steps:
[0055] S1. Scan the foot to obtain the data for modeling and positioning;
[0056] As Figure 2 shown, in S1, mainly use an A4 white paper as a calibration to determine the position of the camera and use it to position the reconstructed foot model;
[0057] Specifically, place an A4 white paper in front of the foot as a reference for calculating foot data, and shoot a video around the foot, ensuring that the foot and the A4 paper always appear in the picture at the same time.
[0058] As Figure 3 shown, S2. Use an image mask to show the position of the A4 paper in the image;
[0059] S3. Perform SfM sparse reconstruction on the image to obtain the sparse reconstruction coordinate system of the image;
[0060] S4. Establish a dense reconstruction model of the A4 paper and establish a real coordinate system;
[0061] As Figure 4 shown, in S4, during the process of establishing the dense reconstruction model of the A4 paper, establish a real coordinate system with the center of the A4 as the origin, the short side direction as the x-axis, the long side direction as the y-axis, and the vertical direction as the z-axis;
[0062] Let the conversion relationship from the real coordinate system to the sparse reconstruction coordinate system be s, R, t;
[0063] S: Scaling factor, R: Rotation matrix, t: Translation vector, as follows:
[0064] R = RxRyRz,
[0065]
[0066]
[0067]
[0068] S5. According to the conversion relationship, transfer the real coordinate system to the reconstructed coordinate system, and then re-project it onto the A4 mask image according to the internal and external parameters of the reconstructed coordinate system, so as to calculate the loss function;
[0069] As Figures 6 - 8 shown, in S5, mainly re-project the obtained dense reconstruction model of the A4 paper onto the mask image according to the internal and external parameters of SfM.
[0070] S6. Find the four vertices of the A4 paper according to the original A4 paper mask, denoted as L i , i = 1, 2, 3, 4, and denote the four vertices of the projection re-projected onto the image mask as L' i , i = 1, 2, 3, 4, and set the mean square error as the loss function;
[0071] In S6, the loss function is as follows:
[0072]
[0073] The larger the mean square error, the greater the difference between the re-projected image and the original image, and further optimization is required;
[0074] In S6, the chamfer distance can also be used as the loss function, and calculate the average value of the minimum distances from a point in the original image to the re-projected image, then there is:
[0075]
[0076] where T is the original image, I is the re-projected image, t is the point in the original image, d(t) is the minimum distance from the point in the original image to the re-projected image, and the larger the distance, the greater the difference between the re-projected image and the original image, indicating that further optimization is needed.
[0077] S7. Feed the calculation result of the loss function back to the parameter calculation in step 4, optimize the transformation parameters based on the loss value, use the optimized transformation parameters to convert the A4 paper in the real coordinate system to the sparse reconstruction coordinate system again, then map it to the original A4 image mask, and recalculate the loss value according to the method in step 6. Repeat the above steps until the loss value is less than the threshold;
[0078] S8. Through the iteration in step S7, an accurate transformation parameter s, R, t from the real coordinate system to the sparse reconstruction coordinate system can be obtained. With these parameters, the transformation parameter s‘, R‘, t‘ from the sparse reconstruction coordinate system to the real coordinate system is obtained;
[0079] In S8, the s, R, t are converted into the transformation parameter s‘, R‘, t‘ from the sparse reconstruction coordinate system to the real coordinate system through the inverse relationship;
[0080] The transformation parameter s‘, R‘, t‘ is as follows:
[0081] R‘ = R x ‘R y ‘R z ‘,
[0082]
[0083]
[0084]
[0085] As Figure 5 shown, S9. Convert the model in the sparse reconstruction to the real coordinate system through the transformation parameter s‘, R′, t‘, obtain the size of the foot in the real coordinate system, and measure and locate the foot parameters accordingly.
[0086] Based on the above technical solutions, in steps S4, S5, S6 and S7, the transformation parameter s, R, t between the real coordinate system and the sparse reconstruction coordinate system is obtained by means of differentiable rendering.
[0087] Finally, it should be noted that the above are only the preferred examples of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A foot scanning method based on differentiable rendering, characterized in that: Use an A4 paper to accurately position the scanned foot, and analyze the foot scan video using the method of differentiable rendering; Specifically, it includes the following steps: S1. Scan the foot to obtain the data for modeling and positioning; S2. Use an image mask to show the position of the A4 paper in the image; S3. Perform SfM sparse reconstruction on the image to obtain the sparse reconstruction coordinate system of the image; S4. Establish a dense reconstruction model of the A4 paper and establish a real coordinate system; S5. According to the conversion relationship, transfer the real coordinate system to the reconstruction coordinate system, and then re-project it onto the A4 mask image according to the internal and external parameters of the reconstruction coordinate system, so as to calculate the loss function; S6. Find the four vertices of the A4 paper according to the original A4 paper mask, denoted as L i , where i = 1, 2, 3, 4, and denote the four vertices of the projection reprojected into the image mask as L' i , where i = 1, 2, 3, 4, and set the mean square error as the loss function. S7. Feed the calculation result of the loss function back to the parameter calculation in step 4, optimize the conversion parameters based on the loss value, use the optimized conversion parameters to re-convert the A4 paper in the real coordinate system to the sparse reconstruction coordinate system, and then map it onto the original A4 image mask, and re-calculate the loss value according to the method in step 6. Repeat the above steps until the loss value is less than the threshold; S8. Through the iteration in step S7, a set of accurate conversion parameters s, R, t from the real coordinate system to the sparse reconstruction coordinate system can be obtained. Through these parameters, the conversion parameters s', R', t' from the sparse reconstruction coordinate system to the real coordinate system are obtained; S9. Use the conversion parameters s', R', t' to convert the model in the sparse reconstruction to the real coordinate system, obtain the size of the foot in the real coordinate system, and measure and position the foot parameters accordingly.
2. The foot scanning method based on differentiable rendering according to claim 1, characterized in that: In S1, mainly use an A4 white paper as a calibration to determine the position of the camera and use it to position the reconstructed foot model; Specifically, place an A4 white paper in front of the foot as a reference for calculating foot data, and shoot a video around the foot, ensuring that the foot and the A4 paper always appear in the picture at the same time.
3. A foot scanning method based on differentiable rendering according to claim 1, characterized in that: In S4, during the process of establishing the dense reconstruction model of the A4 paper, establish a real coordinate system with the center of the A4 as the origin, the short side direction as the x-axis, the long side direction as the y-axis, and the vertical direction as the z-axis; Let the conversion relationship from the real coordinate system to the sparse reconstruction coordinate system be s, R, t; S: Scaling factor, R: Rotation matrix, t: Translation vector, as follows: R = RxRyRz, 4. A foot scanning method based on differentiable rendering according to claim 1, characterized in that: In S5, mainly re-project the obtained dense reconstruction model of the A4 paper onto the mask image according to the internal and external parameters of SfM.
5. A foot scanning method based on differentiable rendering according to claim 1, characterized in that: In S6, the loss function is as follows: The larger the mean square error, the greater the difference between the re-projected image and the original image, and further optimization is required.
6. A foot scanning method based on differentiable rendering according to claim 5, characterized in that: In S6, the chamfer distance can also be used as the loss function to calculate the average value of the minimum distance from a point in the original image to the re-projected image, then there is: where T is the original image, I is the re-projected image, t is the point in the original image, and d(t) is the minimum distance from the point in the original image to the re-projected image. The larger the distance, the greater the difference between the re-projected image and the original image, indicating that further optimization is required.
7. A foot scanning method based on differentiable rendering according to claim 1, characterized in that: In S8, convert s, R, t to the conversion parameters s', R', t' from the sparse reconstruction coordinate system to the real coordinate system through the inverse relationship; The conversion parameters s', R', t' are as follows: R‘ = R x ‘R y ‘R z ‘, 8. A foot scanning method based on differentiable rendering according to claim 1, characterized in that: In the steps S4, S5, S6, and S7, the conversion parameters s, R, and t between the real coordinate system and the sparse reconstruction coordinate system are obtained by means of differentiable rendering.