Foot scanning method based on line segment detection and matching

Abstract

The invention discloses a foot scanning method based on line segment detection and matching, which comprises the following steps: step 1, preparation work: placing a piece of A4 paper in front of a foot as a reference for calculating foot data, shooting a video around the foot, and acquiring data needing modeling and positioning, step 2, representing the position of the A4 paper in an image by using an image mask, according to the method, the A4 paper is placed in front of the foot, the position and form of the A4 paper in the space are better represented through line segment modeling, the center point of the A4 paper is obtained through calculation, the transition coordinate system is established, the coordinate system of the line segment model is converted into the transition coordinate system, and the line segment model is obtained. The scaling parameter s from the line segment model coordinate system to the real coordinate system is calculated according to the sizes of the A4 paper in the two different coordinate systems, the real foot parameters are obtained, the process is simpler, and the foot morphological structure can be better obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and specifically to a foot scanning method based on line segment detection and matching. Background Technique

[0002] Three-dimensional reconstruction refers to establishing a mathematical model suitable for computer representation and processing of three-dimensional objects, which is the basis for processing, operating, and analyzing its properties in a computer environment, and is also the key technology for establishing a virtual reality expressing the objective world in a computer. It is relatively common to construct three-dimensional reconstruction technology for human body parts, such as in the fields of clothing design, furniture design, medical treatment, etc. When performing three-dimensional reconstruction of the foot, by printing a pixel paper, the pixel paper is used to determine the position of the camera in the video taken by the user, and at the same time, it is used as a reference for measuring the size of the foot, so as to measure the parameters of the foot. In terms of estimating and modeling the scene, the point-based method is mostly used, that is, extracting key points in the image and then triangulating. This method has a relatively high reduction degree for scenes with more key points;

[0003] However, currently, foot scanning requires printing a pixel paper in advance, the process is cumbersome, and the method of extracting key points cannot reflect the original line structure of the foot with strong structure. Therefore, a foot scanning method based on line segment detection and matching is needed to streamline the process and better represent the morphological structure of the foot. Summary of the Invention

[0004] The present invention provides a foot scanning method based on line segment detection and matching, which can effectively solve the problems raised in the above background technique that currently foot scanning requires printing a pixel paper in advance, the process is cumbersome, and the method of extracting key points cannot reflect the original line structure of the foot with strong structure.

[0005] To achieve the above object, the present invention provides the following technical solution: A foot scanning method based on line segment detection and matching, including the following steps:

[0006] Step 1, Preparation: Place an A4 paper in front of the foot as a reference for calculating foot data, and shoot a video around the foot to obtain the data required for modeling and positioning;

[0007] Step 2, Use an image mask to show the position of the A4 paper in the image;

[0008] Step 3, Use the video frames taken to perform line segment map reconstruction to obtain a line segment model;

[0009] Step 4, Use the A4 paper mask of the image and the characteristics of the A4 paper to extract four line segments forming the A4 paper from the line segment model;

[0010] Step 5: Extract the four corner points of the A4 paper from the line segments, calculate the coordinates of the center of the A4 paper based on the coordinates of the four corner points, and establish a transition coordinate system with the center of the A4 paper as the origin;

[0011] Step 6: Calculate the transformation parameters s, R, and t from the line segment model coordinate system to the real coordinate system through the correspondence between the A4 paper in the line segment model coordinate system and the A4 paper in the real coordinate system;

[0012] Step 7: On the basis of knowing the transformation parameters s, R, and t from the line segment model coordinate system to the real coordinate system, convert the line segment model of the foot in the line segment model coordinate system to the real coordinate system, and then calculate the foot parameters including the length and width of the foot, and the parameters of the real foot can be obtained.

[0013] According to the above technical solution, in step 4, the line segments in the line segment model are reprojected onto the image, and the distance of each line segment relative to the A4 paper mask is calculated;

[0014] Let the four vertices of the mask quadrilateral be a, b, c, and d respectively, and calculate the distance of the line segment x (the coordinates of the two endpoints are (ex1, ey1) and (ex2, ey2)) to the A4 paper:

[0015] D = min{d i (i = 1, 2, 3, 4)} (1)

[0016] d i is the distance from the line segment to the four sides of the A4 paper. Given the straight line equation Ax + By + C = 0, then:

[0017]

[0018] Let the vertices of the A4 paper side be (x1, y1) and (x2, y2), then:

[0019] A = y1 - y2; B = x2 - x1; C = x1y2 - x2y1 (3)

[0020] Retain the line segments with a distance less than the threshold, and initially obtain the line segment diagram after screening;

[0021] According to the characteristics of the A4 paper in three-dimensional space, further screen the line segments, and finally obtain the four line segments that make up the A4 paper;

[0022] In step 5, create a transition coordinate system with the center of the A4 paper as the origin, extract the four corner points of the A4 paper from the line segments, and calculate the coordinates C of the center of the A4 paper through the coordinates of the four corner points (expressed as (x i , y i , z i )(n = 1, 2, 3, 4));

[0023]

[0024] C is the coordinate of the center of the A4 paper, (x i , y i , z i ) are the coordinates of the four corner points. Taking the center of the A4 paper as the origin, a transitional coordinate system is established. The center coordinate of the A4 paper in the transitional coordinate system is (0, 0, 0). Then the coordinates of the four corner points in the transitional coordinate system: (x i ‘, y i ‘, z i ‘);

[0025] In step 6, the coordinate transformation matrices s, R, and t from the line segment model to the transitional coordinate system are calculated through the known points of the A4 paper;

[0026] Given the coordinates of the center of the A4 paper in the line segment model and the transitional coordinate system, the center of the A4 paper in the line segment model is translated to the origin position of the transitional coordinate system. During this process, the transformation relationship t between the line segment model coordinate system and the transitional coordinate system can be obtained:

[0027] t x = X - 0, t y = Y - 0, t z = Z - 0 (5)

[0028] Then the translation matrix can be obtained:

[0029]

[0030] So that the center of the A4 paper in the line segment model is translated to the origin. After that, the known four corner points are rotated relative to the origin to obtain the rotation matrix R from the line segment model coordinate system to the transitional coordinate system, and the line segment model coordinate system is transformed to the transitional coordinate system:

[0031] Rotation about the x-axis: For the points p1 and p2 on two sides in the x-axis direction, find the intersection coordinates of the sides they are on with the yoz plane as Let it be (x m , y m , z m ). Then let the rotation angle about the x-axis be Then there is:

[0032]

[0033] Rotation about the y-axis: For the points p1 and p2 on two sides in the y-axis direction, find the midpoint coordinates as Let it be (x m , y m , z m ). Then let the rotation angle about the y-axis be Then there is:

[0034]

[0035] Rotate around the z-axis: Make the short side of the A4 paper in the x-axis direction and the long side in the y-axis direction. To this end, find the points p1 and p2 for two sides located in the same y-axis direction, and calculate their midpoint coordinates as Let it be (x m , y m , z m ). Then set the rotation angle around the z-axis as Then there is:

[0036]

[0037] R = R x R y R z (10)

[0038] Since the size of the A4 paper in the real coordinate system is known (set as M×N), and the size of the A4 paper in the transition coordinate system can also be calculated (set as m×n), then the scaling parameter s from the line segment model coordinate system to the real coordinate system can be calculated based on the sizes of the A4 paper in the two different coordinate systems:

[0039] The scaling factor is Then:

[0040]

[0041] Measure the parameters of the foot model in the real coordinate system to obtain the parameters of the real foot;

[0042] Based on the known transformation parameters s, R, and t from the line segment model coordinate system to the real coordinate system, transform the line segment model of the foot in the line segment model coordinate system to the real coordinate system, and then calculate the parameters of the foot length and foot width to obtain the parameters of the real foot.

[0043] According to the above technical solution, in step 1, use an A4 paper as a calibration to determine the position of the camera and position the reconstructed foot model;

[0044] When shooting the foot video, it is necessary to ensure that the foot and the A4 paper appear in the picture at the same time.

[0045] Compared with the prior art, the beneficial effects of the present invention:

[0046] By placing an A4 paper in front of the foot, modeling the A4 paper with line segments to better represent its position and shape in space, calculating the center point of the A4 paper, establishing a transition coordinate system with this center point as the origin, translating the center of the A4 paper in the line segment model to the origin position of the transition coordinate system, obtaining the transformation relationship t between the line segment model coordinate system and the transition coordinate system, translating the center of the A4 paper in the line segment model to the origin, rotating the known four corner points relative to the origin, obtaining the rotation matrix R from the line segment model coordinate system to the transition coordinate system, transforming the line segment model coordinate system to the transition coordinate system, calculating the scaling parameter s from the line segment model coordinate system to the real coordinate system according to the sizes of the A4 paper in two different coordinate systems, and obtaining the real foot parameters, the process is more streamlined and can better obtain the foot morphological structure. Brief Description of the Drawings

[0047] The drawings are used to provide further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0048] Figure 1 is the flowchart of the foot scan of the present invention;

[0049] Figure 2 is the schematic diagram of the position of the foot and the A4 paper in Step 1 of the present invention;

[0050] Figure 3 is the schematic diagram of the image mask in Step 2 of the present invention;

[0051] Figure 4 is the schematic diagram of the line segment model in Step 3 of the present invention;

[0052] Figure 5 is the schematic diagram of the line segment point cloud model in Step 3 of the present invention;

[0053] Figure 6 is the schematic diagram of the preliminary line segment extraction in Step 4 of the present invention;

[0054] Figure 7 is the schematic diagram of the final line segment extraction in Step 4 of the present invention;

[0055] Figure 8 is the schematic diagram of establishing the transition coordinate system of the present invention;

[0056] Figure 9 is the schematic diagram of the line segment model rotating around the x-axis in Step 6 of the present invention;

[0057] Figure 10 is the schematic diagram of the line segment model rotating around the y-axis in Step 6 of the present invention;

[0058] Figure 11It is a schematic diagram of the rotation of the line segment model around the z-axis in step 6 of the present invention. Detailed implementation manners

[0059] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only for illustrating and explaining the present invention, and are not used to limit the present invention.

[0060] Embodiment: As Figures 1-5 shown, the present invention provides a technical solution, a foot scanning method based on line segment detection and matching, including the following steps:

[0061] Step 1, Preparation: Place an A4 paper in front of the foot as a reference for calculating foot data, and take a video around the foot to obtain the data required for modeling and positioning.

[0062] Step 2, Use an image mask to show the position of the A4 paper in the image.

[0063] Step 3, Use the captured video frames to perform line segment map reconstruction to obtain a line segment model.

[0064] Step 4, Use the A4 paper mask of the image and the characteristics of the A4 paper to extract four line segments that make up the A4 paper from the line segment model.

[0065] Step 5, Extract four corner points of the A4 paper from the line segments, calculate the coordinates of the center of the A4 paper through the coordinates of the four corner points, and establish a transition coordinate system with the center of the A4 paper as the origin.

[0066] Step 6, Calculate the transformation parameters s, R, t from the line segment model coordinate system to the real coordinate system through the correspondence between the A4 paper in the line segment model coordinate system and the A4 paper in the real coordinate system.

[0067] Step 7, On the basis of knowing the transformation parameters s, R, t from the line segment model coordinate system to the real coordinate system, convert the line segment model of the foot in the line segment model coordinate system to the real coordinate system, and then calculate the foot parameters including the foot length and the foot width, and the parameters of the real foot can be obtained.

[0068] According to the above technical solution, in step 4, the line segments in the line segment model are re-projected into the image, and the distance of each line segment relative to the A4 paper mask is calculated.

[0069] Let the four vertices of the mask quadrilateral be a, b, c, d respectively, and calculate the distance of the line segment x (the coordinates of the two endpoints (ex1, ey1), (ex2, ey2) to the A4 paper):

[0070] D = min{d i (i = 1, 2, 3, 4)} (1)

[0071] d i d is the distance from the line segment to the four sides of the A4 paper. Given the line equation Ax + By + C = 0, then:

[0072]

[0073] Let the vertices of the A4 paper edge be (x1, y1), (x2, y2), then:

[0074] A = y1 - y2; B = x2 - x1; C = x0y2 - x2y1 (3)

[0075] As Figure 6 shown, retain the line segments with distances less than the threshold, and initially obtain a line segment diagram after screening;

[0076] As Figure 7 shown, according to the A4 paper characteristics in three-dimensional space: the angles at the four vertices are right angles, the opposite sides are equal and parallel to each other, further screen the line segments, and finally obtain the four line segments that make up the A4 paper;

[0077] In step 5, create a transitional coordinate system with the center of the A4 paper as the origin, extract the four corner points of the A4 paper from the line segments, and calculate the coordinates C of the center of the A4 paper through the coordinates of the four corner points (expressed as (x i , y i , z i )(n = 1, 2, 3, 4)):

[0078]

[0079] C is the coordinate of the center of the A4 paper, (x i , y i , z i ) are the coordinates of the four corner points. Taking the center of this A4 paper as the origin, establish a transitional coordinate system as Figure 8 shown. The center coordinate of the A4 paper in the transitional coordinate system is (0, 0, 0), then the coordinates of the four corner points in the transitional coordinate system: (x i ‘, y i ‘, z i ‘);

[0080] In step 6, calculate the coordinate transformation matrices s, R, t from the line segment model to the transitional coordinate system through the known points of the A4 paper;

[0081] Given the coordinates of the center of the A4 paper in the line segment model and the transitional coordinate system, translate the center of the A4 paper in the line segment model to the origin position of the transitional coordinate system. In this process, the transformation relationship t between the line segment model coordinate system and the transitional coordinate system can be obtained:

[0082] t x = X - 0, ty = Y - 0, t z = Z - 0 (5)

[0083] Then the translation matrix can be obtained:

[0084]

[0085] Translate the center of the A4 paper in the line segment model to the origin, and then rotate the known four corner points relative to the origin to obtain the rotation matrix R from the line segment model coordinate system to the transition coordinate system, and transform the line segment model coordinate system to the transition coordinate system:

[0086] As Figure 9 shown, rotate around the x-axis: For the points p1 and p2 on two sides in the x-axis direction, find the intersection coordinates of the sides they are on with the yoz plane as Let it be (x m , y m , z m ), then let the rotation angle around the x-axis be Then there is:

[0087]

[0088] As Figure 10 shown, rotate around the y-axis: For the points p1 and p2 on two sides in the y-axis direction, find their midpoint coordinates as Let it be (x m , y m , z m ), then let the rotation angle around the y-axis be Then there is:

[0089]

[0090] As Figure 11 shown, rotate around the z-axis: Make the short side of the A4 paper in the x-axis direction and the long side in the y-axis direction. For this purpose, find the midpoint coordinates of the points p1 and p2 on two sides in the y-axis direction as Let it be (x m , y m , z m ), then let the rotation angle around the z-axis be Then there is:

[0091]

[0092] R = R x R y R z (10)

[0093] Since the size of the A4 paper in the real coordinate system is known (assumed to be M×N), and the size of the A4 paper in the transition coordinate system can also be calculated (assumed to be m×n), then the scaling parameter s from the line segment model coordinate system to the real coordinate system can be calculated based on the sizes of the A4 paper in the two different coordinate systems:

[0094] The scaling factor is Then:

[0095]

[0096] Measure the parameters of the foot model in the real coordinate system to obtain the parameters of the real foot;

[0097] Based on the known transformation parameters s, R, t from the line segment model coordinate system to the real coordinate system, transform the line segment model of the foot in the line segment model coordinate system to the real coordinate system, and then calculate the parameters of the foot length and foot width to obtain the parameters of the real foot.

[0098] According to the above technical solution, in step 1, use an A4 paper as a calibration to determine the position of the camera and position the reconstructed foot model;

[0099] When shooting the foot video, it is necessary to ensure that the foot and the A4 paper appear in the picture at the same time.

[0100] Finally, it should be noted that the above are only the preferred examples of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A foot scanning method based on line segment detection and matching, characterized in that: It includes the following steps: Step 1, Preparation: Place an A4 paper in front of the foot as a reference for calculating foot data, take a video around the foot to obtain the data required for modeling and positioning; Step 2, Use an image mask to show the position of the A4 paper in the image; Step 3, Use the video frames taken to perform line map reconstruction to obtain a line model; Step 4, Use the A4 paper mask of the image and the characteristics of the A4 paper to extract four lines that make up the A4 paper from the line model; Step 5, Extract four corner points of the A4 paper from the lines, calculate the coordinates of the center of the A4 paper through the coordinates of the four corner points, and establish a transition coordinate system with the center of the A4 paper as the origin; Step 6, Calculate the transformation parameters s, R, t from the line model coordinate system to the real coordinate system through the correspondence between the A4 paper in the line model coordinate system and the A4 paper in the real coordinate system; Step 7, On the basis of knowing the transformation parameters s, R, t from the line model coordinate system to the real coordinate system, convert the line model of the foot in the line model coordinate system to the real coordinate system, and then calculate the foot parameters including foot length and foot width to obtain the parameters of the real foot.

2. The foot scanning method based on line segment detection and matching according to claim 1, characterized in that In the said Step 4, project the lines in the line model onto the image and calculate the distance of each line relative to the A4 paper mask; Let the four vertices of the mask quadrilateral be a, b, c, d respectively, and calculate the distance of line x (the coordinates of the two endpoints (ex1, ey1), (ex2, ey2) to the A4 paper): D = min{d i (i = 1, 2, 3, 4)} (1) d i It is the distance from the line segment to the four sides of the A4 paper. Given the linear equation Ax + By + C = 0, then: Let the vertices of the A4 paper edge be (x1, y1), (x2, y2), then: A = y1 - y2; B = x2 - x1; C = x0y2 - x2y1 (3) Retain the lines with a distance less than the threshold, and initially obtain a line diagram after screening; According to the characteristics of the A4 paper in three-dimensional space, further screen the lines, and finally obtain four lines that make up the A4 paper; In step 5, a transition coordinate system is created with the center of the A4 paper as the origin, and the four corner points of the A4 paper are extracted from the line segment. The coordinates of the center of the A4 paper C are calculated through the four corner point coordinates (expressed as (x i , y i , z i )(n = 1, 2, 3, 4)): C is the coordinate of the center of the A4 paper, (x i , y i , z i ) are the coordinates of the four corner points. Taking the center of the A4 paper as the origin, a transitional coordinate system is established. The center coordinate of the A4 paper in the transitional coordinate system is (0, 0, 0). Then the coordinates of the four corner points in the transitional coordinate system are: (x i ‘, y i ‘, z i ‘); In the said Step 6, calculate the coordinate transformation matrix s, R, t from the line model to the transition coordinate system through the known points of the A4 paper; Given the coordinates of the center of the A4 paper in the line model and the transition coordinate system, translate the center of the A4 paper in the line model to the origin position of the transition coordinate system. During this process, the transformation relationship t between the line model coordinate system and the transition coordinate system can be obtained: t x = X - 0, t y = Y - 0, t z = Z - 0 (5) Then the translation matrix can be obtained: Make the center of the A4 paper in the line model translate to the origin, and then rotate the known four corner points relative to the origin to obtain the rotation matrix R from the line model coordinate system to the transition coordinate system, and transform the line model coordinate system to the transition coordinate system: Rotation about the x-axis: For two points p1 and p2 on two edges in the x-axis direction, find the intersection coordinates of the edges they are on with the yoz plane as Let it be (x m , y m , z m ). Then, let the rotation angle about the x-axis be Then we have: Rotation about the y-axis: For two points p1 and p2 located on two edges in the y-axis direction, find the coordinates of their midpoint as Let it be (x m , y m , z m ). Then, let the rotation angle about the y-axis be Then we have: Rotation about the z-axis: Align the short side of the A4 paper with the x-axis and the long side with the y-axis. To do this, find two points p1 and p2 on the two sides that are in the same y-axis direction, and calculate the coordinates of their midpoint as Let it be (x m , y m , z m ). Then, let the rotation angle about the z-axis be Then we have: R = R x R y R z (10) Since the size of the A4 paper in the real coordinate system is known (set as M×N), and the size of the A4 paper in the transition coordinate system can also be calculated (set as m×n), then the scaling parameter s from the line model coordinate system to the real coordinate system can be calculated according to the sizes of the A4 paper in the two different coordinate systems; The scaling factor is Then: Measure the parameters of the foot model in the real coordinate system to obtain the parameters of the real foot; Based on the transformation parameters s, R, and t from the known line segment model coordinate system to the real coordinate system, the line segment model of the foot in the line segment model coordinate system is transformed into the real coordinate system, and then the parameters of the real foot can be obtained by calculating the length and width parameters of the foot.

3. A foot scanning method based on line segment detection and matching according to claim 1, characterized in that In step 1, a sheet of A4 paper is used as a calibration to determine the position of the camera and locate the reconstructed foot model. When shooting the foot video, it is necessary to ensure that the foot and the A4 paper appear in the picture at the same time.

Images