Measurement method and equipment based on human body mesh model and point cloud data

Abstract

The present application relates to the field of data processing technology, and discloses a measurement method and device based on a human body mesh model and point cloud data, including: determining a first feature point based on the human body mesh model, and determining a second feature point based on the human body mesh model and point cloud data; wherein the first feature point is a point corresponding to a first specified position of the human body; the second feature point is a point corresponding to a second specified position of the human body; the human body is measured using the first feature point and / or the second feature point, and the first feature point and the second feature point are determined respectively using the human body mesh model and point cloud data, thereby avoiding the problem of data missing when the feature points are determined only by the human body mesh model, and the present application uses the first feature point and / or the second feature point to measure the human body, thereby avoiding the problem of inaccurate measurement due to data missing.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a measurement method and device based on a human body mesh model and point cloud data. Background Art

[0002] With the development of 3D human body modeling technology, there is an increasing demand for virtual fitting using 3D human body data. Existing techniques involve obtaining human body point cloud data through depth camera scanning, reconstructing the point cloud data into a human mesh model, then determining feature points in the mesh model using a thermal kernel algorithm or machine learning. The length and girth of the human body are then measured based on these feature points. However, the thermal kernel algorithm requires considerable time, failing to meet the requirements of real-time measurement. Furthermore, the point cloud from the scanned human body is sparse at the top of the head, and the point cloud from the soles of the feet cannot be captured. This results in missing data at the top and bottom of the point cloud, leading to low accuracy in measurements of height, leg length, and other lengths using the mesh model. When measuring girth, errors may occur in reconstructing the mesh model for chest and leg circumference due to the body's posture, where the arms and torso are close together, or where the legs are close together. This results in low measurement accuracy for chest and waist circumference, making it difficult to accurately measure the human body. Summary of the Invention

[0003] The embodiments of the present application provide a measurement method and device based on a human body mesh model and point cloud data, thereby accurately measuring the length and circumference of the human body.

[0004] In a first aspect, an embodiment of the present application provides a measurement method based on a human body mesh model and point cloud data, the method comprising:

[0005] Determining a first feature point based on the human body mesh model, and determining a second feature point based on the human body mesh model and the point cloud data; wherein the first feature point is a point corresponding to a first specified position of the human body; and the second feature point is a point corresponding to a second specified position of the human body;

[0006] The human body is measured using the first feature points and / or the second feature points.

[0007] The present application uses the human body mesh model and point cloud data to respectively determine the first feature point and the second feature point, which can avoid the problem of data missing when the feature point is determined only by the human body mesh model. In addition, the present application uses the first feature point and / or the second feature point to measure the human body, which can avoid the problem of inaccurate measurement due to data missing.

[0008] Optionally, when the coordinates of each vertex in the human body mesh model include at least a first direction coordinate value and a second direction coordinate value, and the first feature point is a perineum point, determining the first feature point based on the human body mesh model includes:

[0009] Sort the second-direction coordinate values of each vertex from small to large. If the second-direction coordinate values of the vertices are the same, sort the vertices from small to large according to the first-direction coordinate values. Filter the second-direction coordinate value of each vertex based on the maximum second-direction coordinate value, the minimum second-direction coordinate value, and the first specified rule to obtain a first vertex set corresponding to the second-direction coordinate value.

[0010] Traverse rightward from the first vertex with the smallest coordinate value in the second direction in the first vertex set. After the traversal is completed, record the maximum coordinate value and the minimum coordinate value in the first direction of the current layer as the first value set, and continue traversing rightward from the vertex before the first vertex.

[0011] After the traversal is completed, if the maximum coordinate value and the minimum coordinate value in the first direction of the current layer meet the second specified rule with the maximum coordinate value and the minimum coordinate value in the first direction of the previous layer, then the first value set is updated according to the new maximum coordinate value and the new minimum coordinate value in the first direction; if the maximum coordinate value and the minimum coordinate value in the first direction of the current layer meet the third specified rule with the maximum coordinate value and the minimum coordinate value in the first direction of the previous layer, then the traversal is terminated;

[0012] Obtain the ninth vertex corresponding to the second direction coordinate value of the layer where the traversal ends, traverse leftward and rightward from the ninth vertex respectively until the traversal ends, mark each traversed vertex and the vertices within the specified neighborhood of each vertex as the vertex of the left leg, and determine the vertex with the largest first direction coordinate value of the layer where the traversal ends as the second vertex;

[0013] Traverse all vertices in the first vertex set except the vertex of the left leg once, and mark them as vertices of the right leg, and determine the vertex with the smallest coordinate value in the first direction at the layer where the traversal ends as the third vertex;

[0014] The perineum point is determined based on the second vertex and the third vertex.

[0015] This application first filters all vertices in the human body mesh model, and then traverses the specified rules based on the filtered vertices to obtain the left leg vertex set and the right leg vertex set, and then uses the vertex with the largest second direction coordinate value and the largest first direction coordinate value in the left leg vertex set and the vertex with the largest second direction coordinate value and the smallest first direction coordinate value in the right leg vertex set to determine the perineum point, so as to obtain an accurate first feature point.

[0016] Optionally, determining the perineum point based on the second vertex and the third vertex includes:

[0017] Obtaining a second vertex set based on vertices within a designated neighborhood of the second vertex and vertices within a designated neighborhood of the third vertex; and determining a first threshold range based on the first direction coordinate value of the second vertex and the first direction coordinate value of the third vertex;

[0018] The fourth vertex in the second vertex set whose coordinate value in the first direction is within the first threshold range and whose coordinate value in the second direction is the smallest and which is traversed upwards and has existing vertices is determined as the perineum point.

[0019] This application further determines the perineum point through different constraints, so that the obtained first feature point is more accurate.

[0020] Optionally, the method further includes:

[0021] An angle between a line connecting the fourth vertex and vertices traversed upward from the fourth vertex and a line connecting the fourth vertex and vertices traversed downward from the fourth vertex is smaller than a second threshold.

[0022] This application further determines the perineum point through different constraints, so that the obtained first feature point is more accurate.

[0023] Optionally, when the human body mesh model and vertices in the point cloud data are in one-to-one correspondence, and the second feature points are left shoulder points and right shoulder points, determining the second feature points based on the human body mesh model and the point cloud data includes:

[0024] Determining a left neck point and a right neck point in the point cloud data based on a preset head height, and determining a fifth vertex with a minimum coordinate value in the first direction and a sixth vertex with a maximum coordinate value in the first direction based on the human body mesh model;

[0025] Based on the specified number of rows in which the left neck point and the right neck point are located, the left shoulder point and the right shoulder point of each row are determined, and the seventh vertex with the maximum angle between the lines connecting the left neck point and the fifth vertex in the point cloud data is determined as the final left shoulder point, and the eighth vertex with the maximum angle between the lines connecting the right neck point and the sixth vertex in the point cloud data is determined as the final right shoulder point.

[0026] Due to the disorder of point cloud data, this application makes a one-to-one correspondence between the human body mesh model and the point cloud data, and then determines the shoulder point by combining the human body mesh model and the point cloud data, so as to obtain an accurate second feature point.

[0027] Optionally, when the point cloud data is single-view point cloud data and there are multiple point cloud data, after determining the seventh vertex and the eighth vertex in each point cloud data, the final left shoulder point and the final right shoulder point are determined by the following method:

[0028] The plurality of point cloud data are subjected to point cloud stitching, and during the stitching process, the final left shoulder point and the final right shoulder point are determined based on the plurality of the seventh vertices and the plurality of the eighth vertices.

[0029] Since there are inaccuracies in determining the shoulder points based only on single-view point cloud data, this application further obtains more accurate shoulder points by fusing the shoulder points in each single-view point cloud data when stitching multiple single-view point cloud data.

[0030] Optionally, measuring the human body by using the first feature point and / or the second feature point includes:

[0031] The first feature points are used to measure leg length, chest circumference, waist circumference, hip circumference and thigh circumference; the second feature points are used to measure arm length and shoulder width.

[0032] By measuring different lengths and circumferences at different feature points, the obtained body measurements are more accurate.

[0033] In a second aspect, an embodiment of the present application provides a measurement device based on a human body mesh model and point cloud data, comprising: a processor and a display;

[0034] The display is used to display a user operation interface;

[0035] The processor is configured to execute:

[0036] Determining a first feature point based on the human body mesh model, and determining a second feature point based on the human body mesh model and the point cloud data; wherein the first feature point is a point corresponding to a first specified position of the human body; and the second feature point is a point corresponding to a second specified position of the human body;

[0037] The human body is measured using the first feature points and / or the second feature points.

[0038] Optionally, when the coordinates of each vertex in the human body mesh model include at least a first direction coordinate value and a second direction coordinate value, and the first feature point is a perineum point, the processor is configured to determine the first feature point based on the human body mesh model by specifically performing:

[0039] Sort the second-direction coordinate values of each vertex from small to large. If the second-direction coordinate values of the vertices are the same, sort the vertices from small to large according to the first-direction coordinate values. Filter the second-direction coordinate value of each vertex based on the maximum second-direction coordinate value, the minimum second-direction coordinate value, and the first specified rule to obtain a first vertex set corresponding to the second-direction coordinate value.

[0040] Traverse rightward from the first vertex with the smallest coordinate value in the second direction in the first vertex set. After the traversal is completed, record the maximum coordinate value and the minimum coordinate value in the first direction of the current layer as the first value set, and continue traversing rightward from the vertex before the first vertex.

[0041] After the traversal is completed, if the maximum coordinate value and the minimum coordinate value in the first direction of the current layer meet the second specified rule with the maximum coordinate value and the minimum coordinate value in the first direction of the previous layer, then the first value set is updated according to the new maximum coordinate value and the new minimum coordinate value in the first direction; if the maximum coordinate value and the minimum coordinate value in the first direction of the current layer meet the third specified rule with the maximum coordinate value and the minimum coordinate value in the first direction of the previous layer, then the traversal is terminated;

[0042] Obtain the ninth vertex corresponding to the second direction coordinate value of the layer where the traversal ends, traverse leftward and rightward from the ninth vertex respectively until the traversal ends, mark each traversed vertex and the vertices within the specified neighborhood of each vertex as the vertex of the left leg, and determine the vertex with the largest first direction coordinate value of the layer where the traversal ends as the second vertex;

[0043] Traverse all vertices in the first vertex set except the vertex of the left leg once, and mark them as vertices of the right leg, and determine the vertex with the smallest coordinate value in the first direction at the layer where the traversal ends as the third vertex;

[0044] The perineum point is determined based on the second vertex and the third vertex.

[0045] Optionally, when the processor is configured to determine the perineum point based on the second vertex and the third vertex, the processor is specifically configured to execute:

[0046] Obtaining a second vertex set based on vertices within a designated neighborhood of the second vertex and vertices within a designated neighborhood of the third vertex; and determining a first threshold range based on the first direction coordinate value of the second vertex and the first direction coordinate value of the third vertex;

[0047] The fourth vertex in the second vertex set whose coordinate value in the first direction is within the first threshold range and whose coordinate value in the second direction is the smallest and which is traversed upwards and has existing vertices is determined as the perineum point.

[0048] Optionally, the processor is further configured to execute:

[0049] An angle between a line connecting the fourth vertex and vertices traversed upward from the fourth vertex and a line connecting the fourth vertex and vertices traversed downward from the fourth vertex is smaller than a second threshold.

[0050] Optionally, the human body mesh model and vertices in the point cloud data are in one-to-one correspondence, and when the second feature points are left shoulder points and right shoulder points, the processor is configured to determine the second feature points based on the human body mesh model and the point cloud data by specifically executing:

[0051] Determining a left neck point and a right neck point in the point cloud data based on a preset head height, and determining a fifth vertex with a minimum coordinate value in the first direction and a sixth vertex with a maximum coordinate value in the first direction based on the human body mesh model;

[0052] Based on the specified number of rows in which the left neck point and the right neck point are located, the left shoulder point and the right shoulder point of each row are determined, and the seventh vertex with the maximum angle between the lines connecting the left neck point and the fifth vertex in the point cloud data is determined as the final left shoulder point, and the eighth vertex with the maximum angle between the lines connecting the right neck point and the sixth vertex in the point cloud data is determined as the final right shoulder point.

[0053] Optionally, when the point cloud data is single-view point cloud data and there are multiple point cloud data, after determining the seventh vertex and the eighth vertex in each point cloud data, the processor determines the final left shoulder point and the final right shoulder point in the following manner:

[0054] The plurality of point cloud data are subjected to point cloud stitching, and during the stitching process, the final left shoulder point and the final right shoulder point are determined based on the plurality of the seventh vertices and the plurality of the eighth vertices.

[0055] Optionally, when the processor is configured to perform the measurement of the human body using the first feature point and / or the second feature point, the processor is specifically configured to perform:

[0056] The first feature points are used to measure leg length, chest circumference, waist circumference, hip circumference and thigh circumference; the second feature points are used to measure arm length and shoulder width.

[0057] In a third aspect, an embodiment of the present application provides a measurement device based on a human body mesh model and point cloud data, the device comprising:

[0058] a feature point determination module, configured to determine a first feature point based on the human body mesh model, and to determine a second feature point based on the human body mesh model and the point cloud data; wherein the first feature point is a point corresponding to a first designated position of the human body; and the second feature point is a point corresponding to a second designated position of the human body;

[0059] A measurement module is used to measure the human body using the first feature point and / or the second feature point.

[0060] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which, when the instructions in the computer-readable storage medium are executed by a processor, enables a measurement device based on a human mesh model and point cloud data to execute any of the methods provided in the first aspect of the present application.

[0061] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements any of the methods provided in the first aspect of the present application.

[0062] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 A schematic diagram of coordinate directions provided for an embodiment of the present application;

[0065] Figure 2 A schematic diagram of the top, bottom, left, and right of a vertex provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram showing an inaccurate determination of the perineal point provided in an embodiment of the present application;

[0067] Figure 4A schematic diagram of determining a shoulder point according to an embodiment of the present application;

[0068] Figure 5a A schematic diagram of a 1-neighborhood search provided in an embodiment of the present application;

[0069] Figure 5b A schematic diagram of a 2-neighborhood search provided in an embodiment of the present application;

[0070] Figure 5c A schematic diagram of a 2-neighborhood search provided in an embodiment of the present application;

[0071] Figure 5d A schematic diagram of a 2-neighborhood search provided in an embodiment of the present application;

[0072] Figure 6 A schematic diagram of a girth calculation point sequence G provided in an embodiment of the present application;

[0073] Figure 7 A schematic diagram of further processing the girth calculation point sequence G using the convex hull Graham algorithm provided in one embodiment of the present application;

[0074] Figure 8 A flow chart of a measurement method based on a human body mesh model and point cloud data provided in one embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to enable ordinary people in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0076] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0077] With the development of 3D human body modeling technology, there is an increasing demand for virtual fitting using 3D human body data. Existing techniques involve obtaining human body point cloud data through depth camera scanning, reconstructing the point cloud data into a human mesh model, then determining feature points in the mesh model using a thermal kernel algorithm or machine learning. The length and girth of the human body are then measured based on these feature points. However, the thermal kernel algorithm requires considerable time, failing to meet the requirements of real-time measurement. Furthermore, the point cloud from the scanned human body is sparse at the top of the head, and the point cloud from the soles of the feet cannot be captured. This results in missing data at the top and bottom of the point cloud, leading to low accuracy in measurements of height, leg length, and other lengths using the mesh model. When measuring girth, errors may occur in reconstructing the mesh model for chest and leg circumference due to the body's posture, where the arms and torso are close together, or where the legs are close together. This results in low measurement accuracy for chest and waist circumference, making it difficult to accurately measure the human body.

[0078] The present application uses the human body mesh model and point cloud data to respectively determine the first feature point and the second feature point, which can avoid the problem of data missing when the feature point is determined only by the human body mesh model. In addition, the present application uses the first feature point and / or the second feature point to measure the human body, which can avoid the problem of inaccurate measurement due to data missing.

[0079] To further illustrate the technical solutions provided by the embodiments of the present application, the following is a detailed description of the technical solutions in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present application provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative work. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiments of the present application.

[0080] like Figure 1 As shown, the direction of the right hand of the human body is defined as the positive direction of the x-axis, the direction of the top of the head of the human body is defined as the positive direction of the y-axis, and the direction of the back of the human body is defined as the positive direction of the z-axis. Figure 2 As shown, the definition of traversing up, down, left, and right for any vertex in the human body mesh model is as follows:

[0081] (1) Vertex traversal to the left refers to the closed loop or line segment consisting of a specified number of vertices in the xz plane, which is traversed in a clockwise direction from the positive direction of the y-axis;

[0082] (2) Vertex rightward traversal refers to the traversal of a closed loop or a line segment consisting of a specified number of vertices in the xz plane in a counterclockwise direction from the positive direction of the y axis;

[0083] (3) Vertex upward traversal refers to the closed loop or line segment consisting of a specified number of vertices in the yz plane, which is traversed in a counterclockwise direction from the positive direction of the x-axis;

[0084] (4) Vertex downward traversal refers to a closed loop or a line segment consisting of a specified number of vertices in the yz plane, which is traversed in a clockwise direction from the positive direction of the x-axis.

[0085] When slicing in the xy plane, the vertex traversal must be performed clockwise or counterclockwise within a line segment consisting of a specified number of vertices, depending on the specific posture of the person. Furthermore, the vertical, horizontal, and vertical traversal of the vertices described above is defined from the frontal perspective of the person. If defined from the back, the vertical and horizontal traversal order is swapped, while the horizontal and vertical traversal remains unchanged.

[0086] Here, the human body mesh model may be a model that has undergone denoising, mesh navigation, and segmentation processing. The specific methods used for denoising and segmentation processing are not limited here and can be adjusted according to actual conditions.

[0087] Next, the human body measurement method of this application is introduced from three aspects: the first is the determination of feature points, the second is the measurement of length, and the third is the measurement of circumference.

[0088] 1. Determination of feature points

[0089] 1. Determination of the first characteristic point - perineum point

[0090] According to the above description, each vertex in the human body mesh model has a first direction coordinate value x, a second direction coordinate value y, and a third direction coordinate value z.

[0091] Sort the second direction coordinate value y of each vertex from small to large. If the second direction coordinate value y of the vertices is the same, sort the vertices from small to large according to the first direction coordinate value x of the vertices. Based on the maximum second direction coordinate value, the minimum second direction coordinate value and the first specified rule, filter the second direction coordinate value of each vertex to obtain the first vertex set corresponding to the second direction coordinate value. For example, assuming that the maximum second direction coordinate value of the vertex is y max , the minimum coordinate value of the second direction is y min , the first specified rule is y up =y min +0.6764*(y max -y min )–343.6362mm, and y is calculated up After that, the second direction coordinate value y of each vertex is greater than y up The first vertex set can be obtained by filtering out the vertices.

[0092] In the first vertex set, traverse rightward from the first vertex with the minimum coordinate value in the second direction until the traversal is completed, and record the maximum coordinate value and the minimum coordinate value in the first direction of the current layer as the first value set. For example, the first direction coordinate value x of the first vertex can be set to be less than -20mm. If the first direction coordinate value x of the vertex corresponding to the minimum coordinate value in the second direction of the first vertex set is greater than -20mm, continue to determine the first direction coordinate value x of the next vertex in the first vertex set sorting until the first vertex that meets the condition of less than -20mm is determined as the first vertex. Start traversing rightward from the first vertex, after the traversal in the current layer is completed, assume that the maximum coordinate value in the first direction of the current layer is curr_x max , the minimum coordinate value of the first direction is curr_x min , that is, the first value set = {curr_x max , curr_x min}.

[0093] Then continue to traverse rightward from the vertex before the first vertex in the first vertex set. If the maximum coordinate value of the first direction of the current layer curr_x is max and the minimum coordinate value curr_x in the first direction min The maximum coordinate value last_x of the first direction of the previous layer max and the minimum coordinate value of the first direction last_x min The second specified rule is met between the two, and the second specified rule is: the second direction coordinate value y of the current vertex is less than y min +300mm, or (curr_x max -curr_x min )≥0.95(last_x max -last_x min ), then update the first value set according to the new maximum coordinate value of the first direction and the new minimum coordinate value of the first direction; if the maximum coordinate value of the first direction of the current layer curr_x max and the minimum coordinate value curr_x in the first direction min The maximum coordinate value last_x of the first direction of the previous layer max and the minimum coordinate value of the first direction last_x min The third specified rule is met, where the third specified rule is: y is greater than y min +300mm, last_x max is greater than 0, and (curr_x max -curr_x min )>1.6(last_x max -last_x min), it means that the body may be located in the hip area or in the adhesion area of the left leg and the right leg, and the traversal can be ended.

[0094] Here, if the maximum coordinate value of the first direction of the current layer curr_x max and the minimum coordinate value curr_x in the first direction min The maximum coordinate value last_x of the first direction of the previous layer max and the minimum coordinate value of the first direction last_x min If neither the second nor the third specified rule is met, the vertex is skipped. Furthermore, during the vertex loop of the current layer, it is necessary to avoid falling into a subring. The specific method for avoiding falling into a subring is not limited here.

[0095] Get the ninth vertex corresponding to the second direction coordinate value y of the layer where the traversal ends, traverse left and right from the ninth vertex respectively, until the traversal ends, mark each vertex traversed and the vertices in the specified neighborhood of each vertex as the vertex of the left leg, and mark the vertex v with the largest first direction coordinate value of the layer where the traversal ends as the vertex of the left leg left_leg_top_right Determine as the second vertex; traverse all vertices in the first vertex set except the vertex of the left leg once, and mark them as the vertices of the right leg, and end the traversal of the vertex v with the smallest coordinate value in the first direction of the layer right_leg_top_left The third vertex is determined; and the perineum point is determined based on the second and third vertices. The specified neighborhood may be a 1-neighborhood. A detailed description of the 1-neighborhood will be provided later and will not be repeated here.

[0096] In order to more accurately determine the perineum point, the present application obtains a second vertex set based on the vertices in the neighborhood of the second vertex and the vertices in the neighborhood of the third vertex; and based on the first direction coordinate value v of the second vertex left_leg_top_ right .x and the first direction coordinate value v of the third vertex right_leg_top_left .x determines the first threshold range; the fourth vertex in the second vertex set whose first direction coordinate value is within the first threshold range and whose second direction coordinate value is the smallest and whose vertices are traversed upwards is determined as the perineum point. Exemplarily, the specified neighborhood can be a 2-neighborhood, and according to (v left_leg_top_right .x+v right_leg_top_left .x) / 2 sets the first threshold range. A further constraint can be added: the fourth vertex can be traversed upward at least twice. A detailed description of the 2-neighborhood will be provided later and will not be repeated here.

[0097] like Figure 3As shown, for the case where the feet are close together and the determined perineum point is relatively low, the fourth vertex determined above is traversed upwards, and the angle between the line between the fourth vertex and the vertex traversed upwards once and the line between the fourth vertex and the vertex traversed downwards once is smaller than the second threshold. For example, the fourth vertex is v k , the vertex of the fourth vertex traversed upward is v k+1 , the vertex that is traversed downward once is v k-1 , if cos <v k+1 v k , v k v k-1 > less than 0.85, the fourth vertex v k The second direction coordinate value y>y up -120mm, the perineum point is more accurate.

[0098] 2. Determination of the second feature point - shoulder point

[0099] Due to the disorder of point cloud data, the depth map collected by the depth camera is ordered. The information of the adjacent pixels of any pixel in the depth map can be obtained. Therefore, the depth value in the depth map is converted into the coordinate value in the point cloud data, and the human body mesh model and the vertices in the point cloud data are obtained in a one-to-one correspondence.

[0100] like Figure 4 As shown in the figure, the shoulder point is generally a convex point. Combined with the human body mesh model, the row position A between the head area (such as head height 250mm) and the shoulder in the point cloud data is determined. Since there is no data in the area other than the human body in the point cloud data, the left neck point v in the row position A can be determined. left_neck and right neck point v right_neck , and the leftmost point v of the human body left_bottom and the rightmost point v of the human body right_bottom , wherein the fifth vertex with the smallest coordinate value in the first direction can also be determined based on the human body mesh model, that is, v left_bottom , the sixth vertex with the largest coordinate value in the first direction is v right_bottom .

[0101] Based on the left neck point v left_neck and right neck point v right_neck Within the specified number of rows, determine the left shoulder point and right shoulder point of each row, and respectively connect them with the left neck point v left_neck 、The fifth vertex v left_bottom The seventh vertex of the maximum angle between the points in the point cloud data is determined as the final left shoulder point, which will be connected to the right neck point v right_neck 、The sixth vertex v right_bottomThe eighth vertex with the maximum angle of the line connecting the points in the point cloud data is determined as the final right shoulder point.

[0102] For example, starting from row position A, shoulder points are searched row by row from high to low. It can be assumed that the number of search rows is 5. Taking the left shoulder point as an example, if the difference between the first direction coordinate value x of the currently searched left shoulder point and the first direction coordinate value x of the last searched left shoulder point is less than 50 mm, or the first direction coordinate value x of the currently searched left shoulder point is equal to the first direction coordinate value x of the last searched left shoulder point, then the shoulder point is skipped. left_neck v and vector vv left_bottom The cosine value of the angle formed is cos <v left_neck v, vv left_bottom The largest vertex v is determined as the left shoulder point.

[0103] The aforementioned point cloud data is single-view point cloud data. When multiple point cloud data are present, after determining the seventh and eighth vertices in each point cloud data, the multiple point cloud data are then stitched together. During the stitching process, the final left and right shoulder points are determined based on the multiple seventh and eighth vertices. For example, if point cloud voxel downsampling is used to update the shoulder point positions, a KD tree is constructed based on the downsampled point cloud to search for the nearest vertex of shoulder point position A, which is used as the updated shoulder point position B.

[0104] 2. Length Measurement

[0105] 1. Height measurement

[0106] One measurement method is to determine the maximum coordinate value y in the second direction based on the human body mesh model. max , the minimum coordinate value y in the second direction min , and the difference between the two is the height h. Another measurement method is to determine the highest and lowest points in the point cloud data based on the point cloud data aligned with the human body mesh model, and the difference between the two is the height h.

[0107] 2. Measurement of arm length

[0108] The single-view point cloud data aligned with the human body mesh model is used for measurement. Taking the left arm length measurement process as an example, the second direction coordinate value y is greater than y in the single-view point cloud data. min +1 / 3·h points, then sort the selected points according to the first direction coordinate value x from small to large. If the x values are equal, sort them according to the y values from small to large. Select the first 1 / 20 vertices after sorting, and place the distance between these vertices and the left shoulder point v left_shoulder The maximum distance L is defined as the left arm length.

[0109] 3. Leg length measurement

[0110] One measurement method is to determine the minimum coordinate value y in the second direction based on the human body mesh model. min , y min The difference between the second direction coordinate value of the perineum point is defined as the left leg length or the right leg length. Another measurement method is to determine the minimum coordinate value y in the second direction based on single view point cloud data or multi-view point cloud data. min , y min The difference between the second direction coordinate value of the perineum point and the left leg length or the right leg length is defined as the left leg length or the right leg length. Here, the multi-view point cloud data is obtained by merging multiple single-view point cloud data.

[0111] 4. Measurement of shoulder width

[0112] For the left arm, we start from the highest point of the left arm segmentation in the human body mesh model after segmentation and traverse along the left side of the neck. The traversal rule is: the third direction coordinate values of the vertices are equal, and the vertices along the positive or negative direction of the x-axis are gradually moved. The second direction coordinate value y of the left shoulder point is obtained through the single view point cloud data. s , use the above traversal rules to determine the two adjacent vertices v of the left shoulder point left_up 、v left_down , satisfying v left_down .y≤y s ≤v left_up .y, and the adjacent vertices can be traversed to the left and right, and the angle between the line between the target traversal point and the highest point of the left arm segmentation and the line between the target traversal point and the positive / negative direction of the x-axis is the largest angle relative to other traversal points. Figure 5a As shown, vertex 1 is the current traversal point, and vertex 2 is the vertex of vertex 1's 1-neighborhood traversal to the right. Figure 5b As shown, vertex 3 is the current traversal point, and vertex 4 is the vertex of vertex 3's 2-neighborhood traversal to the right. Figure 5c As shown, vertex 5 is the current traversal point, and vertex 6 is the vertex of vertex 5's 2-neighborhood approximately traversed to the right. Figure 5d As shown, vertex 7 is the current traversal point, and vertex 7 cannot be traversed to the right within the 2-neighborhood.

[0113] Then, follow the back of the grid from v left_up Start traversing to the right until you reach v right_up , calculate the total length s of the line segments connecting the vertices up , similarly, calculate from v left_down Traverse to v right_down The total length s of the line segments connecting the vertices down Therefore, define t=(y s –v left_down .y) / (v left_up .y), then shoulder width s=s up t+sdown ·(1-t).

[0114] 3. Measurement of circumference

[0115] By combining the human body mesh model and point cloud data to measure the circumference, before measuring the circumference, we first introduce the process of determining the vertex set S for calculating the horizontal circumference:

[0116] Set a starting vertex v start , and v start Add to the vertex set S, from v start Start traversing to the right, assuming v start The vertex traversed to the left is v last , v start The vertex traversed to the right is v next . For the non-existent v next In the case of next , the search rule is v next The y value and v start The absolute difference between the y values is less than the preset error threshold y error , vector v last v start With vector v start v next The dot product of is greater than 0, vector v last v start With vector v start v next The y of the cross product vector is greater than 0 and does not belong to the vertex set S. next Add to the vertex set S, if the above v start If you cannot traverse to the right, choose to traverse to the left. Refer to the above steps for the traversal process.

[0117] In determining v next Afterwards, if v next is found through 2-neighborhood search, then v is searched based on the threshold set by the user. next The y value is corrected, and then the v next The vertices in the range to be measured are combined into point cloud data of the human body by using v next The coordinates of the point cloud data after the stitching is completed are searched in the KD tree to determine k candidate points. start The absolute value of the difference between the y values of start The nearest point v'. When the vector v next When the length of v' is less than the preset threshold A, v next Add to the girth calculation point sequence G, when the vector v nextWhen the length of v' is not less than the preset threshold A, the y value of v' is modified to v next After the y value of v next Add to the girth calculation point sequence G, where the girth calculation point sequence G and the vertex set S are two different expressions of vertex composition. Figure 6 As shown in the figure, the vertices connected to each other are all vertices added to the girth calculation point sequence G, and the unconnected and independent vertices are all vertices not added to the girth calculation point sequence G. Figure 6 The eps in is the preset threshold A.

[0118] In addition, if the vertices in the girth calculation point sequence G are not closed loops, the girth calculation point sequence G can be further processed by using the convex hull Graham algorithm. The specific processing steps are as follows:

[0119] A1, determine the point with the smallest x value in G. If there are multiple points with the smallest x value, continue to determine the point with the smallest z value, such as Figure 7 After deleting p0 from the sequence G, sort them from largest to smallest by inverse tangent atan2(pz-p0.z, px-p0.x) to obtain the sorted results of p1, p2, p3, p4, p5, p6, p7, p8, and p9, and add p0 to the girth calculation point sequence G'.

[0120] A2: Traverse G starting from p1 according to the above sorting results. If G[i] and G[i+1] are equal, skip it. Obtain the last point G', G'.back, and G[i] to form vector v0. G[i] and G[i+1] to form vector v1. Then calculate the angle between these two vectors. If the angle is greater than a negative preset angle threshold (for example, the convex hull can be set to 0), add G[i] to the end of the girth calculation point sequence G'. If G' only contains p0, skip it. Otherwise, set G[i] = G'.back and delete G'.back.

[0121] A3, after completing the traversal, add G.back to G'.

[0122] After introducing the process of determining the vertex set S for calculating the horizontal girth and the specific process of convex hull processing, the following describes the calculation process of the girth:

[0123] B1, determine the center of mass G” of G or G’;

[0124] B2, assuming the number of segments is n seg , and the number of points in G or G' is greater than the number of segments n seg , then calculate the arc tangent value of each point in G or G' according to the formula atan2(G[i].z–G”.z,G[i].xG”.x) and sort them from small to large; otherwise, do not calculate the girth;

[0125] B3, from the point v with the smallest inverse tangent value i Start the calculation, assuming that the current point in G or G' is v i , the next point of the current point is v i+1 , calculate the angle ∠v i+1 G"v i , if the angle ∠v i+1 G"v i If the angle is greater than the preset maximum angle max_angle, the girth of G or G' is not calculated, and the angle ∠v i+1 G"v i Accumulate to accum_angle. If the number of segments n is not set seg Or accum_angle is greater than 2π / n seg , and the angle ∠v i+1 G"v i If it is greater than the preset minimum angle min_angle, v i v i+1 The length of the segment is added to the girth value. seg , minus the specified amount of 2π / n seg Make accum_angle in [0, 2π / n seg ) range. This process continues until each point in the sorted China is looped and the final girth value is obtained.

[0126] 1. Chest measurement

[0127] Based on the segmented human body mesh model and point cloud data, determine the armpit vertex set C on both sides, and use the difference between the minimum y value in the vertex set C and 50mm as the second direction coordinate value to search for the lower limit value y down , y up As the second direction coordinate value, search for the upper limit value, and y up =y min +5 / 6*h–50mm, where h is the height of the human body determined based on the human body mesh model, and y min is the minimum coordinate value of the second direction determined according to the human body mesh model.

[0128] Starting from the perineum, traverse upwards and set the y value at y down and y up The points between y and y can be traversed to the right and added to the vertex set V. Alternatively, among all the vertices of the human mesh model, determine the y value and y up Close, the x value is close to 0, and the vertex that can be traversed to the right and downward is traversed. After continuous downward traversal, the y value is in y down and y up The points between them that can be traversed to the right are added to the vertex set V.

[0129] Then, each vertex in the vertex set V is used as the starting point in the vertex set S to calculate the girth value. Here, a constraint condition can be added: non-arm segmentation points are added to the girth point search range. next The user-set threshold for correcting the y value can be 0mm, 5mm, or -5mm. The convex hull algorithm can be omitted when calculating chest circumference, and max_angle can be π / 60. The maximum calculated circumference value is used as the chest circumference value.

[0130] 2. Waist measurement

[0131] y up As the second direction coordinate value search upper limit value, y down As the second direction coordinate value, search for the lower limit value, and y up =y min +0.7494·h,y down =y min +0.5881·h.

[0132] Starting from the perineum, traverse upwards and set the y value at y down and y up The points between and can be traversed to the right are added to the vertex set V. Then each vertex in the vertex set V is used as the starting point in the vertex set S above to calculate the girth value. next The user-set threshold for correcting the y-value can be 0mm, 5mm, or -5mm. The convex hull algorithm can be omitted during waist circumference calculation, and max_angle can be π / 60. The minimum calculated girth value is used as the waist circumference value.

[0133] 3. Hip circumference measurement

[0134] y up As the second direction coordinate value search upper limit value, y down As the second direction coordinate value, search for the lower limit value, and y up =y min +0.5881·h,y down =y min +0.6764·h-343.6362mm.

[0135] Starting from the perineum, traverse upwards and set the y value at y down and y up The points between and can be traversed to the right are added to the vertex set V. Then each vertex in the vertex set V is used as the starting point in the vertex set S above to calculate the girth value.next The user-set threshold for correcting the y-value can be 0mm, 5mm, or -5mm. The convex hull algorithm with a preset angle threshold of π / 4 can be used to calculate the hip circumference, and max_angle can be π / 60. The maximum calculated girth value is used as the hip circumference.

[0136] 4. Thigh circumference measurement

[0137] y up As the second direction coordinate value, search for the upper limit value, and the y value of the perineum point as the second direction coordinate value to search for the lower limit value y down , and y up =y min +0.6764·h-343.6362mm.

[0138] Take the left thigh circumference as an example. Start traversing upward from the vertex with the largest y value in the left leg segmentation point set, which can be traversed to the right and upward, and find the vertex with the largest y value in y. down and y up The points between and can be traversed to the right are added to the vertex set V. Then each vertex in the vertex set V is used as the starting point in the vertex set S above to calculate the girth value. Here, a constraint condition can be added: points with smaller x values than the perineum point are added to the girth point search range. next The user-defined threshold for correcting the y-value can be 0mm, 5mm, or -5mm. The convex hull algorithm can be omitted when calculating the thigh circumference, and max_angle can be π / 60. The maximum calculated circumference value is used as the hip circumference.

[0139] like Figure 8 FIG. 1 is a flow chart of a measurement method based on a human body mesh model and point cloud data according to an embodiment of the present application, which specifically includes the following steps:

[0140] S801, determining a first feature point based on a human body mesh model, and determining a second feature point based on the human body mesh model and point cloud data; wherein the first feature point is a point corresponding to a first specified position on the human body; and the second feature point is a point corresponding to a second specified position on the human body;

[0141] S802: Measure the human body using the first feature point and / or the second feature point.

[0142] The present application uses the human body mesh model and point cloud data to respectively determine the first feature point and the second feature point, which can avoid the problem of data missing when the feature point is determined only by the human body mesh model. In addition, the present application uses the first feature point and / or the second feature point to measure the human body, which can avoid the problem of inaccurate measurement due to data missing.

[0143] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0144] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A measurement method based on a human body mesh model and point cloud data, characterized in that: The method comprises: Determining a first feature point based on the human body mesh model, and determining a second feature point based on the human body mesh model and the point cloud data; wherein the first feature point is a point corresponding to a first specified position of the human body; and the second feature point is a point corresponding to a second specified position of the human body; Measuring the human body using the first feature point and / or the second feature point; The coordinates of each vertex in the human body mesh model include at least a first direction coordinate value and a second direction coordinate value, and when the first feature point is a perineum point, determining the first feature point based on the human body mesh model includes: Sort the second-direction coordinate values of each vertex from small to large. If the second-direction coordinate values of the vertices are the same, sort the vertices from small to large according to the first-direction coordinate values. Filter the second-direction coordinate value of each vertex based on the maximum second-direction coordinate value, the minimum second-direction coordinate value, and the first specified rule to obtain a first vertex set corresponding to the second-direction coordinate value. Traverse rightward from the first vertex with the smallest coordinate value in the second direction in the first vertex set. After the traversal is completed, record the maximum coordinate value and the minimum coordinate value in the first direction of the current layer as the first value set, and continue traversing rightward from the vertex before the first vertex. After the traversal is completed, if the maximum coordinate value and the minimum coordinate value in the first direction of the current layer meet the second specified rule with the maximum coordinate value and the minimum coordinate value in the first direction of the previous layer, then the first value set is updated according to the new maximum coordinate value and the new minimum coordinate value in the first direction; if the maximum coordinate value and the minimum coordinate value in the first direction of the current layer meet the third specified rule with the maximum coordinate value and the minimum coordinate value in the first direction of the previous layer, then the traversal is terminated; Obtain the ninth vertex corresponding to the second direction coordinate value of the layer where the traversal ends, traverse leftward and rightward from the ninth vertex respectively until the traversal ends, mark each traversed vertex and the vertices within the specified neighborhood of each vertex as the vertex of the left leg, and determine the vertex with the largest first direction coordinate value of the layer where the traversal ends as the second vertex; Traverse all vertices in the first vertex set except the vertex of the left leg once, and mark them as vertices of the right leg, and determine the vertex with the smallest coordinate value in the first direction at the layer where the traversal ends as the third vertex; The perineum point is determined based on the second vertex and the third vertex.

2. The method according to claim 1, characterized in that The determining the perineum point based on the second vertex and the third vertex includes: Obtaining a second vertex set based on vertices within a designated neighborhood of the second vertex and vertices within a designated neighborhood of the third vertex; and determining a first threshold range based on the first direction coordinate value of the second vertex and the first direction coordinate value of the third vertex; The fourth vertex in the second vertex set whose coordinate value in the first direction is within the first threshold range and whose coordinate value in the second direction is the smallest and which is traversed upwards and has existing vertices is determined as the perineum point.

3. The method according to claim 2, characterized in that The method further comprises: An angle between a line connecting the fourth vertex and vertices traversed upward from the fourth vertex and a line connecting the fourth vertex and vertices traversed downward from the fourth vertex is smaller than a second threshold.

4. The method according to claim 1, wherein The human body mesh model and the vertices in the point cloud data are in one-to-one correspondence, and when the second feature points are left shoulder points and right shoulder points, determining the second feature points based on the human body mesh model and the point cloud data includes: Determining a left neck point and a right neck point in the point cloud data based on a preset head height, and determining a fifth vertex with a minimum coordinate value in the first direction and a sixth vertex with a maximum coordinate value in the first direction based on the human body mesh model; Based on the specified number of rows in which the left neck point and the right neck point are located, the left shoulder point and the right shoulder point of each row are determined, and the seventh vertex with the maximum angle between the lines connecting the left neck point and the fifth vertex in the point cloud data is determined as the final left shoulder point, and the eighth vertex with the maximum angle between the lines connecting the right neck point and the sixth vertex in the point cloud data is determined as the final right shoulder point.

5. The method according to claim 4, characterized in that When the point cloud data is single-view point cloud data and there are multiple point cloud data, after determining the seventh vertex and the eighth vertex in each point cloud data, the final left shoulder point and the final right shoulder point are determined by the following method: The plurality of point cloud data are subjected to point cloud stitching, and during the stitching process, the final left shoulder point and the final right shoulder point are determined based on the plurality of the seventh vertices and the plurality of the eighth vertices.

6. The method according to claim 1, characterized in that The measuring of the human body by using the first feature point and / or the second feature point includes: The first feature points are used to measure leg length, chest circumference, waist circumference, hip circumference and thigh circumference; the second feature points are used to measure arm length and shoulder width.

7. A measurement device based on a human body mesh model and point cloud data, characterized in that: include: processor and display; The display is used to display a user operation interface; The processor is configured to execute: Determining a first feature point based on the human body mesh model, and determining a second feature point based on the human body mesh model and the point cloud data; wherein the first feature point is a point corresponding to a first specified position of the human body; and the second feature point is a point corresponding to a second specified position of the human body; Measuring the human body using the first feature point and / or the second feature point; The coordinates of each vertex in the human body mesh model include at least a first direction coordinate value and a second direction coordinate value, and when the first feature point is the perineum, the processor is configured to determine the first feature point based on the human body mesh model, specifically configured to execute: Sort the second-direction coordinate values of each vertex from small to large. If the second-direction coordinate values of the vertices are the same, sort the vertices from small to large according to the first-direction coordinate values. Filter the second-direction coordinate value of each vertex based on the maximum second-direction coordinate value, the minimum second-direction coordinate value, and the first specified rule to obtain a first vertex set corresponding to the second-direction coordinate value. Traverse rightward from the first vertex with the smallest coordinate value in the second direction in the first vertex set. After the traversal is completed, record the maximum coordinate value and the minimum coordinate value in the first direction of the current layer as the first value set, and continue traversing rightward from the vertex before the first vertex. After the traversal is completed, if the maximum coordinate value and the minimum coordinate value in the first direction of the current layer meet the second specified rule with the maximum coordinate value and the minimum coordinate value in the first direction of the previous layer, then the first value set is updated according to the new maximum coordinate value and the new minimum coordinate value in the first direction; if the maximum coordinate value and the minimum coordinate value in the first direction of the current layer meet the third specified rule with the maximum coordinate value and the minimum coordinate value in the first direction of the previous layer, then the traversal is terminated; Obtain the ninth vertex corresponding to the second direction coordinate value of the layer where the traversal ends, traverse leftward and rightward from the ninth vertex respectively until the traversal ends, mark each traversed vertex and the vertices within the specified neighborhood of each vertex as the vertex of the left leg, and determine the vertex with the largest first direction coordinate value of the layer where the traversal ends as the second vertex; Traverse all vertices in the first vertex set except the vertex of the left leg once, and mark them as vertices of the right leg, and determine the vertex with the smallest coordinate value in the first direction at the layer where the traversal ends as the third vertex; The perineum point is determined based on the second vertex and the third vertex.

8. The device according to claim 7, characterized in that When the processor is configured to determine the perineum point based on the second vertex and the third vertex, the processor is specifically configured to execute: Obtaining a second vertex set based on vertices within a designated neighborhood of the second vertex and vertices within a designated neighborhood of the third vertex; and determining a first threshold range based on the first direction coordinate value of the second vertex and the first direction coordinate value of the third vertex; The fourth vertex in the second vertex set whose coordinate value in the first direction is within the first threshold range and whose coordinate value in the second direction is the smallest and which is traversed upwards and has existing vertices is determined as the perineum point.

Images