3D skin measuring method based on high-precision measurement of concave-convex degree area of human face

Abstract

The invention relates to the technical field of medical cosmetology and computer vision, in particular to a 3D skin measurement method based on high-precision measurement of a face concave-convex degree region, and the method comprises the following steps: arranging a structured light camera to collect a face RGB image and depth data, and carrying out the filtering and denoising to generate a smooth point cloud; detecting an RGB image based on FaceMesh to obtain face key points, and constructing a multi-region segmentation algorithm; segmenting an RGB local area and converting the RGB local area into point cloud data; iDW interpolation is carried out to generate a dense point cloud, and a two-dimensional contour line is extracted through a Marking Square algorithm; and mapping the color of the matched color card to an image area to generate a contour map. According to the method, the face recognition model is constructed through the FaceMesh architecture, accurate positioning of multiple face types is adapted, two-dimensional contour mapping is adopted to three-dimensional contour mapping, data are intercepted, local processing is combined, the operation load is reduced by 60%, and efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the fields of medical aesthetics and computer vision technology, and particularly relates to a 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face. Background Art

[0002] With the development of social economy and the wide application of medical imaging technology, the demand for quantitative analysis of facial features in the fields of medical aesthetics and skin health management shows a significant growth trend. Traditional three-dimensional depth detection technologies mainly include the following two implementation paradigms:

[0003] (1) Two-dimensional image analysis method

[0004] This method constructs a feature recognition model based on the color difference of planar images, and mainly relies on the RGB or HSV color space for facial light and dark region segmentation. Although it is effective in detecting features with significant chromaticity contrast such as wrinkles, there are still obvious technical limitations in practical applications:

[0005] ① It is significantly restricted by environmental lighting conditions, and the lighting error rate in the actual shooting scene can exceed 30%.

[0006] ② It can only obtain two-dimensional plane projection data and is difficult to accurately represent the three-dimensional curvature features of the face.

[0007] ③ The detection sensitivity for large-area high-light flat regions (such as apple muscles, drooping corners of the mouth, etc.) is insufficient.

[0008] (2) Three-dimensional global modeling method

[0009] The structured light scanning technology is used to obtain high-density point cloud data of the whole face (5 - 8 points / mm 2 ), and a three-dimensional facial model is generated through a triangular mesh reconstruction algorithm. Although the data dimension is improved, there are still the following technical bottlenecks:

[0010] ① The data scale generated by a single scan exceeds one million levels (> 1×10^6 data points).

[0011] ② The extraction of local features requires global modeling as a prerequisite, resulting in a time cost increase of more than 200%.

[0012] There is a significant trade-off contradiction between "detection accuracy" and "processing efficiency" in the existing technology system: the average processing time of the two-dimensional image method < 3 seconds but the accuracy is limited, and the three-dimensional modeling method has higher accuracy but the time consumption > 30 seconds. This technical bottleneck severely restricts the feasibility of related devices in clinical real-time detection scenarios.

[0013] Existing technical solutions:

[0014] The current mainstream technical solutions mainly include two implementation paths:

[0015] ① Facial contour reconstruction method based on two-dimensional image processing.

[0016] ② Regional feature extraction method based on full-face three-dimensional modeling.

[0017] Defects of the prior art:

[0018] ① The two-dimensional image processing method is vulnerable to environmental light interference, with a relatively high detail accuracy error rate (>15%), and can only identify dominant texture features larger than 2 mm, making it difficult to meet medical-grade detection standards.

[0019] ② The three-dimensional modeling method adopts the technical route of "overall modeling - local interception", and the single feature extraction takes more than 30 seconds, resulting in too long waiting time in real-time application scenarios, seriously restricting the feasibility of clinical real-time detection. Summary of the invention

[0020] In view of the deficiencies of the prior art, the present invention discloses a 3D skin measurement method based on high-precision measurement of the concave and convex regions of the human face, which realizes the omission-free acquisition of human face point cloud data and establishes an accurate mapping between two-dimensional images and three-dimensional coordinates. A smart region segmentation algorithm is constructed to adapt to different face shapes and enhance the generalization ability of contour localization. A sparse point cloud interpolation algorithm is developed to ensure the stability of the face measurement accuracy based on physical constraints. An efficient local feature processing pipeline is established to compress the time of different depth feature algorithms to within 10 seconds.

[0021] The present invention is realized through the following technical solutions:

[0022] The present invention provides a 3D skin measurement method based on high-precision measurement of the concave and convex regions of the human face, characterized by including the following steps:

[0023] Structured light cameras are arranged in three directions to collect RGB images and depth data of the human face, and after filtering and denoising processing, high-precision smooth point clouds are generated;

[0024] Detect the RGB image based on the FaceMesh model, obtain facial key points, and construct a multi-region segmentation algorithm in combination with B-spline curves;

[0025] Call the corresponding algorithm to segment the local RGB region according to the index effect, and find the corresponding point cloud data of the local region using the principle of the same pixel coordinate system;

[0026] Use the IDW interpolation method to process the point cloud data, generate dense three-dimensional point clouds, and then obtain multi-level contour lines through the Marching Squares algorithm and convert them into two-dimensional contour lines;

[0027] Match the color of the color card according to the contour line depth, map it to the index region of the RGB image, and generate a concave and convex contour map.

[0028] Furthermore, the filtering and denoising process adopts a point cloud statistical filtering method based on the spatial neighborhood, sets the neighborhood range and the point cloud standard deviation threshold, and removes the outliers in the original face point cloud data that significantly deviate from the local point cloud density statistical characteristics.

[0029] Furthermore, the specific steps of the statistical filtering method are as follows:

[0030] Neighborhood calculation: For each point P i , search for its k nearest neighbor points, where j is a positive integer and N is the global points;

[0031] Statistic calculation: Calculate the coordinate mean of the points in the neighborhood and the standard deviation Calculate the coordinate mean of the global points and the standard deviation

[0032] Threshold filtering: Set the coordinate threshold σ of the points in the neighborhood th = σ stdmul × σ stddev Set σ stdmul = 0.8; If σ i > σ th , then determine that P i is an outlier to be removed; Then use the radius filtering algorithm to perform outlier removal processing on the remaining point cloud again.

[0033] Furthermore, use the radius filtering algorithm to perform outlier removal processing on the remaining point cloud again, and the specific steps are as follows:

[0034] Neighborhood density check: For the point cloud after statistical filtering, traverse each point p' i , count the number of neighborhood points n within the radius r i ;

[0035] Density threshold removal: Set the minimum number of points n min , if n i < n min , remove p' i , to obtain the secondary filtered point cloud P″;

[0036] Then smooth it by moving least squares, and on the premise of maintaining the curvature characteristics of the point cloud, ensure that the key contour information of the face is not lost during smoothing.

[0037] Furthermore, the specific steps to ensure that the key contour information of the face is not lost during smoothing are as follows:

[0038] Local surface fitting: For each point P″ in P″ k , construct a local coordinate system within its neighborhood with a radius R = 6 mm, and use weighted least squares to fit the surface: Adopt Gaussian weights where h = 3mm is the attenuation coefficient, and the quadratic polynomial basis B(x, y, z) = [1, x, y, z, x 2 , y 2 , z 2 , xy, yz, zx];

[0039] Projection smoothing: Project P″ k onto the fitted surface to obtain the smoothed coordinates Update the point cloud to

[0040] After filtering, denoising, and smoothing operations, the optimized face point cloud data is obtained.

[0041] Furthermore, the steps of constructing a multi-region segmentation algorithm by combining B-spline curves are as follows:

[0042] Obtain the coordinates of the positioning key points of the face: Put the RGB captured by the structured light into the FaceMesh face 2D key point recognition model for recognition, and obtain the normalized coordinates of the face key points;

[0043] Convert the coordinates of the positioning key points to the corresponding coordinates in the original face image space: Convert the coordinate points output by the model to the coordinates of the original captured face image, and select the feature point sets for constructing different regions;

[0044] Obtain the smooth contour region from the feature point set by the B-spline curve method: First, obtain the number N of feature points and the order n of the B-spline according to the feature point set of the selected region, fix the step size t, and traverse the factorial of i. First, calculate the combination number Then, combine the obtained combination number with the step size and calculate the weight of each control point through the Bernstein basis function Then, through the curve formula Obtain the position coordinates of the curve points corresponding to each point on the B-spline curve; Finally, store each point in an array as the contour line coordinate set of the selected region;

[0045] Take the minimum bounding rectangle of the contour region from the contour line coordinate set by the boundingRect method, and perform region segmentation through the coordinates of the bounding rectangle.

[0046] Furthermore, in order to improve the operation efficiency and segment accurate point cloud data, use the feature region algorithm to crop the original captured RGB image to obtain the regional RGB cropped image. Since the face point cloud data and the RGB image are in the same pixel coordinate system, judge the concave-convex features of the region through the depth attribute of the point cloud, convert the depth information in the optimized point cloud data into a depth feature image, and then segment the depth feature image through the feature region segmentation algorithm to obtain the point cloud depth information of the feature region, so as to perform segmentation of the point cloud data.

[0047] Furthermore, an interpolation method based on inverse distance weighted interpolation is used. According to the data values and spatial position relationships of the known point clouds in the target area, new data points are inserted by weighted summation with the inverse distance, making the point clouds more densely and evenly distributed in the target area, filling the data missing parts caused by measurement resolution limitations or occlusion reasons, and further enriching the facial concave and convex detail information contained in the point cloud data. The specific steps are as follows:

[0048] Determine the interpolation point and search radius: Divide the target area into uniform grids, and the center point of each grid is used as the point to be interpolated P(x p , y p ); Taking P as the center, calculate its distance d to the nearest 3 points, and use the K-nearest neighbor search to quickly calculate the distance to the nearest point;

[0049] Inverse distance weight calculation:

[0050] Calculate the Euclidean distance from P to each point Calculate the weight according to the inverse distance weight formula

[0051] Depth value interpolation calculation: Perform weighted summation on the interpolation depth Zp of P; If a point coincides with P

[0052] Directly use the depth value of this point to avoid the denominator being zero.

[0053] Furthermore, the generation of concave and convex contour maps is as follows:

[0054] For the points without point cloud depth data in the target area, calculate the distance from this point to all the points with point cloud depth data nearby, and determine the depth of this point according to the distance from this point to all the points with point cloud depth data nearby and their depth data;

[0055] Calculate the maximum and minimum values of the point cloud data in the target area after interpolation, and calculate all the contour values C in this area according to the given skin depth distance; Divide the obtained depth data into mxn regular square grids, and each vertex (i, j) corresponds to a scalar value f(i, j); For each square grid cell, consider the relationship between the scalar values of its four vertices and the given contour value C′ (C′∈C); Each cell is composed of four vertices, numbered in clockwise order; For the four vertices of each grid cell, mark the status as 1 (value≥C′) or 0 (value<C′), generating a 4-bit binary status code, ranging from 0 to 15; According to the status code, determine which edges are crossed by the contour line through a predefined look-up table; If the statuses of the two ends of an edge are different, there is an intersection point; For the two end points (x1, y1) and (x2, y2) of the edge, its interpolation parameter The intersection point coordinates are (x1 + t(x2 - x1), y1 + t(y2 - y1));

[0056] Connect the points within the same depth range into lines, and gradually construct a contour map that can clearly and intuitively display the concave and convex features of the human face. These contour lines depict the concave and convex degrees and changing trends at different positions on the human face surface.

[0057] Furthermore, for the dense and smooth point cloud data in the ROI region, draw a depth gradient contour through the depth distance of the point cloud, then convert the depth gradient contour into an RGB depth contour map, and finally render it onto the 3D model.

[0058] The beneficial effects of the present invention are as follows:

[0059] Based on the FaceMesh human face model architecture, the present invention constructs a facial concave and convex region recognition model through a key point positioning algorithm, which can adapt to various facial morphological features such as round faces, thin faces, and oval faces, realize flexible and accurate positioning and recognition, and ensure that the concave and convex regions corresponding to various facial features can be accurately extracted and presented.

[0060] The present invention adopts a technical path of mapping a two-dimensional contour region to a three-dimensional space and intercepting the corresponding three-dimensional data, and performing local processing in combination with an interpolation algorithm, effectively avoiding the traditional overall point cloud operation mode of the human face. Through the strategy conversion from global operation to local operation, the system operation load is reduced by about 60%, significantly improving the operation efficiency. Description of the Drawings

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0062] Figure 1 It is a step diagram of a 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face;

[0063] Figure 2 It is a flow block diagram of a 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face;

[0064] Figure 3 It is a schematic diagram of converting the coordinate points obtained from the model to the coordinate of the original captured human face picture;

[0065] Figure 4 It is to convert the depth information in the optimized point cloud data into a depth feature map. Detailed Embodiments

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0067] In one embodiment, as shown in Figure 1 a 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face is provided, which is characterized by including the following steps:

[0068] Structured light cameras are arranged in three directions to collect RGB images and depth data of the human face, and after filtering and denoising processing, high-precision smooth point clouds are generated;

[0069] Based on the FaceMesh model, the RGB image is detected to obtain facial key points, and a multi-region segmentation algorithm is constructed in combination with B-spline curves;

[0070] According to the index effect, the corresponding algorithm is called to segment the local RGB region, and the corresponding point cloud data of the local region is found using the principle of the same pixel coordinate system;

[0071] The IDW interpolation method is used to process the point cloud data to generate a dense three-dimensional point cloud, and then the Marching Squares algorithm is used to obtain multi-level isocontours and convert them into two-dimensional contour lines;

[0072] The color of the color card is matched according to the depth of the isocontour and mapped to the index region of the RGB image to generate a concave and convex contour map.

[0073] This embodiment proposes a three-dimensional human face data acquisition and processing method that combines three-dimensional vision and structured light. First, multi-view RGB images and registered depth point cloud data are obtained through a synchronous acquisition system to establish a unified pixel coordinate system. 468 human face key point coordinates are extracted based on the FaceMesh human face key point detection model to construct a multi-region segmentation algorithm. Multilevel optimization processing is performed on the original depth point cloud: statistical filtering (Statistical Outlier Removal, SOR) is used to eliminate outlier noise points, radius filtering (Radius Outlier Removal, ROR) is applied for spatial consistency processing, and surface smoothing reconstruction is achieved through moving least squares (Moving Least Squares, MLS) to finally obtain optimized three-dimensional point cloud data.

[0074] Select the optimal segmentation algorithm according to clinical evaluation indicators such as the malar region and drooping corners of the mouth, extract the ROI contour of the RGB image of the target area, and synchronously map it to the depth data space to obtain corresponding three-dimensional features. Use the thin plate spline interpolation algorithm to densify the sparse point cloud and construct a high-resolution smooth surface. Based on the depth gradient field analysis technology, convert the three-dimensional geometric features into a two-dimensional depth gradient map, and generate an RGB depth feature map through pseudo-color mapping. Finally, integrate the multi-modal feature data into a three-dimensional face model for visual presentation.

[0075] The specific implementation process of this embodiment includes the following steps:

[0076] Three-dimensional structured light point cloud acquisition and optimization: Set three groups of structured light camera devices to synchronously capture the face from the left, middle, and right directions respectively, and obtain three groups of RGB images and corresponding depth data sets. Process the depth data through point cloud filtering and denoising to generate a high-precision smooth three-dimensional point cloud model.

[0077] Face key point feature extraction and regional segmentation algorithm construction: Use the FaceMesh face key point detection model to analyze the RGB image and locate 468 feature coordinate points. According to the facial anatomical features, intelligently select a set of specific area coordinate points, and establish a multi-modal face area segmentation algorithm system in combination with the Bezier curve algorithm.

[0078] Point cloud data regional segmentation: According to the requirements of target detection indicators, call the corresponding regional segmentation algorithm to obtain the target RGB sub-region. Based on the consistency of the pixel coordinate systems of the face point cloud data and the RGB image, realize the accurate mapping and cutting of the RGB sub-region to the three-dimensional point cloud data.

[0079] Point cloud contour generation: Perform inverse distance weighting (IDW) processing on the segmented face area point cloud to generate a high-density smooth three-dimensional point cloud. Use the marching squares algorithm to extract multi-level contour lines, and obtain the two-dimensional face contour feature lines through coordinate transformation.

[0080] Contour visualization presentation: Establish a color depth corresponding chromatographic system, map the depth values of the two-dimensional contour feature lines to the corresponding areas of the RGB image, and generate a multi-color contour map with depth gradient representation.

[0081] In one embodiment, the implementation process of the three-dimensional structured light point cloud acquisition and optimization method is as follows:

[0082] As the basic data source for three-dimensional reconstruction, the accuracy and quality of the point cloud data directly determine the theoretical upper limit of the subsequent analysis, modeling, and application effects. The specific implementation steps of the three-dimensional structured light point cloud acquisition and optimization technical solution adopted in this embodiment are as follows:

[0083] The acquisition of the original face point cloud data is carried out by using a three - camera structured light imaging system for multi - spectral synchronous acquisition to obtain the phase grating image and the RGB texture image of the face area. Through the phase unwrapping algorithm and the three - dimensional coordinate calculation model, the collected grating phase information is converted into an initial three - dimensional point cloud data set.

[0084] The optimization and processing of the face point cloud data. For the original point cloud data set P = {p i |i = 1, 2,..., N}, first implement the spatial statistical filtering algorithm based on the k - neighborhood. This algorithm constructs a spatial distribution feature matrix of the point cloud and establishes a dynamic threshold setting mechanism to effectively remove the spatial outliers caused by surface reflection characteristics, ambient light interference, or sensor random noise. Specifically, for each sampling point, calculate the mean and standard deviation of the point cloud distribution within its k - neighborhood. When the deviation of the target point from the neighborhood statistical characteristics exceeds the preset threshold, it is determined as an abnormal point and removed.

[0085] In this embodiment, the statistical filtering method is further optimized. The specific implementation steps are as follows:

[0086] Neighborhood analysis: For each sampling point Pi in the point cloud data, use the k - nearest neighbor algorithm to establish a local spatial neighborhood. In this method, k = 50 is set (this parameter can be adjusted adaptively according to the point cloud density).

[0087] Statistic modeling: Based on the established local neighborhood, calculate the mean of the point cloud coordinates within each neighborhood and the standard deviation Simultaneously calculate the mean of the global point cloud coordinates and the standard deviation Construct a multi - scale statistical feature space.

[0088] Threshold decision: Set the threshold condition of the neighborhood coordinate statistic σ th = σ stdmul ×σ stddev In this scheme, σ stdmul = 0.8 is used as the discrimination threshold (this parameter can be dynamically adjusted according to the distribution characteristics of the point cloud outliers). When the condition σ i > σ th is satisfied, Pi is determined as a spatial outlier and the removal operation is performed.

[0089] In this embodiment, to optimize the denoising effect, this study introduces a radius filtering algorithm on the basis of statistical filtering to construct a two - level outlier removal system, and significantly suppresses the influence of noise interference on the point cloud quality through a multi - level filtering mechanism. The algorithm implementation process is as follows:

[0090] Neighborhood density analysis: For the point cloud data after statistical filtering, calculate the number of points nip' within a spherical neighborhood with a radius r = 10 mm (which can be dynamically adjusted according to the outlier distribution) for each point. i , to evaluate the local point density distribution characteristics.

[0091] Density-based threshold screening: Set the minimum neighborhood point number threshold nmin = 80. When nip' i < nmin, determine that this point is a noise point and eliminate it. Finally, obtain the optimized secondary filtered point cloud P″.

[0092] In this embodiment, then apply the moving least squares method for feature-preserving smoothing processing. On the basis of effectively maintaining the curvature characteristics of the original point cloud, achieve the complete retention of the key contour information of the human face. The algorithm execution steps include:

[0093] Local surface fitting: For each sampling point P″ in P″ k , establish a local coordinate system within its neighborhood with a radius R = 6 mm, and use the weighted least squares method for surface fitting: Based on the Gaussian weight function (where the attenuation coefficient h = 3 mm), select the quadratic polynomial basis function B(x, y, z) = [1, x, y, z, x2, y2, z2, xy, yz, zx] to construct the surface model.

[0094] Projection smoothing processing: Orthogonally project the original coordinates P″ k onto the fitted surface to obtain the smoothed coordinates Through iterative update to achieve the optimized reconstruction of the point cloud data .

[0095] Through the above multi-level filtering, denoising and smoothing processing flow, finally obtain an optimized human face point cloud dataset, whose geometric accuracy and data stability are significantly improved, and can more accurately represent the morphological characteristics of the human face surface.

[0096] The implementation steps of the human face key point feature extraction and region segmentation algorithm proposed in this embodiment are as follows:

[0097] To achieve the accurate dynamic positioning of the human face feature region, it is necessary to perform facial feature calibration based on the human face model. Through the key point positioning coordinates, use the Bezier curve algorithm to construct the contour of a specific region, and then establish a multi-region segmentation model. The specific implementation process is as follows:

[0098] Extraction of human face key point coordinates: Use the FaceMesh two-dimensional key point recognition model to process the RGB image collected by the structured light system, obtain the normalized coordinate matrix of the human face containing 468 feature points, and establish the facial feature topology structure.

[0099] Coordinate space transformation and feature point screening, mapping the normalized coordinates output by the model to the original image space coordinate system (as shown in the appendix Figure 3 ), and selecting the topological feature point sets of each functional area based on facial anatomical features to establish a mathematical basis for subsequent area segmentation.

[0100] Bézier curve contour modeling, setting the corresponding relationship between the order n of the Bézier curve and the number N of feature points, performing uniform sampling with a fixed step size t = 0.02, and obtaining a sequence of 51 contour points;

[0101] Calculating the combination number Combined with the Bernstein basis function Solving the weights of each control point;

[0102] Through the parametric equation Iteratively calculating the coordinates of the sampling points to generate a smooth closed curve and forming a coordinate set of the area contour.

[0103] Implementation of the area segmentation algorithm: Applying the boundingRect algorithm to extract the minimum bounding rectangle of the contour and establishing an image segmentation mask based on the rectangle parameters. This algorithm framework can be extended and applied to multi-area parallel segmentation, and the adaptive division of different anatomical areas can be achieved by adjusting the feature point set..

[0104] The implementation steps of the point cloud data segmentation method in this embodiment are as follows:

[0105] Based on the optimized point cloud data and the feature area segmentation algorithm obtained in the foregoing steps, in order to improve the operation efficiency and achieve high-precision point cloud segmentation, this embodiment adopts a dual processing mechanism of the feature area segmentation algorithm: First, perform area cropping on the original RGB image to generate an area RGB cropped image; then, in view of the fact that the face point cloud data and the RGB image have a unified pixel coordinate system and the concave-convex features of the area need to be analyzed through depth attributes, this study first converts the depth information in the optimized point cloud data into a depth feature map (as shown in the appendix Figure 4 ); Subsequently, the depth feature map is analyzed by the feature area segmentation algorithm to accurately extract the point cloud depth information of the feature area, and finally the three-dimensional space segmentation of the point cloud data is realized. This technical path effectively improves the geometric accuracy of depth feature extraction while maintaining data relevance.

[0106] In one embodiment, the contour lines of the point cloud data are drawn as follows:

[0107] Optimized point cloud data often exhibits uneven spatial distribution and sparse features in local areas. If it is directly applied to represent the concavity and convexity depth of the face, it will lead to a significant reduction in the accuracy of contour drawing. Therefore, in order to obtain high-precision analysis results, interpolation filling processing needs to be performed on the optimized point cloud. Using the inverse distance weighted interpolation method, based on the spatial topological relationship and numerical characteristics of the known point cloud in the target area, weighted interpolation is implemented through the inverse ratio of spatial distance, and new data points with dense and uniform distribution can be generated in the target area. This method effectively compensates for the data loss caused by the limitation of measurement resolution and occlusion effect, significantly enhances the expression ability of point cloud data for the subtle undulation features of the face surface, and lays a complete data foundation for subsequent high-precision contour drawing.

[0108] Furthermore, the specific implementation process is as follows:

[0109] Interpolation point positioning and neighborhood definition: Discretize the target area into a uniform grid array (recommended size 1mm×1mm), and set the center point of each grid as the interpolation point P(x p , y p ). Based on the K-nearest neighbor algorithm (KNN), retrieve the three nearest neighbors of the interpolation point P, and calculate its maximum neighborhood radius d.

[0110] Inverse distance weight coefficient solution:

[0111] Establish the Euclidean distance matrix between the interpolation point P and the neighborhood points According to the inverse distance weight formula Complete the weight assignment.

[0112] Depth value interpolation calculation:

[0113] Perform weighted interpolation operation Obtain the depth value Zp. When there is a neighborhood point with the same spatial coordinates as P (i.e., dj = 0), directly inherit the depth value of this point to avoid the zero denominator problem.

[0114] When performing concave-convex contour drawing in this embodiment, based on the final point cloud data set obtained from the aforementioned interpolation optimization processing, the concave-convex contour representing the surface topography is generated through the depth gradient contour mapping algorithm.

[0115] The implementation path of the concave-convex contour generation technology is as follows:

[0116] For the coordinate points of the missing depth data in the target area, by calculating the Euclidean distance between this point and the adjacent valid point cloud data points, combined with the corresponding depth observation values, perform spatial weighted interpolation operations to finally deduce the depth parameters of this point.

[0117] After the interpolation process is completed, first calculate the extreme value parameters of the point cloud data in the target area, and calculate the contour line sequence C according to the preset skin depth threshold. Construct the depth data set into an m×n normalized square grid array, and the vertex coordinates (i, j) correspond to the scalar function value f(i, j). For each grid cell, analyze the logical relationship between the scalar values of its four vertices and the contour threshold C′ (C′∈C). Encode in clockwise orientation order (e.g., lower left, lower right, upper right, upper left), and perform binary marking on the vertex state: 1 (scalar value ≥ C′) or 0 (scalar value < C′), thereby generating a 4-bit binary feature code (value space 0-15). For example, the vertex state sequence [0, 1, 1, 0] corresponds to the feature code 0110. Based on the feature code index, pre-define the edge topology table to determine the contour line crossing edge. For the edge endpoints (x1, y1) and (x2, y2) with different phase states, through the linear interpolation parameter calculate the exact intersection coordinates (x1 + t(x2 - x1), y1 + t(y2 - y1)).

[0118] Perform topological connection on the intersection points within the same depth domain, and systematically construct a three-dimensional topographic contour network with clear representation ability. This contour line system accurately quantifies and represents the change law of the face surface curvature gradient, and completely reveals the deformation characteristics and morphological distribution trends of different anatomical regions.

[0119] The finally generated contour map is transmitted to a dedicated detection device through a high-precision display interface to achieve professional-level visualization output with a standardized graphical interface. This technical solution provides visualization decision support with measurement accuracy in application scenarios such as preoperative simulation and effect evaluation in the field of medical beauty and plastic surgery, optimization design of industrial human factors engineering models, and morphological comparison of biometric recognition systems, effectively strengthening the technical reliability of scientific decision-making in related fields.

[0120] To achieve accurate registration of multi-source point cloud data, this embodiment develops an iterative closest point optimization algorithm based on feature descriptors (ICP-NDT). For the multi-viewpoint cloud data collected by the three-eye system, first extract the FPFH feature descriptors to construct a spatial correspondence matrix, and screen out valid matching pairs through two-way consistency verification. Use the singular value decomposition method to calculate the initial rigid transformation matrix [T|R], and apply an improved voxelized covariance registration strategy for non-rigid deformation compensation, and finally achieve sub-millimeter spatial alignment of the multi-viewpoint cloud.

[0121] In the point cloud fusion stage, this solution proposes a multi-resolution voxel grid fusion mechanism. The registered point cloud data is divided into voxel units of 0.5mm×0.5mm×0.5mm, and a weighted Gaussian mixture model is used to model the probability distribution of the point cloud within the unit. By establishing a spatial confidence evaluation function, the weight coefficients of each data source are dynamically adjusted to effectively eliminate data holes caused by perspective occlusion and generate a topologically complete three-dimensional human face surface model.

[0122] To further improve the reliability of feature extraction, this system introduces a multi-scale curvature analysis module. The covariance matrix method is used to calculate the principal curvature parameters, and the HKS heat kernel feature descriptor is constructed. By designing a scale adaptive selection algorithm, it automatically switches to the micron-level analysis mode in high-curvature areas such as the bridge of the nose and nasolabial folds, significantly enhancing the ability to capture fine wrinkle features. Experimental data shows that this method reduces the feature point positioning error to ±0.03mm, meeting the medical-grade measurement accuracy requirements.

[0123] Aiming at the problem of dynamic expression interference, a motion artifact elimination system based on the LSTM network is developed. By establishing a time series point cloud change prediction model, the coordinate offsets caused by physiological micro-expressions and measurement noise are separated in real time. This network adopts a two-stream architecture design. The spatial stream branch processes the geometric features of the current frame, and the temporal stream branch analyzes the motion trajectories of 10 consecutive frames. After the feature fusion layer, a corrected stable point cloud sequence is output. Clinical verification shows that this system can still maintain a feature extraction accuracy of 98.7% in the natural micro-expression state of the subjects.

[0124] In summary, based on the FaceMesh human face model architecture, this invention constructs a facial concave and convex area recognition model through a key point positioning algorithm, which can adapt to various facial morphological features such as round faces, thin faces, and oval faces, realizing flexible and accurate positioning and recognition, and ensuring that the concave and convex areas corresponding to various face type features can be accurately extracted and presented.

[0125] This invention adopts a technical path of mapping a two-dimensional contour area to three-dimensional space and intercepting the corresponding three-dimensional data, combined with an interpolation algorithm for local processing, effectively avoiding the traditional overall point cloud operation mode of human faces. Through the strategy conversion from global operation to local operation, the system operation load is reduced by about 60%, significantly improving the operation efficiency.

[0126] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face, characterized in that, It includes the following steps: The structured light cameras arranged in three directions collect the face RGB images and depth data, which are processed by filtering and denoising to generate high-precision smooth point clouds; Based on the FaceMesh model, the RGB image is detected to obtain facial key points, and a multi-region segmentation algorithm is constructed by combining with Bezier curves; According to the index effect, the corresponding algorithm is called to segment the local RGB region, and it is converted into the corresponding point cloud data by using the same pixel coordinate system; The IDW interpolation method is used to process the point cloud data to generate a dense three-dimensional point cloud, and then the Marching Squares algorithm is used to obtain multi-level contour lines and convert them into two-dimensional contour lines; According to the depth of the contour line, the color of the color card is matched and mapped to the index area of the RGB image to generate a concave-convex contour map.

2. The 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face according to claim 1, wherein In the method, the filtering and denoising process adopts a point cloud statistical filtering method based on the spatial neighborhood, sets the neighborhood range and the point cloud standard deviation threshold, and removes the outliers in the original face point cloud data that significantly deviate from the local point cloud density statistical characteristics.

3. The 3D skin measurement method based on high-precision measurement of human face concavity and convexity degree regions according to claim 2, characterized in that, In the method, the specific steps of the statistical filtering method are: Neighborhood calculation: For each point P i , search for its k nearest neighbor points, where j is a positive integer and N is the global points; Statistic calculation: Calculate the coordinate mean of points in the neighborhood and standard deviation Calculate the coordinate mean of global points and standard deviation Threshold filtering: Set the coordinate threshold σ of the points within the neighborhood th = σ stdmul × σ stddev , set σ stdmul = 0.8; if σ i > σ th , then determine that P i is an outlier to be removed; then use the radius filtering algorithm to remove outliers from the remaining point cloud again.

4. The 3D skin measurement method based on high-precision measurement of human face concavity and convexity regions according to claim 3, wherein In the method, the radius filtering algorithm is used to perform outlier removal processing on the remaining point cloud again. The specific steps are: Neighborhood density check: For the point cloud after statistical filtering, traverse each point p′ i , and count the number of neighborhood points n within the radius r i ; Density threshold rejection: Set the minimum number of points n min , if n i < n min , reject p′ i , and obtain the secondarily filtered point cloud P″; Then, it is smoothed by the moving least squares method. On the premise of maintaining the curvature characteristics of the point cloud, it is ensured that the key contour information of the face is not lost while smoothing.

5. The 3D skin measurement method based on high-precision measurement of human face concavity and convexity degree regions according to claim 4, wherein, In the method, the specific steps to ensure that the key contour information of the face is not lost while smoothing are: Local surface fitting: For each point P″ in P″ k , a local coordinate system is constructed within its neighborhood with a radius R = 6 mm, and the surface is fitted using weighted least squares: Gaussian weights are adopted where h = 3 mm is the attenuation coefficient, and the quadratic polynomial basis B(x, y, z) = [1, x, y, z, x 2 , y 2 , z 2 , xy, yz, zx]; Projection smoothing: project P″ k onto the fitted surface to obtain the smoothed coordinates Update the point cloud to After filtering, denoising and smoothing operations, the optimized face point cloud data is obtained.

6. The 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face according to claim 1, characterized in that, In the method, the steps of constructing a multi-region segmentation algorithm by combining with Bezier curves are as follows: Obtain the coordinate of the positioning key point of the face: Put the RGB captured by the structured light into the FaceMesh face 2D key point recognition model for recognition, and obtain the normalized coordinate of the face key point; Convert the coordinate of the positioning key point to the corresponding coordinate in the original face image space: Convert the coordinate points output by the model to the coordinate of the original face captured image, and select the feature point sets for constructing different regions; The set of feature points is used to obtain a smoothed contour region through the Bezier curve method: First, the number N of feature points and the order n of the Bezier curve are obtained from the set of feature points in the selected region. The fixed step size t is set, and the factorial of i is traversed. First, the combination number is calculated. Then, the weights of each control point are calculated through the Bernstein basis function by combining the obtained combination number with the step size Next, through the curve formula the position coordinates of the curve points corresponding to each point on the Bezier curve are obtained; finally, each point is stored in an array as the contour line coordinate set of the selected region. Take the minimum bounding rectangle of the contour area through the boundingRect method for the contour line coordinate set, and perform region segmentation through the coordinates of the bounding rectangle.

7. The 3D skin measurement method based on high-precision measurement of facial concave and convex degree regions according to claim 1, wherein In the method, in order to improve the operation efficiency and segment accurate point cloud data, the feature region algorithm is used to crop the original captured RGB image to obtain the regional RGB cropped image. Then, because the face point cloud data and the RGB image have the same pixel coordinate system, the concave-convex characteristics of the region are judged by the depth attribute of the point cloud. The depth information in the optimized point cloud data is converted into a depth feature image, and then the depth feature image is segmented by the feature region segmentation algorithm to obtain the point cloud depth information of the feature region, so as to achieve the segmentation of the point cloud data.

8. The 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face according to claim 1, wherein, In the method, an interpolation method based on the inverse distance weighted interpolation method is used. According to the data values and spatial position relationships of the known point clouds in the target area, new data points are inserted by weighted summation with the inverse of the distance, so that the point cloud is more densely and evenly distributed in the target area, filling the data missing part caused by measurement resolution limitations or occlusion reasons, and further enriching the face concave-convex detail information contained in the point cloud data. The specific steps are as follows: Determine the interpolation points and search radius: Divide the target area into a uniform grid, and use the center point of each grid as the interpolation point P(x p , y p ); With P as the center, calculate the distances d from it to the nearest 3 points, and use the K-nearest neighbor search to quickly calculate the distances to the nearest points; Inverse distance weighting calculation: Calculate the Euclidean distance from P to each point Calculate the weights according to the inverse distance weighting formula Depth value interpolation calculation: perform weighted summation on the interpolated depth Zp of P; if a point coincides with P Directly adopt the depth value of this point to avoid a zero denominator.

9. The 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face according to claim 1, characterized in that, In the described method, the generation of the concave-convex contour map is specifically as follows: For a point without point cloud depth data in the target area, calculate the distance from this point to all nearby points with point cloud depth data, and determine the depth of this point based on the distance from this point to all nearby points with point cloud depth data and their depth data; Calculate the maximum and minimum values of the point cloud data in the target area after interpolation, and calculate all the contour values C in this area according to the given skin depth distance; divide the obtained depth data into mxn regular square grids, and each vertex (i, j) corresponds to a scalar value f(i, j); for each square grid cell, consider the relationship between the scalar values of its four vertices and the given contour value C′ (C′∈C); each cell is composed of four vertices, numbered in clockwise order; for the four vertices of each grid cell, mark the status as 1 (value ≥ C′) or 0 (value < C′), generate a 4-bit binary status code, ranging from 0 to 15; according to the status code, determine which edges are crossed by the contour through a predefined look-up table; if the status of the two end vertices of each edge is different, there is an intersection point; for the two end points (x1, y1) and (x2, y2) of the edge, its interpolation parameter The intersection point coordinates are (x1 + t(x2 - x1), y1 + t(y2 - y1)); Connect the points within the same depth range into lines, and gradually construct a contour map that can clearly and intuitively display the concave-convex features of the human face. These contour lines depict the concave-convex degree and change trend of different positions on the human face surface.

10. The 3D skin measurement method based on high-precision measurement of the concave and convex degree regions of the human face according to claim 1, wherein, In the described method, the dense and smooth point cloud data in the ROI area is used to draw a depth gradient contour through the point cloud depth distance, and then the depth gradient contour is converted into an RGB depth contour map and finally rendered onto the 3D model.

Images