Load-bearing position foot three-dimensional scanning and model reconstruction method

Abstract

The invention provides a load-bearing position foot three-dimensional scanning and model reconstruction method, comprising the following steps: S1, providing a detection device, the detection device is provided with an interactive interface, and the interactive interface is set to be suitable for obtaining a sole form when a collected person stands on the interactive interface under a load-bearing condition; s2, collecting a sole form in a load-bearing position state; and S3, processing and integrating the form data to generate a foot three-dimensional model. According to the load-bearing position foot three-dimensional scanning and model reconstruction method, the flexible interaction interface is used for replacing a traditional hard plane, the flexible interaction interface can be attached to the plantar curved surface in the load-bearing state, the stability of plantar soft tissue deformation is guaranteed, and therefore measurement errors caused by the traditional hard plane are remarkably reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of foot morphology scanning, and in particular to a three-dimensional scanning and model reconstruction method for the foot in the weight-bearing position. Background Art

[0002] The foot morphology is one of the important structures for human upright walking. Its specific morphological characteristics lay the foundation for providing support and propulsion. This morphology varies significantly due to various factors, including age, gender, race, weight, genetic diseases, and musculoskeletal-related diseases (such as hallux valgus, osteoarthritis, etc.). Accurately obtaining foot morphology data is an important prerequisite for the clinical diagnosis of foot diseases, orthosis customization, and shoe design. Traditional foot morphology measurement methods mainly rely on calipers, tape measures, and footprint methods, etc. Although the above methods are simple to operate, their accuracy and efficiency are limited. In the medical field, imaging techniques (such as X-rays) are often used to detect the bony structure of the foot, especially deformities caused by bony abnormalities. However, X-rays can only reflect the bony structure and are difficult to comprehensively present the soft tissue morphology of the foot.

[0003] Existing three-dimensional digital scanning technologies have achieved the acquisition of morphological data of different parts from the sole to the whole foot and are widely used in foot research. However, most of the interaction interfaces of existing morphology detection devices adopt a hard plane design, which has limitations in foot morphology detection. Specifically, when the sole of the foot contacts the hard plane, its deformation is restricted by the plane and cannot truly reflect the curved surface morphology of the sole in the natural weight-bearing state, resulting in deviations in the accuracy and authenticity of the collected data. The data deviation directly affects the diagnostic accuracy of foot diseases, the customization effect of orthoses, and the rationality and adaptability of shoe design. Although the non-weight-bearing detection method can obtain the curved surface morphology of the sole, since there is no actual contact between the sole and the detection device and no deformation occurs, its data lacks reference significance for the actual weight-bearing state and is difficult to meet the practical needs in clinical and design. The arch area of the foot is particularly special. Due to the presence of a large amount of soft tissue, this area is prone to deformation during weight-bearing, making the measurement results of the key parameter of arch height unstable and unreliable.

[0004] Therefore, to solve the problems existing in the above-mentioned prior art that the hard interaction interface cannot truly reflect the curved surface morphology of the sole in the natural weight-bearing state, the non-weight-bearing detection method lacks contact deformation, and the measurement results of soft tissues are unstable, the present invention proposes a three-dimensional scanning and model reconstruction method for the foot in the weight-bearing position. Summary of the Invention

[0005] To solve the problems existing in the application of the above-mentioned existing foot morphology detection devices, the present invention provides a three-dimensional scanning and model reconstruction method for the foot in the weight-bearing position.

[0006] According to an object of the present invention, the present invention provides a three-dimensional scanning and model reconstruction method for the foot in a weight-bearing position, including the following steps:

[0007] S1. Provide a detection device, on which an interaction interface is provided, and the interaction interface is configured to obtain the sole morphology when the subject stands on the interaction interface under weight-bearing conditions;

[0008] S2. Collect the sole morphology in the weight-bearing position, including:

[0009] S21. The subject stands still on the interaction interface and remains stable;

[0010] S22. Obtain the point cloud data of the subject's foot morphology through multi-view three-dimensional scanning. A multi-view three-dimensional scanning component is provided on the detection device, and the three-dimensional scanning component includes a light scanning part and a camera. The light scanning part scans through structured light or laser, and the light scanning part combines with the camera to capture the point cloud data of the subject's foot morphology;

[0011] S3. Process and integrate the morphological data to generate a three-dimensional foot model, including:

[0012] S31. Perform noise reduction processing on the point cloud obtained in step S22, set a range and frame the point cloud under each scanned view, and remove the point cloud outside the framed area;

[0013] S32. Downsample the point cloud within the framed area;

[0014] S33. Estimate the normal of the point cloud under each view;

[0015] S34. Redirect the normal of the point cloud according to the position of the camera, and make all the point clouds obtained through the camera face the corresponding camera position;

[0016] S35. Align and register the point clouds under multiple views to obtain a full-foot model;

[0017] S36. Fill the hole areas in the scanned data;

[0018] S37. Three-dimensionally reconstruct the full-foot point cloud and smooth the model surface.

[0019] Preferably, the interaction interface can adopt a flexible material, or a position sensor array, or a transparent viscoelastic material;

[0020] When the interaction interface adopts a flexible material, the interaction interface is configured to generate a deformation consistent with the sole when the subject stands on the interaction interface under weight-bearing conditions;

[0021] When the interactive interface adopts a position sensor array, the position sensors in the array record the displacement of the sole of the foot under the weight-bearing state, and the sole shape is reconstructed by combining an inversion algorithm.

[0022] When the interactive interface adopts a transparent viscoelastic material, an optical capture component corresponding to the viscoelastic material is further arranged on the interactive interface, and the optical capture component is arranged to indirectly reconstruct the sole shape by capturing the deformation and thickness change of the material.

[0023] Preferably, in step S31, the box selection area is set according to the detection device, and the point cloud outside the box selection area is the non-foot point cloud that cannot be detected by the light scanner and can be scanned by the camera, and the abnormal points are removed based on the neighborhood point statistics.

[0024] Preferably, in step S32, the voxel grid downsampling method is used to reduce the point cloud density and reduce the calculation amount of the point cloud.

[0025] Preferably, in step S33, the normal vectors of the point cloud in each view are estimated by KD-Tree respectively.

[0026] Preferably, the camera is a movable camera. In step S34, the normal vector of the point cloud is set along the direction of the shortest connection line between the point cloud and the moving axis of the camera.

[0027] Preferably, the camera adopts a ring-shaped multi-camera structure, and the camera is arranged to be able to capture the three-dimensional data of the sole and the dorsum of the foot at one time.

[0028] Preferably, in step S35, the point cloud alignment and registration method under multiple views includes:

[0029] Using a feature matching algorithm for preliminary global registration and using the iterative closest point algorithm to optimize the point cloud alignment for accurate registration;

[0030] Or,

[0031] Performing point cloud stitching according to the world coordinates of the camera.

[0032] Preferably, in step S36, an interpolation or depth completion algorithm is used to fill the hole area of the scanned data in the area that cannot be scanned.

[0033] Preferably, in step S37, the methods that can be adopted for three-dimensional reconstruction of the full-foot point cloud include:

[0034] Poisson method;

[0035] Or,

[0036] Deep learning method, directly generating a three-dimensional model of the foot from the scanned two-dimensional image or sparse point cloud;

[0037] Or,

[0038] For point cloud alignment based on marker points, specific marker points are set on the sole of the foot and the three-dimensional scanning component, and point cloud registration is completed through the marker points.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] This three-dimensional scanning and model reconstruction method for the foot in the weight-bearing position adopts a flexible interaction interface to replace the traditional rigid plane. The flexible interaction interface can fit the sole surface in the weight-bearing state, ensuring the stability of the deformation of the sole soft tissue, thereby significantly reducing the measurement error caused by the traditional rigid plane;

[0041] Adopting three-dimensional scanning in the weight-bearing state and using the weight-bearing scanning method can capture the real morphological data of the sole of the foot in the natural weight-bearing state, including the sole surface morphology and the deformation parameters of the arch area, solving the deficiency of lack of contact deformation in the non-weight-bearing method, and improving the authenticity and practicality of the detection data;

[0042] Adopting high-precision three-dimensional model reconstruction, based on the high-precision morphological data obtained by scanning for three-dimensional model reconstruction, the model reconstruction process can not only reflect the real surface characteristics of the sole of the foot, but also accurately describe the deformation of the soft tissue in the weight-bearing position, providing complete and reliable data support for the clinical diagnosis of foot diseases, orthosis customization and shoe design.

[0043] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. Description of the Drawings

[0044] Figure 1 Schematic diagram of a full-foot three-dimensional model obtained by the prior art;

[0045] Figure 2 Schematic diagram of a three-dimensional model of the foot obtained by the three-dimensional scanning and model reconstruction method for the foot in the weight-bearing position described in the present invention. Detailed Embodiments

[0046] The following description is used to elaborate the present invention in detail so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles defined in the following description can be applied to other implementation schemes, deformation schemes, improvement schemes, equivalent schemes and other technical schemes without departing from the spirit and scope of the present invention.

[0047] The present invention provides a technical solution: a three-dimensional scanning and model reconstruction method for the foot in the weight-bearing position, including the following steps:

[0048] S1. Provide a detection device, on which an interaction interface is provided. The interaction interface is the foot contact surface, and is configured to generate a deformation consistent with the sole of the foot when the person to be collected stands on the interaction interface under a load, so as to indirectly obtain the shape of the sole of the foot by subsequently scanning the shape of the flexible interaction interface. The flexible interface can adapt to the natural deformation of the sole curve under a load state, maintain the stability of the soft tissue deformation, significantly reduce the shape measurement error caused by the restriction of a hard plane, and ensure that the shape data of the sole soft tissue under the natural load state is more real and stable;

[0049] Regarding the interaction interface, the present invention provides the following embodiments including but not limited to:

[0050] Embodiment 1, the interaction interface is made of a flexible material, and is configured to generate a deformation consistent with the sole of the foot when the person to be collected stands on the interaction interface under a load;

[0051] Embodiment 2, the interaction interface uses a high-density position sensor array to replace the flexible material in Embodiment 1, and reconstructs the sole shape by recording the displacement of the sole under a load state and combining with an inversion algorithm;

[0052] Embodiment 3, the interaction interface combines a viscoelastic material with optical capture. Specifically, a transparent viscoelastic material is used as the interaction interface, and a high-precision camera system is arranged below it. The sole shape is indirectly reconstructed by capturing the deformation and thickness change of the material.

[0053] S2. Collect the sole shape in the weight-bearing position state, and the steps are as follows:

[0054] S21. The person to be collected stands still on the interaction interface and remains stable, and the flexible material generates a deformation consistent with the sole of the foot;

[0055] S22. Obtain the point cloud data of the foot shape of the person to be collected by multi-view three-dimensional scanning;

[0056] Specifically, the three-dimensional scanning is performed by structured light or laser scanning, and the point cloud data of the dorsal surface of the foot and the flexible material of the person to be collected is captured by combining with a high-precision camera. More specifically, a multi-view three-dimensional scanning component is provided on the detection device. The three-dimensional scanning component includes a light scanning part and a camera. Among them, both the light scanning part and the camera in the multi-view three-dimensional scanning component are provided with a plurality of them. The light scanning part and the camera are spaced apart and distributed on the outer periphery of the detection device to scan the foot of the person to be collected standing on the detection device from multiple angles. The light scanning part performs structured light or laser scanning, and combines with a high-precision camera to capture the point cloud data of the dorsal surface of the foot and the flexible material of the person to be collected.

[0057] S3. 3D model reconstruction: Process and integrate the morphological data obtained from the scanning in step S2 through a 3D reconstruction algorithm to generate a high-precision 3D model of the foot.

[0058] Step S3 includes the following steps:

[0059] S31. Denoise the point cloud obtained in step S22. Set a range and frame the point cloud at each perspective obtained from the scanning, and remove the point cloud outside the framed area. In step S31, the framed area is set according to the detection device, and the point cloud outside the framed area is the non-foot point cloud that cannot be detected by the optical scanner but can be scanned by the camera. Remove the abnormal points based on the neighborhood point statistics;

[0060] S32. Downsample the point cloud within the framed area; specifically, reduce the point cloud density by the voxel grid downsampling method to reduce the computational amount of the point cloud;

[0061] S33. Estimate the normal of the point cloud at each perspective; specifically, estimate the normal of the point cloud at each perspective through the KD-Tree (k-dimensional tree) respectively;

[0062] S34. Redirect the normal of the point cloud according to the position of the camera, so that all the point clouds obtained by the camera are oriented towards the corresponding camera position.

[0063] Regarding the scanning through the camera, the present invention provides the following embodiments including but not limited to:

[0064] Embodiment 1: Use the movable camera, where the camera can move in a straight line. When the camera moves along a straight line, the direction of the shortest connection line between the point cloud and the moving axis of the camera is used as the normal direction. This scanning method requires stitching of multiple perspectives obtained by the camera;

[0065] Embodiment 2: The camera can capture the 3D data of the sole and the dorsal surface of the foot at one time, avoiding multi-perspective stitching. Specifically, a ring-shaped multi-camera structure is adopted, that is, the number of cameras is multiple, and the cameras are annularly distributed on the outer periphery of the detection device. When in use, when scanning the foot through the camera, the camera can capture the 3D data of the sole and the dorsal surface of the foot at one time.

[0066] S35. Align and register the point clouds at multiple perspectives to obtain a full-foot model. The methods for aligning and registering the point clouds at multiple perspectives include at least the following two, namely Method 1 and Method 2. Among them, Method 1 is to perform preliminary global registration using a feature matching algorithm (such as RANSAC), and use the iterative closest point algorithm (ICP) to optimize the point cloud alignment for accurate registration. Among them, Method 2 is to perform point cloud stitching according to the world coordinates of the camera.

[0067] S36. Fill the hole areas in the scanned data.

[0068] Specifically, for areas that cannot be scanned, interpolation or depth completion algorithms are used to fill the void areas in the scanned data.

[0069] S37. Three-dimensionally reconstruct the full-foot point cloud and smooth the model surface.

[0070] Regarding the three-dimensional reconstruction of the full-foot point cloud, the present invention provides the following embodiments including but not limited to:

[0071] Embodiment 1: Use the Poisson method to perform three-dimensional reconstruction on the full-foot point cloud, and apply methods such as Gaussian filtering or Laplace to smooth the surface.

[0072] Embodiment 2: Use a deep learning method to directly generate a three-dimensional foot model from the scanned two-dimensional images or sparse point clouds.

[0073] Embodiment 3: Based on the point cloud alignment of marker points, specific marker points are set on the sole of the foot and the three-dimensional scanning component, and the point cloud registration is directly completed through the marker points.

[0074] For the full-foot three-dimensional model obtained by the above three-dimensional scanning and model reconstruction method of the foot in the weight-bearing position, refer to Figure 2 .

[0075] In summary, for the three-dimensional scanning and model reconstruction method of the foot in the weight-bearing position, a flexible interaction interface is adopted to replace the traditional rigid plane. The flexible interaction interface can fit the sole curve under the weight-bearing state, ensuring the stability of the deformation of the sole soft tissue, thereby significantly reducing the measurement error caused by the traditional rigid plane.

[0076] Three-dimensional scanning in the weight-bearing state is adopted. Using the weight-bearing scanning method, the true morphological data of the sole under the natural weight-bearing state can be captured, including the sole curve morphology and the deformation parameters of the arch area, solving the deficiency of the lack of contact deformation in the non-weight-bearing method, and improving the authenticity and practicality of the detection data.

[0077] High-precision three-dimensional model reconstruction is adopted. Based on the high-precision morphological data obtained by scanning, the three-dimensional model reconstruction can not only reflect the true curve characteristics of the sole, but also accurately describe the deformation of the soft tissue in the weight-bearing position, providing complete and reliable data support for the clinical diagnosis of foot diseases, orthosis customization, and shoe design.

[0078] The three-dimensional scanning of the foot in the weight-bearing position can more effectively reflect the morphological characteristics of the sole in the real use scenario, laying a foundation for the refined development of foot medical research and product design, improving the practicality and reliability of related applications. Through the comprehensive optimization of the interaction interface design, data acquisition, and model reconstruction process, the limitations of the existing technology are broken through, providing a scientific, efficient, and practical new solution for the field of foot morphology detection.

[0079] The embodiments described above are only used to illustrate the technical idea and features of the present invention. The purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The scope of the patent adoption of the present invention cannot be limited only by these embodiments. That is, any equivalent changes or modifications made according to the spirit disclosed by the present invention still fall within the scope of the patent of the present invention.

Claims

1. A method for three-dimensional scanning and model reconstruction of a weight-bearing foot, characterized in that: The following steps are involved: S1. Providing a detection device, wherein the detection device is provided with an interactive interface, wherein the interactive interface is configured to obtain the sole morphology when the subject stands on the interactive interface under a weight-bearing condition; S2. Collect the plantar morphology in the weight-bearing state, including: S21. The subject stands still on the interactive interface and remains stable; S22. Acquire the point cloud data of the foot shape of the subject by multi-view three-dimensional scanning, wherein the detection device is provided with a multi-view three-dimensional scanning component, the three-dimensional scanning component includes an optical scanning component and a camera, the optical scanning component scans by structured light or laser, and the optical scanning component combines with the camera to capture the point cloud data of the foot shape of the subject; S3. Process and integrate morphological data to generate a three-dimensional model of the foot, including: S31. De-noising the point cloud obtained in step S22, setting a range and selecting the point cloud at each viewing angle obtained by scanning, and removing the point cloud outside the selected area; S32. downsampling the point cloud within the selected area; S33. Estimate the normal of the point cloud at each viewing angle; S34. redirecting the normal of the point cloud according to the position of the camera, so that all point clouds acquired by the camera are oriented toward the corresponding camera position; S35. Align and register point clouds from multiple views to obtain a full foot model; S36. Filling the empty areas of the scanned data; S37. 3D reconstruction of the entire foot point cloud and smoothing of the model surface.

2. A method for three-dimensional scanning and model reconstruction of a foot in a weight-bearing position according to claim 1, characterized in that: The interactive interface may be made of a flexible material, or a position sensor array, or a transparent viscoelastic material; When the interactive interface is made of a flexible material, the interactive interface is configured to produce a deformation consistent with the sole of the foot when the subject stands on the interactive interface under a load condition; When the interactive interface uses a position sensor array, the position sensors of the array reconstruct the sole morphology by recording the displacement of the sole under weight-bearing state and combining the inversion algorithm; When the interactive interface adopts a transparent viscoelastic material, an optical capture component corresponding to the viscoelastic material is also provided on the interactive interface, and the optical capture component is configured to indirectly reconstruct the plantar morphology by capturing the deformation and thickness change of the material.

3. A method for three-dimensional scanning and model reconstruction of a foot in a weight-bearing position according to claim 1, characterized in that: In step S31, the framed area is set according to the detection device, and the point cloud outside the framed area is removed as the non-foot point cloud that cannot be detected by the optical scanner but can be scanned by the camera, and abnormal points are removed based on neighborhood point statistics.

4. A method for three-dimensional scanning and model reconstruction of a foot in a weight-bearing position according to claim 1, characterized in that: In step S32, the point cloud density is reduced by voxel grid downsampling method to reduce the amount of point cloud calculation.

5. A method for three-dimensional scanning and model reconstruction of a foot in a weight-bearing position according to claim 1, characterized in that: In step S33, the normal of the point cloud at each viewing angle is estimated respectively by KD-Tree.

6. A method for three-dimensional scanning and model reconstruction of a foot in a weight-bearing position according to claim 1, characterized in that: The camera is a movable camera. In step S34, the normal of the point cloud is set along the direction of the shortest line between the point cloud and the camera moving axis.

7. According to the method for three-dimensional scanning and model reconstruction of the foot in a weight-bearing position described in claim 1, the camera adopts a ring-shaped multi-camera structure, and the camera is configured to capture three-dimensional data of the sole and the instep at one time.

8. A method for three-dimensional scanning and model reconstruction of a foot in a weight-bearing position according to claim 1, characterized in that: In step S35, the point cloud alignment and registration method under multiple perspectives includes: Use feature matching algorithm for preliminary global registration, and use iterative closest point algorithm to optimize point cloud alignment for precise registration; or, Point cloud stitching is performed according to the world coordinates of the camera.

9. A method for three-dimensional scanning and model reconstruction of a foot in a weight-bearing position according to claim 1, characterized in that: In step S36, interpolation or depth completion algorithm is used to fill the empty areas of the scan data for the areas that cannot be scanned.

10. A method for three-dimensional scanning and model reconstruction of a foot in a weight-bearing position according to claim 1, characterized in that: In step S37, the method that can be used to 3D reconstruct the full foot point cloud includes: Poisson method; or, Deep learning methods to generate 3D models of the foot directly from scanned 2D images or sparse point clouds; or, Based on the point cloud alignment of marker points, specific marker points are set on the sole and 3D scanning components, and the point cloud alignment is completed through the marker points.

Images