A foot model construction method applied to diabetic foot

Abstract

The application provides a foot model construction method applied to diabetic foot, comprising the following steps: S1, scanning a patient's foot by a scanning device to obtain multiple foot color images and multiple foot depth images, and collecting initial posture data of the scanning device; S2, predicting estimated posture data of the scanning device according to the initial posture data and a posture estimation model obtained through pre-training, and processing to obtain an initial three-dimensional model of the foot; S3, screening all the foot color images to obtain multiple key images, and obtaining all the mapping points of the to-be-colored vertices of the initial three-dimensional model in all the key images; S4, coloring the to-be-colored vertices corresponding to the mapping points in the initial three-dimensional model according to the color values of all the mapping points, and generating a foot three-dimensional model. The beneficial effect is that the color of the ulcer and the depression degree of the ulcer part in the foot three-dimensional model are accurately restored, and it is convenient for medical staff to evaluate the condition of the diabetic foot.

Description

Technical Field

[0001] This invention relates to the technical field of three-dimensional model reconstruction, and in particular to a method for constructing a foot model for diabetic foot. Background Technology

[0002] Diabetic foot, specifically diabetic foot ulcers, is an ischemic, neurological, and neurovascular lesion of the foot caused by diabetes, leading to varying degrees of infection, ulceration, and gangrene. In the clinical treatment of diabetic foot ulcers, the prognosis needs to be assessed based on the ulcer area and depth to determine the appropriate treatment plan.

[0003] However, due to the high heterogeneity of clinical presentations among different patients, the assessment of diabetic foot ulcers is difficult to standardize.

[0004] Currently, the assessment methods for diabetic foot ulcers include:

[0005] The sterile film outlining method involves using a pre-marked grid of sterile film to adhere tightly to the patient's foot ulcer. A marker is used to record the surface shape of the ulcer along the edge of the ulcer. However, this method has the drawback of not being able to record the ulcer depth on the ulcer surface.

[0006] Two-dimensional images are used to simulate the volume of the ulcer surface. By acquiring images of the patient's foot and combining them with a spherical cone that closely resembles the curvature of the foot surface to simulate the volume of the ulcer wound, there is a drawback: the volume of the ulcer wound obtained from the processing is difficult to accurately calculate the ulcer depth.

[0007] After collecting data on the ulcer area and depth on the foot, it is necessary to classify the patient's diabetic foot condition according to a grading system and various indicators of the ulcer wound. This process is complex and has limited accuracy when assessing diabetic foot in patients. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention provides a method for constructing a foot model for diabetic foot, comprising:

[0009] Step S1: A scanning device continuously scans a patient's foot to obtain multiple color images of the foot and multiple depth images of the foot containing all foot features, and acquires an initial posture data of the scanning device at the start of the scan.

[0010] Step S2: Based on the initial posture data and a pre-trained posture estimation model, predict the estimated posture data corresponding to the scanning device when acquiring each foot depth image respectively, and process all the foot depth images, the initial posture data and all the estimated posture data to obtain an initial three-dimensional model of the foot.

[0011] Step S3: Filter all the foot color images to obtain multiple key images, and subdivide the initial 3D model to obtain multiple vertices to be colored. Then, obtain all the mapping points of the vertices to be colored in all the key images.

[0012] Step S4: For each vertex to be colored, a coloring value is obtained by processing the color values ​​of all the mapping points contained in all the key images. The vertex to be colored corresponding to the mapping point in the initial three-dimensional model is colored according to the coloring value to generate a three-dimensional model of the foot containing the surface texture of the foot, so as to assist medical staff in assessing the diabetic foot condition of the patient.

[0013] Preferably, step S3 includes an image filtering process, the steps of which are as follows:

[0014] Step A1: Calculate the blur level value corresponding to each of the foot color images;

[0015] Step A2: Based on the blur level value and a preset filtering standard, filter to obtain each blur level value that meets the filtering standard and the corresponding foot color image, and add each foot color image to a primary filtering set;

[0016] Step A3: Extract each foot color image from the primary filtering set according to a preset keyframe density and add them to each secondary filtering set respectively;

[0017] Step A4: For each of the secondary filtering sets, the foot color image corresponding to the lowest blur level value is output as the key image.

[0018] Preferably, step S2 includes:

[0019] Step S21: Obtain the time nodes of each foot depth image acquired by the scanning device, and sort all the foot depth images in order from first to last according to all the time nodes to obtain an image sequence;

[0020] Step S22: Obtain the foot depth image that is first in the image sequence, process the foot depth image and the initial pose data to obtain a local three-dimensional foot model, and delete the foot depth image from the image sequence;

[0021] Step S23: Obtain the foot depth image that is first in the image sequence, process the initial pose data and the pose estimation model to obtain estimated pose data, and update and save the local foot 3D model based on the foot depth image and the estimated pose data.

[0022] Step S24: Determine whether the foot depth image exists in the image sequence;

[0023] If so, output the estimated attitude data as the initial attitude data and return to step S23;

[0024] If not, the updated local foot 3D model will be output as the initial 3D model.

[0025] Preferably, step S2 includes an attitude estimation process, the steps of which are as follows:

[0026] Step B1: Obtain the foot depth image corresponding to the initial pose data, and extract features from the foot depth image to obtain multiple feature pixels.

[0027] Step B2: For each feature pixel, a first spatial coordinate is obtained by processing the first image coordinate of the feature pixel in the foot depth image and the corresponding depth value of the feature pixel.

[0028] Step B3: Input all the first spatial coordinates into the attitude estimation model to obtain the estimated attitude data and output it.

[0029] Preferably, step B3 further includes processing the estimated attitude data and a pre-configured error calibration function to obtain calibrated attitude data, and outputting the calibrated attitude data as the estimated attitude data.

[0030] Preferably, the foot depth image includes a domain and a range. Therefore, before performing step S2, a preprocessing process is further included, as follows:

[0031] Step D1: Based on the foot depth image processing, obtain the first weighted value of the foot depth image in the defined domain and the second weighted value of the foot depth image in the value domain;

[0032] Step D2: A filter weight function is obtained by processing the first weighted value, the second weighted value, and a pre-configured standard filter function.

[0033] Step D3: Process the foot depth image and the filtering weight function to obtain a filtered foot depth image and output it as the foot depth image.

[0034] Preferably, before performing step S3, a model optimization process is further included, with the following steps:

[0035] Step E1: Perform voxelization on the initial 3D model to obtain a voxel model of a foot containing each volume pixel.

[0036] Step E2: For each volumetric pixel, a second spatial coordinate is obtained by processing according to a pre-configured error function and the first spatial coordinate. Each volumetric pixel in the foot voxel model is moved to the corresponding second spatial coordinate to form an adjusted foot voxel model. The adjusted foot voxel model is then output as the initial 3D model.

[0037] Preferably, step S4 includes:

[0038] Step S41: In all the foot depth images, the feature pixel that is closest to the volume pixel is matched, and the color value of the feature pixel is obtained by processing the corresponding foot depth image.

[0039] Step S42: A color weight is obtained by processing the color value and a pre-configured light parameter;

[0040] Step S43: Assign color values ​​to all volumetric pixels in the foot voxel model according to the color weights and a preset interpolation method to form the foot surface texture of the foot voxel model, and output the foot voxel model containing the foot surface texture as the foot three-dimensional model to assist medical staff in assessing the patient's diabetic foot condition.

[0041] Preferably, the scanning device includes a first camera for acquiring the color image of the foot and a second camera for acquiring the depth image of the foot. Before performing step S1, a first camera calibration process is further included, as follows:

[0042] Step F1: Control the first camera to acquire images of a near-spherical feature at multiple different acquisition angles, and the first camera acquires each first calibration image;

[0043] Step F2: Extract features from the first calibration image to obtain multiple first feature points that characterize the contour of the feature body, and process all the first feature points to obtain the coordinates of the first detection center of the feature body;

[0044] Step F3: The feature body has a standard center coordinate. The parameter information of the first camera is obtained by processing the standard center coordinate and the first detection center coordinate, and then output so that the debugging personnel can adjust the parameters of the first camera to complete the calibration process of the first camera.

[0045] Preferably, a second camera calibration process is included before performing step S1, with the following steps:

[0046] Step G1: Control the second camera to acquire images of the moving feature body to obtain multiple corresponding second calibration images;

[0047] Step G2: Based on a pre-configured projection matrix and each of the second calibration images, a point cloud image containing the feature body is obtained, and features are extracted from the point cloud image to obtain multiple second feature points. Based on each of the second feature points, a three-dimensional model of the feature points is obtained.

[0048] Step G3: Project the center of the sphere of the 3D model of the feature points onto all the second calibration images respectively, and obtain the coordinates of the second detection center corresponding to the center of the sphere in each of the second calibration images;

[0049] Step G4: Determine whether there are any second detection center coordinates among all the second detection center coordinates that do not coincide with the standard center coordinates;

[0050] If so, the second camera is calibrated based on the difference between the coordinates of the second detection center and the coordinates of the standard center.

[0051] If not, complete the second camera calibration process and exit.

[0052] The above technical solution has the following advantages or beneficial effects:

[0053] (1) Medical personnel do not need to touch the patient's feet. The surface texture of the three-dimensional model of the patient's feet is used to assist medical personnel in assessing the condition of diabetic foot. When the degree of ulceration on the patient's feet is different, the color of different areas on the surface of the patient's feet is different. In addition, the skin surface of the ulcer is depressed due to the degree of ulceration. The surface texture of the foot contained in the three-dimensional model of the foot accurately represents the different colors of the ulceration on the surface of the patient's feet and the degree of depression of the ulceration on the patient's feet, which improves the efficiency of medical personnel in assessing the condition of diabetic foot.

[0054] (2) Screening of each foot color image to obtain key images with low blur value, and color values ​​obtained by processing key images are closer to the real color of the patient's foot, enhancing the realism of the foot surface texture, making the condition assessment results obtained by medical personnel based on the three-dimensional foot model more accurate.

[0055] (3) During the updating process of the foot three-dimensional model, the estimated posture data of the scanning device is calibrated to obtain calibrated posture data. The foot three-dimensional model is updated and optimized based on the calibrated posture data and foot depth image, which improves the accuracy and reliability of the foot three-dimensional model. Attached Figure Description

[0056] Figure 1A flowchart of a foot model construction method in a preferred embodiment of the present invention;

[0057] Figure 2 In a preferred embodiment of the present invention, a flowchart of the image selection process in the foot model construction method is provided.

[0058] Figure 3 In a preferred embodiment of the present invention, a detailed flowchart of step S2 in the foot model construction method is provided.

[0059] Figure 4 In a preferred embodiment of the present invention, a flowchart of the pose estimation process in the foot model construction method is provided.

[0060] Figure 5 In a preferred embodiment of the present invention, a flowchart of the preprocessing procedure in the foot model construction method is provided.

[0061] Figure 6 In a preferred embodiment of the present invention, a flowchart of the model optimization process in the foot model construction method is provided.

[0062] Figure 7 In a preferred embodiment of the present invention, a detailed flowchart of step S4 in the foot model construction method is provided.

[0063] Figure 8 In a preferred embodiment of the present invention, a flowchart of the first camera calibration process in the foot model construction method is provided.

[0064] Figure 9 In a preferred embodiment of the present invention, a flowchart of the second camera calibration process in the foot model construction method is provided. Detailed Implementation

[0065] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment; other embodiments that conform to the spirit of the present invention may also fall within the scope of the present invention.

[0066] In a preferred embodiment of the present invention, based on the above-mentioned problems existing in the prior art, a method for constructing a foot model for diabetic foot is provided, such as... Figure 1 As shown, it includes:

[0067] Step S1: A scanning device continuously scans a patient's foot to obtain multiple color images of the foot and multiple depth images of the foot containing all foot features, and acquires initial posture data of the scanning device at the start of the scan.

[0068] Step S2: Based on the initial pose data and a pre-trained pose estimation model, the estimated pose data corresponding to the scanning device when acquiring each foot depth image is obtained. Based on all foot depth images, the initial pose data and all estimated pose data, an initial three-dimensional model of the foot is obtained.

[0069] Step S3: Filter all foot color images to obtain multiple key images, and subdivide the initial 3D model to obtain multiple vertices to be colored. Then, obtain all mapping points of the vertices to be colored in all key images.

[0070] Step S4: For each vertex to be colored, a coloring value is obtained by processing the color values ​​of all mapping points contained in all key images. The vertex to be colored corresponding to the mapping points in the initial 3D model is then colored according to the coloring value to generate a 3D model of the foot containing the surface texture of the foot, so as to assist medical staff in assessing the condition of diabetic foot in patients.

[0071] Specifically, in this embodiment, the three-dimensional foot model includes the color information and the degree of depression of the ulcer site on the patient's foot. Medical personnel select multiple monitoring points in the three-dimensional foot model to obtain the ulcer depth data and color information corresponding to each monitoring point in the three-dimensional foot model. Medical personnel use the ulcer depth data and corresponding color information of each monitoring point to assess the ulcer level of the patient's foot, thereby achieving an efficient and accurate assessment of the patient's diabetic foot condition.

[0072] In another embodiment, the scanning device performs multi-directional scanning of the patient's feet under the control of a drive system.

[0073] The drive system includes a stepper motor mounted on the scanning device for controlling a first camera and a second camera in the scanning device to rotate horizontally. The drive system also includes a horizontally arranged threaded guide rail on which the scanning device is mounted to move horizontally.

[0074] Under the control of the drive system, the scanning device moves along the threaded guide rail at a uniform moving speed and rotates horizontally at a uniform rotational speed, adjusting the shooting angle of the scanning device. With uniform moving and rotating speeds, the probability of image blurring due to changes in the position of the scanning device is reduced, thus improving the quality of foot color images and foot depth images.

[0075] The first camera is used to capture color images of the foot, and the second camera is used to capture depth images of the foot.

[0076] In a preferred embodiment of the present invention, step S3 includes an image filtering process, such as... Figure 2As shown, the steps are as follows:

[0077] Step A1: Calculate the blur level value corresponding to each foot color image;

[0078] Step A2: Based on the blur level value and a preset screening criterion, filter to obtain each blur level value that meets the screening criterion and the corresponding foot color image, and add each foot color image to a primary screening set;

[0079] Step A3: Extract each foot color image from the primary filtering set according to a preset keyframe density and add them to each secondary filtering set respectively;

[0080] Step A4: For each secondary selection set, output the foot color image corresponding to the lowest blur value as the key image.

[0081] Specifically, in this embodiment, each foot color image is initially screened according to the screening criteria. The screened foot color images are added to a primary screening set. Then, all foot color images in the primary screening set are sorted in chronological order. Based on the keyframe density, all sorted foot color images in the primary screening set are sampled, and the sampled foot color images are added to a secondary screening set.

[0082] Based on the blur level values ​​of each foot color image in the secondary selection set, the foot color image with the lowest blur level is selected as the key image. At this point, the key image obtained in the most recent selection is located at the end of the secondary selection set. Then, the time frame corresponding to the last selected key image is obtained (t). - ,t + The system collects color images of each foot within a given time period and adds all the acquired color images of the feet to a primary filter set. This process is repeated until all color images of the feet have been filtered, resulting in a secondary filter set containing all the key images.

[0083] In another embodiment, (t) - ,t + ) = (1,2).

[0084] In another embodiment, a mean blurring image blurring algorithm is used to calculate the blur level value of each foot color image.

[0085] In a preferred embodiment of the present invention, such as Figure 3 As shown, step S2 includes:

[0086] Step S21: Obtain the time nodes of each foot depth image acquired by the scanning device, and sort all foot depth images in chronological order according to all time nodes to obtain an image sequence;

[0087] Step S22: Obtain the foot depth image that is first arranged in the image sequence, process the foot depth image and the initial pose data to obtain a local three-dimensional foot model, and delete the foot depth image from the image sequence.

[0088] Step S23: Obtain the foot depth image that is first in the image sequence, process the initial pose data and pose estimation model to obtain estimated pose data, and update and save the local three-dimensional foot model based on the foot depth image and estimated pose data.

[0089] Step S24: Determine whether a foot depth image exists in the image sequence;

[0090] If so, output the estimated attitude data as the initial attitude data and return to step S23;

[0091] If not, output the updated local foot 3D model as the initial 3D model.

[0092] Specifically, in this embodiment, the local three-dimensional model of the foot can be supplemented by foot depth images and estimated posture data, eliminating the step of real-time acquisition of posture data from the scanning device. Compared with the method of acquiring posture data from the scanning device, the step of time axis calibration of the acquired posture data and foot depth images is eliminated, which improves the efficiency of medical staff in assessing the patient's condition.

[0093] In a preferred embodiment of the present invention, step S2 includes a pose estimation process, such as... Figure 4 As shown, the steps are as follows:

[0094] Step B1: Obtain the foot depth image corresponding to the initial pose data, and extract features from the foot depth image to obtain multiple feature pixels.

[0095] Step B2: For each feature pixel, a first spatial coordinate is obtained by processing the first image coordinate of the feature pixel in the foot depth image and the corresponding depth value of the feature pixel.

[0096] Step B3: Input all first spatial coordinates into the attitude estimation model to obtain estimated attitude data and output it.

[0097] Specifically, in this embodiment, the first image coordinates are (i,j), and the expression for the corresponding first spatial coordinates is as follows:

[0098]

[0099] Where z represents the depth data corresponding to the feature pixel.

[0100] The pose estimation model includes a symbolic distance function. In step B3, the estimated pose data is obtained by processing the initial pose data and the symbolic distance function. That is, the actual physical distance between the foot and the scanning device is obtained by processing the points mapped to the current feature pixels in the three-dimensional model of the foot. Then, the estimated pose data is predicted based on the physical distance and the pose change trend of the scanning device.

[0101] The smaller the error in the estimated posture data, the more accurate the updated 3D foot model obtained from the estimated posture data and foot depth image processing.

[0102] In a preferred embodiment of the present invention, step B3 further includes processing the estimated attitude data and a pre-configured error calibration function to obtain calibrated attitude data, and outputting the calibrated attitude data as the estimated attitude data.

[0103] Specifically, in this embodiment, the expression for the error calibration function is as follows:

[0104]

[0105] Where E(R,t) is the error calibration function, R is the rotation matrix of the scanning device, t is the translation vector of the scanning device, and ψ(x) is the physical distance between the first spatial coordinate and the surface of the foot.

[0106] The rotation matrix R and translation vector t of the scanning device are derived from the coordinates x of the feature pixel in the reference frame of the scanning device. ij The results are as follows:

[0107] Define foot color image Define foot depth image Then the symbolic distance function

[0108] Given the image coordinates (i,j) of a feature pixel and its corresponding depth value z = I D (i,j), we get

[0109] Assuming the noise in the depth data of each foot depth image follows a Gaussian distribution and all pixels are independently and identically distributed, then the foot depth image I observed by the scanning device corresponding to the rotation matrix R and translation vector t... D The possibilities are:

[0110]

[0111] To find p(I) D The largest rotation matrix R(R,t) * and translation vector t * ,need:

[0112] The above formula is then simplified to obtain the error calibration function.

[0113] In another embodiment, in order to minimize the error of the estimated attitude data, the attitude of the scanning device is represented as ξ=(ω1,ω2,ω3,υ1,υ2,υ3);

[0114] Where (υ1,υ2,υ3) represents the translation vector and (ω1,ω2,ω3) represents the rotation matrix, the error calibration function is rewritten as:

[0115] Where, ψ ij (ξ)=ψ(Rx ij +t);

[0116] Next, we need to find ξ that minimizes the error, as follows:

[0117] Write the initial attitude data as ξ (0) and for ξ (0) If linearization is performed at this point, then...

[0118]

[0119] in, For the signed distance function at the k-th iteration, ξ (k) Substituting the partial derivative at point A into the rewritten error calibration function, we obtain an approximate function:

[0120]

[0121] Differentiating the approximate function yields:

[0122] Solving for ξ yields the value with the minimum.

[0123] In the subsequent calculations, the above formula is transformed into:

[0124]

[0125] Among them, the feature pixel points correspond to a six-dimensional vector b. ij , For b ij , Let A be a 6×6 matrix. ij ,definition:

[0126]

[0127]

[0128] Get: b+Aξ-Aξ (k)If = 0, then the estimated attitude data with the smallest error is:

[0129] ξ (k+1) =ξ (k) -A -1 b.

[0130] When calculating the next estimated attitude data, the above calculation steps are followed, wherein the initial attitude data is replaced by the estimated attitude data corresponding to the previous moment.

[0131] Due to the different feature pixels, vector b ij With matrix A ij The algorithms are independent of each other, so they can be computed in parallel using GPUs, which improves the efficiency of the algorithm.

[0132] In a preferred embodiment of the present invention, the foot depth image includes a domain and a range; therefore, before performing step S2, a preprocessing procedure is further included, such as... Figure 5 As shown, the steps are as follows:

[0133] Step D1: Based on the foot depth image processing, obtain the first weighted value of the foot depth image in the domain and the second weighted value of the foot depth image in the value range;

[0134] Step D2: A filter weight function is obtained by processing the first weight value, the second weight value, and a pre-configured standard filter function.

[0135] Step D3: Process the foot depth image and the filtering weight function to obtain a filtered foot depth image and output it as the foot depth image.

[0136] Specifically, in this embodiment, the expression for the standard filtering function is as follows:

[0137]

[0138] w(i,j,k,l)=d(i,j,k,l)×r(i,j,k,l);

[0139] Where d(i,j,k,l) ​​is the first weighting value, r(i,j,k,l) ​​is the second weighting value, w(i,j,k,l) ​​is the product of the first weighting value and the second weighting value, and f(k,l) is the pixel value of each pixel in the foot depth image.

[0140] In a preferred embodiment of the present invention, a model optimization process is further included before performing step S3, such as... Figure 6 As shown, the steps are as follows:

[0141] Step E1: Voxelize the initial 3D model to obtain a voxel model of a foot containing each volume pixel.

[0142] Step E2: For each volumetric pixel, a second spatial coordinate is obtained by processing according to a pre-configured error function and the first spatial coordinate. Each volumetric pixel in the foot voxel model is moved to the corresponding second spatial coordinate to form an adjusted foot voxel model. The adjusted foot voxel model is then output as the initial 3D model.

[0143] Specifically, in this embodiment, two voxel grids with a resolution of m are used to represent the symbolic distance function, where one voxel grid is used to store the average distance D and the other voxel grid is used to store the weights W.

[0144] When converting a 3D foot model into a foot voxel model, it is necessary to calculate the signed distance function corresponding to each volume pixel, update the error function based on the signed distance function, and obtain the second spatial coordinates based on the first spatial coordinates and the error function. The position of the volume pixels is then updated based on the second spatial coordinates.

[0145] The error function is as follows:

[0146]

[0147] Where, d I W is the cutoff distance of the volume pixel in the foot depth image I. I represents the weight of the volume pixels in the foot depth image I.

[0148] In a preferred embodiment of the present invention, such as Figure 7 As shown, step S4 includes:

[0149] Step S41: Match the feature pixel that is closest to the volume pixel in all foot depth images, and process the feature pixel to obtain its color value according to the corresponding foot depth image.

[0150] Step S42: Obtain a color weight value by processing the color value and a pre-configured light parameter;

[0151] Step S43: Assign color values ​​to all volumetric pixels in the foot voxel model according to color weights and a preset interpolation method to form the foot surface texture of the foot voxel model, and output the foot voxel model containing the foot surface texture as a three-dimensional foot model to assist medical staff in assessing the condition of diabetic foot in patients.

[0152] Specifically, in this embodiment, four additional voxel grids are used, namely:

[0153] The voxel mesh R represents the red channel of the color texture;

[0154] The voxel mesh G represents the green channel of the color texture;

[0155] Voxel mesh B represents the blue channel of the color texture;

[0156] Voxel mesh W c , representing the weight of color confusion.

[0157] For each volumetric pixel, the voxel meshes are independent of each other, and can be accelerated in parallel using a GPU.

[0158] The distance between the volume pixel and the patient's foot surface is: ||d||<ε, indicating that the distance is infinitely close, according to (r,g,b). T =I RGB The color values ​​are obtained by processing (i,j), and then the voxel mesh is updated using an iterative weighted average method:

[0159]

[0160]

[0161]

[0162]

[0163] in, The color weight is the value at the current moment, and the expression for the color weight is as follows:

[0164]

[0165] θ is the angle between the projected ray of a volume pixel and the optical axis, i.e., the ray parameter.

[0166] In another embodiment, the interpolation method is trilinear interpolation.

[0167] In a preferred embodiment of the present invention, the scanning device includes a first camera for acquiring color images of the foot and a second camera for acquiring depth images of the foot. Therefore, a first camera calibration process is included before performing step S1. Figure 8 As shown, the steps are as follows:

[0168] Step F1: Control the first camera to acquire images of a near-spherical feature at multiple different acquisition angles, and the first camera acquires each first calibration image;

[0169] Step F2: Extract features from the first calibration image to obtain multiple first feature points that characterize the contour of the feature body, and process all the first feature points to obtain the coordinates of the first detection center of the feature body;

[0170] Step F3: The feature body has a standard center coordinate. The parameter information of the first camera is obtained by processing the standard center coordinate and the first detection center coordinate and output, so that the debugging personnel can adjust the parameters of the first camera to complete the calibration process of the first camera.

[0171] In a preferred embodiment of the present invention, a second camera calibration process is further included before performing step S1, such as... Figure 9 As shown, the steps are as follows:

[0172] Step G1: Control the second camera to acquire images of the moving feature body to obtain multiple corresponding second calibration images;

[0173] Step G2: Based on a pre-configured projection matrix and each second calibration image, a point cloud image containing feature volumes is obtained. Feature extraction is performed on the point cloud image to obtain multiple second feature points. Based on each second feature point, a feature point three-dimensional model is obtained.

[0174] Step G3: Project the center of the sphere of the feature point 3D model onto all the second calibration images respectively, and obtain the coordinates of the second detection center corresponding to the center of the sphere in each second calibration image;

[0175] Step G4: Determine whether there are any second detection center coordinates among all the second detection center coordinates that do not coincide with the standard center coordinates;

[0176] If so, calibrate the second camera based on the difference between the coordinates of the second detection center and the coordinates of the standard center;

[0177] If not, complete the second camera calibration process and exit.

[0178] Specifically, in this embodiment, the second camera is an Intel RealSense SR300 camera.

[0179] In another embodiment, a two-stage tracking and reconstruction method is adopted. By changing the values ​​of ε and δ, the estimated posture data is recalculated, and the posture data obtained from the second calculation is reapplied to the signed distance function to reconstruct a three-dimensional foot model with sharper edges, making the various foot features contained in the three-dimensional foot model clearer.

[0180] The values ​​of ε and δ in the second calculation are smaller than those in the first calculation.

[0181] During the application, the reconstruction of the foot's 3D model was performed on a personal computer equipped with an Intel Core i5 quad-core processor and an NVIDIA GTX 1060 graphics card.

[0182] During the scanning process, the Intel RealSense SR300 RGB-D camera acquired data at a frame rate of 30 FPS. The entire scanning process lasted about 20 seconds, and a total of about 600 RGB-D frames were obtained.

[0183] The geometric model reconstruction algorithm took 18 seconds, while the surface texture generation algorithm took 5 minutes after 80 optimization iterations.

[0184] For the two-stage tracking reconstruction, ε = 2mm and δ = 9mm were used in the first stage, and ε = 1mm and δ = 6mm were used in the second stage. This resulted in sharper edges between the toes in the reconstructed 3D foot model, improving the model's performance.

[0185] In summary, by filtering various foot color images to obtain key images with low blur values, the color values ​​obtained from processing these key images more closely resemble the true color of the patient's foot, enhancing the realism of the foot surface texture. This leads to more accurate disease assessments by medical personnel based on the 3D foot model. During the updating process of the 3D foot model, error calibration is performed on the estimated posture data from the scanning equipment to obtain calibrated posture data. Based on this calibrated posture data and foot depth images, the 3D foot model is updated and optimized, improving its accuracy and reliability.

[0186] The 3D foot model accurately represents the different colors of the ulcers on the patient's foot surface, including the texture of the foot surface, and also shows the degree of depression of the ulcer site, thus improving the efficiency of medical personnel in assessing the condition of diabetic foot.

[0187] The above description is merely a preferred embodiment of the present invention and does not limit the implementation and protection scope of the present invention. Those skilled in the art should realize that any equivalent substitutions and obvious changes made using the content of this specification and illustrations should be included within the protection scope of the present invention.

Claims

1. A method for constructing a foot model for diabetic foot, characterized in that, include: Step S1: A scanning device continuously scans a patient's foot to obtain multiple color images of the foot and multiple depth images of the foot containing all foot features, and acquires an initial posture data of the scanning device at the start of the scan. Step S2: Based on the initial posture data and a pre-trained posture estimation model, predict the estimated posture data corresponding to the scanning device when acquiring each foot depth image respectively, and process all the foot depth images, the initial posture data and all the estimated posture data to obtain an initial three-dimensional model of the foot. Step S3: Filter all the foot color images to obtain multiple key images, and subdivide the initial 3D model to obtain multiple vertices to be colored. Then, obtain all the mapping points of the vertices to be colored in all the key images. Step S4: For each vertex to be colored, a coloring value is obtained by processing the color values ​​of all the mapping points contained in all the key images. The vertex to be colored corresponding to the mapping point in the initial three-dimensional model is colored according to the coloring value to generate a three-dimensional model of the foot containing the surface texture of the foot, so as to assist medical staff in assessing the diabetic foot condition of the patient.

2. The foot model construction method according to claim 1, characterized in that, Step S3 includes an image filtering process, the steps of which are as follows: Step A1: Calculate the blur level value corresponding to each of the foot color images; Step A2: Based on the blur level value and a preset filtering standard, filter to obtain each blur level value that meets the filtering standard and the corresponding foot color image, and add each foot color image to a primary filtering set; Step A3: Extract each foot color image from the primary filtering set according to a preset keyframe density and add them to each secondary filtering set respectively; Step A4: For each of the secondary filtering sets, the foot color image corresponding to the lowest blur level value is output as the key image.

3. The foot model construction method according to claim 1, characterized in that, Step S2 includes: Step S21: Obtain the time nodes of each foot depth image acquired by the scanning device, and sort all the foot depth images in order from first to last according to all the time nodes to obtain an image sequence; Step S22: Obtain the foot depth image that is first in the image sequence, process the foot depth image and the initial pose data to obtain a local three-dimensional foot model, and delete the foot depth image from the image sequence; Step S23: Obtain the foot depth image that is first in the image sequence, process the initial pose data and the pose estimation model to obtain estimated pose data, and update and save the local foot 3D model based on the foot depth image and the estimated pose data. Step S24: Determine whether the foot depth image exists in the image sequence; If so, output the estimated attitude data as the initial attitude data and return to step S23; If not, the updated local foot 3D model will be output as the initial 3D model.

4. The foot model construction method according to claim 1, characterized in that, Step S2 includes an attitude estimation process, the steps of which are as follows: Step B1: Obtain the foot depth image corresponding to the initial posture data, and extract features from the foot depth image to obtain multiple feature pixels. Step B2: For each feature pixel, a first spatial coordinate is obtained by processing the first image coordinate of the feature pixel in the foot depth image and the corresponding depth value of the feature pixel. Step B3: Input all the first spatial coordinates into the attitude estimation model to obtain the estimated attitude data and output it.

5. The foot model construction method according to claim 4, characterized in that, Step B3 further includes processing the estimated attitude data and a pre-configured error calibration function to obtain calibrated attitude data, and outputting the calibrated attitude data as the estimated attitude data.

6. The foot model construction method according to claim 3, characterized in that, The foot depth image includes a domain and a range. Before performing step S2, a preprocessing process is also included, as follows: Step D1: Based on the foot depth image processing, obtain the first weighted value of the foot depth image in the defined domain and the second weighted value of the foot depth image in the value domain; Step D2: A filter weight function is obtained by processing the first weighted value, the second weighted value, and a pre-configured standard filter function. Step D3: Process the foot depth image and the filtering weight function to obtain a filtered foot depth image and output it as the foot depth image.

7. The foot model construction method according to claim 4, characterized in that, Before performing step S3, a model optimization process is also included, with the following steps: Step E1: Perform voxelization on the initial 3D model to obtain a voxel model of a foot containing each volume pixel. Step E2: For each volumetric pixel, a second spatial coordinate is obtained by processing according to a pre-configured error function and the first spatial coordinate. Each volumetric pixel in the foot voxel model is moved to the corresponding second spatial coordinate to form an adjusted foot voxel model. The adjusted foot voxel model is then output as the initial 3D model.

8. The foot model construction method according to claim 7, characterized in that, Step S4 includes: Step S41: In all the foot depth images, the feature pixel that is closest to the volume pixel is matched, and the color value of the feature pixel is obtained by processing the corresponding foot depth image. Step S42: A color weight is obtained by processing the color value and a pre-configured light parameter; Step S43: Assign color values ​​to all volumetric pixels in the foot voxel model according to the color weights and a preset interpolation method to form the foot surface texture of the foot voxel model, and output the foot voxel model containing the foot surface texture as the foot three-dimensional model to assist medical staff in assessing the patient's diabetic foot condition.

9. The foot model construction method according to claim 1, characterized in that, The scanning device includes a first camera for acquiring color images of the foot and a second camera for acquiring depth images of the foot. Before performing step S1, a first camera calibration process is also included, as follows: Step F1: Control the first camera to acquire images of a near-spherical feature at multiple different acquisition angles, and the first camera acquires each first calibration image; Step F2: Extract features from the first calibration image to obtain multiple first feature points that characterize the contour of the feature body, and process all the first feature points to obtain the coordinates of the first detection center of the feature body; Step F3: The feature body has a standard center coordinate. The parameter information of the first camera is obtained by processing the standard center coordinate and the first detection center coordinate, and then output so that the debugging personnel can adjust the parameters of the first camera to complete the calibration process of the first camera.

10. The foot model construction method according to claim 9, characterized in that, Before performing step S1, a second camera calibration process is also included, the steps of which are as follows: Step G1: Control the second camera to acquire images of the moving feature body to obtain multiple corresponding second calibration images; Step G2: Based on a pre-configured projection matrix and each of the second calibration images, a point cloud image containing the feature body is obtained, and features are extracted from the point cloud image to obtain multiple second feature points. Based on each of the second feature points, a three-dimensional model of the feature points is obtained. Step G3: Project the center of the sphere of the 3D model of the feature points onto all the second calibration images respectively, and obtain the coordinates of the second detection center corresponding to the center of the sphere in each of the second calibration images; Step G4: Determine whether there are any second detection center coordinates among all the second detection center coordinates that do not coincide with the standard center coordinates; If so, the second camera is calibrated based on the difference between the coordinates of the second detection center and the coordinates of the standard center. If not, complete the second camera calibration process and exit.

Images