A method for modeling body surface contours

Abstract

The present invention relates to the field of modeling technology, and particularly relates to a method for modeling the body surface contour, comprising the following steps: S1, loading the original medical image and performing preprocessing; S2, respectively detecting the skin body surface contour in each layer of the preprocessed medical image and removing redundant components outside the main cross-section area; S3, integrating the skin body surface contours of all layers and preliminarily constructing an overall 3D model of the body surface; S4, performing smoothing processing on the preliminarily constructed 3D model of the body surface to obtain the final 3D model of the body surface. The present invention can realize the rapid and accurate construction of a 3D model of the body surface contour.

Description

Technical Field

[0001] The present invention relates to the technical field of modeling, and more particularly to a method for modeling the body surface contour. Background Art

[0002] Malignant tumors in parts such as the liver or cranial nerve functional diseases are major diseases that seriously threaten human health and quality of life. As an effective minimally invasive interventional treatment method, the thermal ablation technology based on microwave or radio frequency has been widely used in clinical practice in recent years. Among them, realizing the path planning and execution of high-precision puncture of the ablation needle is the most important requirement and the key to ensuring the treatment effect.

[0003] In clinical practice, manual puncture guided by medical images such as magnetic resonance, CT, and ultrasound is a common surgical method for high-precision puncture. Among them, the planning of the ablation surgery is mainly carried out by doctors by browsing the tomographic scan images of the patient layer by layer, based on a rough estimate of the size, location, and shape of the patient's tumor, and avoiding major blood vessels, nerves, bones, or other important tissues, and performing surgical path planning in the mind. This manual surgical path planning method is time-consuming and laborious, and highly dependent on the doctor's puncture experience and professional skills, resulting in limited clinical application.

[0004] In recent years, robot-assisted puncture ablation interventional surgery has gradually entered the clinic. It can provide support for doctors' surgical operations from the aspects of vision, hearing, and touch, can achieve precise control of surgical instruments in the field of minimally invasive surgery, effectively improve the accuracy and efficiency of puncture surgery, and reduce the excessive dependence on the operator's experience.

[0005] However, in the robot-assisted path planning and puncture execution links, it is necessary to extract the outer contour of the body surface (skin surface) part layer by layer from medical images, and then construct a complete 3D surface model, so that when the doctor plans the path, it can provide auxiliary suggestions for the possible needle insertion positions of the puncture needle on the body surface, and before the robotic arm moves, detect and avoid the risk of possible collision with the abdomen or other parts.

[0006] Therefore, how to construct a 3D model of the body surface contour more quickly and accurately has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a method for modeling the body surface contour, which can quickly and accurately construct a 3D model of the body surface contour.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] A method for modeling the body surface contour, comprising the following steps:

[0010] S1. Load the original medical image and perform preprocessing;

[0011] S2. In each layer of the preprocessed medical image, detect the skin body surface contour respectively, and remove the redundant components outside the main body cross-section area;

[0012] S3. Integrate the skin body surface contours of all layers, and preliminarily construct the overall 3D body surface model;

[0013] S4. Smooth the preliminarily constructed 3D body surface model to obtain the final 3D body surface model.

[0014] Further, S1 includes:

[0015] S11. Read the medical image in a preset format;

[0016] S12. Load the data of each layer of the medical image to form volume data;

[0017] S13. Parse the corresponding metadata from the volume data, and the metadata includes at least the origin coordinates, direction vectors, pixel spacings, and slice spacings of each layer;

[0018] S14. Perform preprocessing on the loaded original medical image.

[0019] Further, in S14, the method for preprocessing the original medical image data includes: normalizing the pixel gray value distribution range in the original medical image, and performing filtering to remove isolated noise points.

[0020] Further, S2 includes:

[0021] S21. For each layer in the preprocessed medical image, detect all possible edge lines;

[0022] S22. Remove the redundant components outside the main body area in the medical image;

[0023] S23. Use convex hull operation to obtain the outer contour line that contains all the edge lines in the main body area.

[0024] Further, S22 includes:

[0025] S221. Project all the edge lines in the horizontal and vertical axes respectively, set the places with edge lines as 1 and other places as 0, and take the union;

[0026] S222. Search for all continuous segments with a value of 1 on both axes respectively, and take the longest one;

[0027] S223. Retain the rectangular area where the longest segments of the vertical and horizontal axes intersect as the candidate area for the target, discard the contour lines outside the candidate area, and remove the redundant components other than the main area.

[0028] Further, S3 includes:

[0029] S31. Take the points on the body surface contour lines of every two adjacent layers as the vertices of the 3D model, extend them in a clockwise direction to form a closed loop, and construct an annular triangular strip.

[0030] S32. Merge the triangular strips constructed for all adjacent layers together to form a preliminary 3D model of the body surface.

[0031] Further, S31 includes:

[0032] S311. Denote the set of contour points of adjacent layer 1 as arr1, the set of contour points of layer 2 as arr2, and the unit length direction vector perpendicular to the scanning plane as Z_norm.

[0033] S312. Find the point P1 with the minimum Y coordinate value from the set arr1, take it as the starting point, and arrange the other contour points in arr1 in the original order behind it to form a new point set pts1.

[0034] S313. Find the point Q1 with the minimum Euclidean distance from P1 from the set arr2, take it as the starting point, and arrange the other contour points in arr2 in the original order behind it to form a new point set pts2.

[0035] S314. Assume the point after P1 is P2, denote the direction vector from P1 to P2 as R1 = P2 - P1, and its length as D1; denote the point where the line starting from P1 along the Z_norm direction intersects the image plane of layer 2 as H1, H1 = P1 + Z_norm * layer spacing.

[0036] S315. Assume the point after Q1 is Q2, denote the direction vector from H1 to Q2 as R2 = Q2 - H1, and its vector length as D2.

[0037] S316. Take the dot product of vector R1 and vector R2, the result represents the cosine of the angle between the two, denoted as theta; if theta > 0 and D1 < D2, then take P2 as the selected point, otherwise take Q2 as the selected point; form a triangle with the selected point, P1, and Q1, and put it into a set S.

[0038] S317. In S316, if the selected point is P2, then take it as the new starting point P1 in pts1; if the selected point is Q2, then take it as the new starting point Q1 in pts2.

[0039] S318. Repeat the above steps S313 - S317 until all the contour points in pts1 and pts2 are traversed, and the process ends. At this time, all the triangles in the set S form a triangle strip between layer 1 and layer 2.

[0040] Further, S4 includes:

[0041] S41. Smooth the preliminarily constructed 3D body surface model using a surface mesh smoothing algorithm.

[0042] S42. Remove the redundant parts through an edge - trimming post - processing method to obtain the final 3D body surface model.

[0043] Further, in S41, the surface mesh smoothing algorithm adopted is Poisson resampling method, Taubin filtering or rolling ball method.

[0044] Further, S42 includes:

[0045] S421. For each vertex of the triangles that make up the preliminarily constructed 3D body surface model, subtract its coordinates from the image plane origin coordinates in the first layer and the last layer metadata of the entire volume data respectively to construct two vectors, denoted as V1 and V2.

[0046] S422. Take the dot product of vectors V1 and V2 with the normal vector Z_norm along the Z - axis direction, denoted as theta1 and theta2 respectively. Determine whether the vertex of the triangle is in the valid region or in the invalid region outside the upper / lower layer according to the positive and negative values of theta1 and theta2.

[0047] The judgment criterion is: If theta1 < 0, then it is determined that the vertex of the triangle is outside the first layer and is an invalid point; otherwise, if theta2 > 0, then it is determined that the vertex of the triangle is outside the last layer and is also an invalid point; only when theta1 ≥ 0 and theta2 ≤ 0, it is determined that the vertex of the triangle is in the valid region between the first layer and the last layer.

[0048] S423. Judge the set S of triangles containing these vertices: If any vertex of a certain triangle is in the invalid region, discard the triangle, and then reconstruct a surface mesh set from all the remaining triangles to form the final 3D body surface model.

[0049] From the above - mentioned technical solutions, compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. The present invention constructs an initial 3D model based on the layer-by-layer body contour, which corresponds one-to-one with the spatial structure of the original scanned image, facilitating the subsequent improvement of the model by combining relevant features within and between layers. For example, smoothing processing is used to address the blurring or signal loss in local scanning areas.

[0051] 2. The process of constructing triangular strips using the contour points of adjacent layers in the present invention is simple and direct, without redundant operations, with a small amount of calculation, and can achieve the goal of fast or even real-time modeling.

[0052] 3. The present invention is applicable to robot-assisted puncture surgery under magnetic resonance or CT guidance. When planning the puncture path, it can quickly construct a 3D model of the patient's body surface contour from the scanned images, thereby providing auxiliary suggestions for planning possible needle entry positions on the body surface, and detecting and avoiding in a timely manner the risk that the movement trajectories of the joints of the robotic arm may collide with the patient's body surface before the robotic arm moves. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0054] Figure 1 It is a flowchart of the modeling method for the body surface contour provided by the present invention;

[0055] Figure 2 It is a schematic diagram of the outer contour of the main area in a certain layer provided by the present invention;

[0056] Figure 3 It is a schematic diagram of the construction process of the annular triangular strip provided by the present invention;

[0057] Figure 4 It is a schematic diagram of the structure after smoothing the preliminarily constructed body surface 3D model provided by the present invention;

[0058] Figure 5 It is a schematic diagram of the principle of whether a vertex of a certain triangle is in the effective area provided by the present invention;

[0059] Figure 6 It is a schematic diagram of the structure of the finally constructed body surface 3D model provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0061] As Figure 1 shown, an embodiment of the present invention discloses a method for modeling the body surface contour, including the following steps:

[0062] S1. Load the original medical image and perform preprocessing;

[0063] S2. In each layer of the preprocessed medical image, detect the skin body surface contour respectively, and remove the redundant components outside the main cross-section area;

[0064] S3. Integrate the skin body surface contours of all layers to initially construct an overall 3D body surface model;

[0065] S4. Smooth the initially constructed 3D body surface model to obtain the final 3D body surface model.

[0066] Next, the above steps will be further described.

[0067] S1. Load the original medical image and perform preprocessing, specifically including:

[0068] S11. Read the medical image in a preset format, and the format here includes but is not limited to the Dicom or Nifti format;

[0069] S12. Load the data of each layer of the medical image to form volume data (Volume); each layer refers to each frame of the original image scanned by magnetic resonance or CT, usually scanned along the axial direction;

[0070] S13. Parse the corresponding metadata from the Dicom or Nifti file of the volume data, and the metadata at least includes the origin coordinates, direction vectors, pixel pitch, and slice thickness of each layer;

[0071] S14. Perform preprocessing on the loaded original medical image. The specific preprocessing methods include: normalizing the pixel gray distribution range in the original medical image and performing filtering to remove isolated noise points.

[0072] Among them, pixel adjustment is used to brighten the overall darker scanned image to a moderate level, which helps with subsequent recognition processing. It includes two methods. Method 1: Calculate the cumulative histogram curve, truncate it at 1‰ downward from the maximum value, and then normalize it to the range of 0 - 255. Method 2: Normalize it to the range of 0 - 255 according to the maximum and minimum values of the image.

[0073] The filtering method can adopt Gaussian filtering, median filtering, etc.

[0074] S2. In each layer of the pre - processed medical image, detect the skin body surface contour respectively, and remove the redundant components outside the main body cross - section area, specifically including:

[0075] S21. For each layer of the pre - processed medical image, detect all possible edge lines; the Canny operator can be applied to detect all possible edge lines.

[0076] S22. Remove the redundant components outside the main body area in the medical image, specifically including:

[0077] S221. Project all the edge lines in the horizontal and vertical directions respectively. Set the places with edge lines as 1 and other places as 0, and take the union.

[0078] S222. Search for all continuous segments with a value of 1 on both axes respectively, and take the longest one.

[0079] S223. Keep the rectangular area where the longest segments on the vertical and horizontal axes intersect as the candidate area of the target, discard the contour lines outside the candidate area, and remove the redundant components outside the main body area.

[0080] S23. Use the convex hull operation to obtain the outer contour line that contains all the edge lines in the main body area.

[0081] After this step, the obtained result is as Figure 2 shown, Figure 2 In it, the green ones are the detected edge lines; the small white areas on both sides are the external components excluded by the projection method; the blue closed curve is the outer contour of the body surface area of this layer (formed by connecting the contour points).

[0082] S3. Integrate the skin body surface contours of all layers and initially construct the overall 3D body surface model, specifically including:

[0083] S31. Take the points on the body surface contours of every two adjacent layers as the vertices of the 3D model, extend them in the clockwise direction to form a closed loop, and construct an annular triangular strip, as Figure 3 shown; among them, the method of constructing the triangular strip of adjacent layers includes:

[0084] S311. Denote the set of contour points of adjacent layer 1 as arr1, the set of contour points of layer 2 as arr2, and the unit length direction vector perpendicular to the scanning plane (i.e., the normal vector) as Z_norm, which can be obtained by taking the cross product of the unit length and orthogonal X-direction vector (X_vect) and Y-direction vector (Y_vect) in the layer plane of the metadata.

[0085] S312. Find the point P1 with the minimum Y coordinate value in the set arr1 (if there are multiple points corresponding to different X values that meet this condition, choose any one), use it as the starting point, and arrange the other contour points in arr1 in the original direction (i.e., clockwise) behind it to form a new point set pts1.

[0086] S313. Find the point Q1 with the minimum Euclidean distance from P1 in the set arr2, use it as the starting point, and arrange the other contour points in arr2 in the original direction (clockwise) behind it to form a new point set pts2.

[0087] S314. Assume the point after P1 is P2, denote the direction vector from P1 to P2 as R1 = P2 - P1, and its length as D1; denote the point where the line starting from P1 along the Z_norm direction intersects the image plane where layer 2 is located as H1, H1 = P1 + Z_norm * layer spacing.

[0088] S315. Assume the point after Q1 is Q2, denote the direction vector from H1 to Q2 as R2 = Q2 - H1, and its vector length as D2.

[0089] S316. Take the dot product of vector R1 and vector R2, and the result represents the cosine of the angle between them, denoted as theta; if theta > 0 and D1 < D2, then take P2 as the selected point, otherwise take Q2 as the selected point; form a triangle with the selected point, P1, and Q1 and put it in a set S.

[0090] S317. In S316, if the selected point is P2, then use it as the new starting point P1 in pts1; if the selected point is Q2, then use it as the new starting point Q1 in pts2.

[0091] S318. Repeat the above steps S313 - S317 until all the contour points in pts1 and pts2 have been traversed (participated in the formation of triangles), and the process ends. At this time, all the triangles in the set S form a triangular strip between layer 1 and layer 2.

[0092] S32. Merge all the triangular strips constructed for adjacent layers together to form a preliminary 3D body surface model.

[0093] However, the connection between the layers of this model is not very smooth and continuous, so further smoothing is required.

[0094] S4. Smooth the preliminarily constructed 3D body surface model to obtain the final 3D body surface model, including:

[0095] S41. Smooth the preliminarily constructed 3D body surface model using a surface mesh smoothing algorithm; the surface mesh smoothing algorithm adopted is Poisson resampling, Taubin filtering, or the rolling ball method.

[0096] Some smoothing algorithms may naturally extend a non-existent smooth contour at the upper and lower edges of the 3D body surface model, that is, the part beyond the ROI of the scanned image, as Figure 4 shown, so a post-processing step is needed to trim them off.

[0097] S42. Remove the redundant parts through an edge trimming post-processing method to obtain the final 3D body surface model, including:

[0098] S421. For each vertex of the triangles that make up the preliminarily constructed 3D body surface model, subtract its coordinates from the origin coordinates of the image plane in the first layer and the last layer of the entire volume data to construct two vectors, denoted as V1 and V2 respectively;

[0099] S422. Take the dot product of vectors V1 and V2 with the normal vector Z_norm along the Z-axis direction, denoted as theta1 and theta2 respectively, and judge whether the vertex of this triangle is in the valid area or the invalid area outside the upper / lower layer according to the positive and negative values of theta1 and theta2 (i.e., the cosine of the corresponding included angle). The judgment principle is as Figure 5 shown.

[0100] The judgment criterion is: if theta1 < 0, then judge that the vertex of this triangle is outside the first layer and is an invalid point; otherwise, if theta2 > 0, then judge that the vertex of this triangle is outside the last layer and is also an invalid point; only when theta1 ≥ 0 and theta2 ≤ 0, then judge that the vertex of this triangle is in the valid area between the first layer and the last layer;

[0101] S423. Judge the set S of triangles containing these vertices: if any vertex of a certain triangle is in the invalid area, then discard this triangle, and then reconstruct a set of surface meshes from all the remaining triangles to form the final 3D model, as Figure 6 shown.

[0102] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0103] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for modeling body surface contours, characterized in that: It includes the following steps: S1. Load the original medical image and perform preprocessing; S2. In each layer of the preprocessed medical image, detect the skin surface contour respectively, and remove the redundant components outside the main cross-section area; S3. Integrate the skin surface contours of all layers and preliminarily construct an overall 3D body surface model; S3 includes: S31. Take the points on the surface contour lines of every two adjacent layers as the vertices of the 3D model, extend them in a clockwise direction to form a closed loop, and construct an annular triangular strip; S31 includes: S311. Denote the set of contour points of adjacent layer 1 as arr1, the set of contour points of layer 2 as arr2, and the unit length direction vector perpendicular to the scanning plane as Z_norm; S312. Find the point P1 with the minimum Y coordinate value from the set arr1, take it as the starting point, and arrange the other contour points in arr1 in the original order behind it to form a new point set pts1; S313. Find the point Q1 with the minimum Euclidean distance from the set arr2 to P1, take it as the starting point, and arrange the other contour points in arr2 in the original order behind it to form a new point set pts2; S314. Assume that the point after P1 is P2, the direction vector from P1 to P2 is denoted as R1 = P2 - P1, and its length is denoted as D1; the point where the line starting from P1 along the Z_norm direction intersects the image plane of layer 2 is denoted as H1, H1 = P1 + Z_norm * layer spacing; S315. Assume that the point after Q1 is Q2, the direction vector from H1 to Q2 is denoted as R2 = Q2 - H1, and its vector length is denoted as D2; S316. Take the dot product of vector R1 and vector R2, and the result represents the cosine of the angle between the two, denoted as theta; if theta > 0 and D1 < D2, then take P2 as the selected point, otherwise take Q2 as the selected point; form a triangle with the selected point, P1, and Q1, and put it in a set S; S317. In S316, if the selected point is P2, then take it as the new starting point P1 in pts1; if the selected point is Q2, then take it as the new starting point Q1 in pts2; S318. Repeat the above steps S313 - S317 until all the contour points in pts1 and pts2 are traversed and the process ends. At this time, all the triangles in the set S form the triangular strip between layer 1 and layer 2; S32. Merge the triangular strips constructed for all adjacent layers together to form a preliminary 3D body surface model; S4. Smooth the preliminarily constructed 3D body surface model to obtain the final 3D body surface model, S4 includes: S41. Use the surface mesh smoothing algorithm to smooth the preliminarily constructed 3D body surface model; S42. Remove the redundant parts through the edge trimming post-processing method to obtain the final 3D body surface model, S42 includes: S421, for each vertex constituting a triangle in the initially constructed body surface 3D model, subtract it from the coordinates of the image plane origin in the first layer and the last layer of metadata of the entire volume data, respectively, to construct two vectors, which are recorded as V1 and V2 respectively; S422, do dot multiplication of the vectors V1 and V2 with the normal vector Z_norm along the Z axis, record them as theta1 and theta2 respectively, and judge whether the vertex of the triangle is in the valid area or in the invalid area outside the upper layer / lower layer according to the positive and negative values ​​of theta1 and theta2; The judgment criteria are: if theta1<0, the triangle vertex is judged to be outside the first layer and is an invalid point; otherwise, if theta2>0, the triangle vertex is judged to be outside the last layer and is also an invalid point; only when theta1≥0 and theta2≤0, the triangle vertex is judged to be in the valid area between the first and last layers; S423. Make a judgment on the triangle set S containing these vertices: if any vertex of a triangle is in an invalid area, discard the triangle, and then reconstruct a surface mesh set with all the remaining triangles to form a final body surface 3D model.

2. The method for modeling body surface contour according to claim 1, characterized in that: S1 includes: S11, reading a medical image in a preset format; S12, loading the data of each layer of the medical image to form volume data; S13, parsing corresponding metadata from the volume data, wherein the metadata at least includes the origin coordinates, direction vector, pixel spacing and layer spacing of each layer; S14, preprocessing the loaded original medical images.

3. The method for modeling body surface contour according to claim 2, characterized in that: In S14, the method of preprocessing the original medical image data includes: normalizing the grayscale distribution range of pixels in the original medical image, and filtering to remove isolated noise points.

4. The method for modeling body surface contour according to claim 1, characterized in that S2 include: S21, detecting all possible edge lines for each layer in the preprocessed medical image; S22, removing redundant components outside the main area of ​​the medical image; S23, using convex hull operation to obtain an outer contour line containing all edge lines in the main area.

5. The method for modeling body surface contour according to claim 4, characterized in that: S22 includes: S221, project all edge lines in the horizontal and vertical directions respectively, set the places with edge lines to 1, and set the other places to 0, and take the union; S222, searching for all segments with consecutive values ​​of 1 on the two axes respectively, and taking the longest segment; S223. The rectangular area where the longest sections of the vertical axis and the horizontal axis intersect is retained as the candidate area of ​​the target, and the contour lines outside the candidate area are discarded, and the redundant parts except the main area are removed.

6. The method for modeling body surface contour according to claim 1, characterized in that: In S41, the surface mesh smoothing algorithm adopted is Poisson resampling method, Taubin filtering or rolling ball method.

Images