Target surface reconstruction method, device and equipment based on three-dimensional image and medium

Abstract

The application relates to a target surface reconstruction method, device and equipment based on three-dimensional images and a medium. The method comprises the following steps: acquiring initial three-dimensional image data, and segmenting a target in the initial three-dimensional image data to obtain a segmentation mask; processing the segmentation mask to obtain a target contour; and performing surface reconstruction on the initial three-dimensional image data based on the target contour to obtain surface reconstruction data of the target. The application also relates to a target surface reconstruction method based on three-dimensional images, which comprises the following steps: obtaining surface reconstruction data of the target based on the foregoing method; performing surface reconstruction on the segmentation mask to obtain initial reconstruction data; and obtaining final surface reconstruction data of the target according to the initial reconstruction data and the surface reconstruction data of the target. The method can improve the reconstruction accuracy.

Description

[0001] Note: This application is a divisional application filed in response to the parent application with application number "202110228811.7", application date "2021-03-02", and invention title "Target Surface Reconstruction Method, Apparatus, Device and Medium Based on Three-Dimensional Image". Technical Field

[0002] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and medium for target surface reconstruction based on three-dimensional images. Background Technology

[0003] With the development of computer technology, the 3D surface (mesh) reconstruction of target organs in medical images such as CT or MR has become increasingly sophisticated. Mesh reconstruction of targets can be used for 3D visualization; furthermore, mesh data can serve as a flexible and lightweight data storage method for recording the morphology and position of targets with high precision, and for quantitative calculation of target characteristic parameters. It can also represent the motion or deformation of targets through extremely convenient calculations, which can be used for dynamic real-time tracking of target objects. These advantages of mesh reconstruction determine its extremely wide range of applications in the fields of medical imaging-assisted diagnosis and treatment, such as 3D visualization, position and shape recording, quantitative parameter calculation, and real-time tracking of target organs, tissues, or lesions.

[0004] Traditional methods for reconstructing 3D medical images involve first segmenting the target structure in CT or MR images using image segmentation algorithms to obtain a segmentation mask; then, using the segmentation mask, a Marching Cube algorithm is applied directly to reconstruct the mesh. However, the reconstruction accuracy of this segmented mesh depends entirely on the accuracy of the segmentation algorithm. Because the segmentation result is still stored in the form of a 3D matrix, this discrete storage method limits the segmentation accuracy to an upper limit of one pixel interval, thus limiting the accuracy of the reconstructed mesh. Furthermore, because the edges of the segmentation mask exhibit a stepped shape (pixel mesh effect), the reconstructed mesh also appears stepped. Although this phenomenon can be eliminated through mesh smoothing, it leads to the loss of detail in the reconstructed target structure.

[0005] To address the aforementioned issues, traditional techniques extract edge pixel coordinates from the contour of a segmented mask in CT images, converting them into point cloud data. This data is then reconstructed using point cloud mesh reconstruction methods, such as Poisson surface reconstruction, with the contour's normal direction serving as a constraint to aid in mesh reconstruction. However, because edge pixels remain discrete coordinate points, the mesh reconstruction accuracy cannot exceed the single-pixel limit, resulting in low target accuracy after reconstruction. Summary of the Invention

[0006] Therefore, it is necessary to provide a method, apparatus, device, and medium for target surface reconstruction based on three-dimensional images that can improve reconstruction accuracy in response to the above-mentioned technical problems.

[0007] A method for reconstructing a target surface based on three-dimensional images, the method comprising:

[0008] Acquire initial 3D image data, and segment the targets in the initial 3D image data to obtain a segmentation mask;

[0009] The target contour is obtained by processing the segmentation mask;

[0010] The surface reconstruction data of the target is obtained by performing surface reconstruction on the initial three-dimensional image data based on the target contour.

[0011] In one embodiment, the step of performing surface reconstruction on the initial three-dimensional image data based on the target contour to obtain the surface reconstruction data of the target includes:

[0012] Based on the target contour, extract the three-dimensional image data to be processed from the initial three-dimensional image data;

[0013] The surface reconstruction data of the target is obtained by performing surface reconstruction on the three-dimensional image data to be processed.

[0014] In one embodiment, the step of extracting the three-dimensional image data to be processed from the initial three-dimensional image data based on the target contour includes:

[0015] Pixel dilation is performed on the target contour to determine the three-dimensional image data to be processed.

[0016] In one embodiment, the step of performing surface reconstruction on the three-dimensional image data to be processed to obtain the surface reconstruction data of the target includes:

[0017] Obtain the image grayscale value corresponding to the three-dimensional image data to be processed;

[0018] Based on the image grayscale values, isosurfaces are constructed using the moving cube interpolation method.

[0019] The surface reconstruction data of the target is obtained by performing surface reconstruction on the three-dimensional image data to be processed based on the constructed isosurface.

[0020] In one embodiment, constructing an isosurface using moving cube interpolation based on the image grayscale values ​​includes:

[0021] Obtain the grayscale value of the target contour and the background grayscale value of the background corresponding to the target contour;

[0022] A fixed threshold is determined based on the outline grayscale value and the background grayscale value;

[0023] Based on the fixed threshold, isosurfaces are constructed using the moving cube interpolation method.

[0024] In one embodiment, constructing an isosurface using moving cube interpolation based on the image grayscale values ​​includes:

[0025] Obtain grayscale information of pixels whose distance from the target contour is less than or equal to a preset distance;

[0026] An adaptive threshold is calculated based on the grayscale information;

[0027] Based on the adaptive threshold, isosurfaces are constructed using the moving cube interpolation method.

[0028] A method for reconstructing a target surface based on three-dimensional images, the method comprising:

[0029] The surface reconstruction data of the target is obtained based on the above method;

[0030] Initial reconstruction data is obtained by performing surface reconstruction on the segmentation mask;

[0031] The final surface reconstruction data of the target is obtained based on the initial reconstruction data and the surface reconstruction data of the target.

[0032] In one embodiment, obtaining the final surface reconstruction data of the target based on the initial reconstruction data and the surface reconstruction data of the target includes:

[0033] The final surface reconstruction data of the target is obtained by registering the initial reconstruction data and the surface reconstruction data of the target.

[0034] A target surface reconstruction device based on three-dimensional images, the device comprising:

[0035] The segmentation module is used to acquire initial 3D image data and segment the targets in the initial 3D image data to obtain a segmentation mask;

[0036] The target contour extraction module is used to process the segmentation mask to obtain the target contour;

[0037] The first reconstruction module is used to perform surface reconstruction on the initial three-dimensional image data based on the target contour to obtain the surface reconstruction data of the target.

[0038] A target surface reconstruction device based on three-dimensional images, the device comprising:

[0039] The reconstruction data acquisition module is used to obtain surface reconstruction data of the target based on the above-mentioned device;

[0040] The second reconstruction module is used to perform surface reconstruction of the segmentation mask to obtain initial reconstruction data;

[0041] The integrated reconstruction module is used to obtain the final surface reconstruction data of the target based on the initial reconstruction data and the surface reconstruction data of the target.

[0042] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in any of the above embodiments.

[0043] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0044] The aforementioned target surface reconstruction method, apparatus, device, and medium based on three-dimensional images extract the target contour from the segmented mask, thereby reconstructing the surface based on the target contour. It can fit the edge of the leaking part by combining the surrounding information and reconstruct high-precision target surface reconstruction data. Attached Figure Description

[0045] Figure 1 This is an application environment diagram of a target surface reconstruction method based on three-dimensional images in one embodiment;

[0046] Figure 2 This is a flowchart illustrating a target surface reconstruction method based on three-dimensional images in one embodiment;

[0047] Figure 3 This is a schematic diagram of a segmentation mask in one embodiment;

[0048] Figure 4 A schematic diagram of surface reconstruction data of a target in one embodiment;

[0049] Figure 5 This is a grayscale image from one embodiment;

[0050] Figure 6 This is a flowchart illustrating a target surface reconstruction method based on three-dimensional images in another embodiment;

[0051] Figure 7 This is a schematic diagram of the initial reconstruction data in one embodiment;

[0052] Figure 8 A schematic diagram of the final surface reconstruction data of the target in one embodiment;

[0053] Figure 9 This is a flowchart illustrating a target surface reconstruction method based on three-dimensional images in another embodiment;

[0054] Figure 10 This is a structural block diagram of a target surface reconstruction device based on three-dimensional images in one embodiment;

[0055] Figure 11 This is a structural block diagram of a target surface reconstruction device based on three-dimensional images in another embodiment;

[0056] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] The target surface reconstruction method based on 3D imagery provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with medical imaging device 104 via a network. Terminal 102 can receive initial 3D image data scanned by medical imaging device 104, or obtain initial 3D image data scanned by medical imaging device 104 from a database, etc. Then, it segments the target in the initial 3D image data to obtain a segmentation mask, processes the segmentation mask to obtain the target contour, and performs surface reconstruction on the initial 3D image data based on the target contour to obtain the target's surface reconstruction data. In this way, the target contour is extracted from the segmentation mask, and surface reconstruction is performed based on the target contour. It can also combine surrounding information to fit the edge of the leak location, reconstructing high-precision surface reconstruction data.

[0059] The terminal 102 may be, but is not limited to, various personal computers, laptops, smartphones, tablets, portable wearable devices, and functional modules and dedicated circuits of the medical imaging equipment itself. In this embodiment, the terminal 102 may include the patient's mobile terminal device and / or the medical operator's mobile terminal device. The medical imaging equipment 104 includes, but is not limited to, various imaging devices, such as CT imaging equipment (CT: Computed Tomography, which uses a precisely collimated X-ray beam and a highly sensitive detector to perform a series of cross-sectional scans around a part of the human body, and can reconstruct precise three-dimensional images of tumors, etc. through CT scans), magnetic resonance imaging equipment (which is a type of tomographic imaging that uses the magnetic resonance phenomenon to obtain electromagnetic signals from the human body and reconstruct human body information images), positron emission tomography (PET) equipment, positron emission tomography / magnetic resonance imaging (PET / MR) systems, etc.

[0060] In one embodiment, such as Figure 2 As shown, a target surface reconstruction method based on three-dimensional images is provided, which can be applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0061] S202: Acquire initial 3D image data and segment the targets in the initial 3D image data to obtain a segmentation mask.

[0062] Specifically, the initial 3D image data includes 3D medical image data such as CT and MRI, which can be stored as a 3D matrix. Segmentation refers to the process of dividing the initial 3D image data into segments to determine the target structure. Image segmentation techniques include, but are not limited to, image segmentation techniques based on deep learning fully convolutional networks, or traditional machine learning methods (such as segmentation methods based on graph cut, clustering, active contour models, level sets, and thresholding), and manual or semi-automatic segmentation methods using interactive software (such as segmentation using Mimics, ITK-Snap, 3D Slicer, MITK, etc.). Figure 3 This is a schematic diagram of the hip bone segmentation effect in one embodiment.

[0063] Taking the hip bone in a hip replacement surgery as an example, the terminal can segment the hip bone in CT data using image segmentation technology based on a deep learning fully convolutional network, obtaining segmentation mask data for one hip bone, as shown in the example below. Figure 3 As shown.

[0064] S204: Process the segmentation mask to obtain the target contour.

[0065] Specifically, the terminal can extract the contour line by using an edge detection algorithm on the segmentation mask, and the edge contour line is the target contour.

[0066] S206: Surface reconstruction data of the target is obtained by performing surface reconstruction on the initial three-dimensional image data based on the target contour.

[0067] Specifically, surface reconstruction based on the target contour utilizes the initial 3D image data. This initial 3D image data contains rich pixel information, such as pixel grayscale information. Through 3D linear interpolation, the more precise position of vertices on the preset isosurface can be obtained, thereby improving the accuracy of surface reconstruction.

[0068] In this embodiment, compared to the segmentation mask, the initial 3D image data contains complete pixel grayscale information. By setting the grayscale value of the target structure edge as the isosurface reconstruction threshold, and combining it with the position of the target contour determined by the segmentation mask, the initial 3D image data can be reconstructed within the vicinity of the target contour of the segmentation mask, resulting in more accurate target surface reconstruction data. Specifically, the target surface reconstruction data can be found in [reference needed]. Figure 4 As shown.

[0069] The aforementioned target surface reconstruction method based on 3D images extracts the target contour from the segmentation mask, and then performs surface reconstruction based on the target contour. It can fit the edge of the leaking part by combining the surrounding information and reconstruct high-precision surface reconstruction data.

[0070] In one embodiment, surface reconstruction of the initial three-dimensional image data based on the target contour is used to obtain surface reconstruction data of the target, including: extracting three-dimensional image data to be processed from the initial three-dimensional image data based on the target contour; and performing surface reconstruction of the three-dimensional image data to be processed to obtain surface reconstruction data of the target.

[0071] Specifically, in this embodiment, since the initial three-dimensional image data is very complex, without constraints, a large number of isosurfaces with set thresholds may be reconstructed, making it difficult to extract only the surface of the target. Therefore, the terminal extracts the three-dimensional image data to be processed from the initial three-dimensional image data based on the target contour. The three-dimensional image data to be processed is only a part of the initial three-dimensional image data. Specifically, the three-dimensional image data to be processed can refer only to the image data near the target contour in the initial three-dimensional image data.

[0072] Optionally, extracting the three-dimensional image data to be processed from the initial three-dimensional image data based on the target contour may include: performing pixel dilation on the target contour to determine the three-dimensional image data to be processed. For example, after extracting the target contour, the terminal can perform pixel dilation on the target contour in various dimensions of three-dimensional space, such as performing [N] pixel dilation on each of the three dimensions. x N y N z The dilation of ] pixels, where N x N y N z The value of N is related to the pixel resolution of the specific image, and is usually a positive number between 0 and 10. In this embodiment, the pixel spacing of the image is 1 mm in three directions, and N... x N y N z All values ​​are set to 3, meaning that Marching Cube reconstruction is performed within a 3mm radius of the outline.

[0073] In the above embodiments, constraints were first applied based on the target contour, thereby avoiding the reconstruction of a large number of isosurfaces with set thresholds and simplifying the complexity of peeling off the surface portion of the target.

[0074] In one embodiment, surface reconstruction of the three-dimensional image data to be processed to obtain surface reconstruction data of the target includes: acquiring the image grayscale value corresponding to the three-dimensional image data to be processed; constructing an isosurface based on the image grayscale value using the moving cube interpolation method; and performing surface reconstruction of the three-dimensional image data to be processed according to the constructed isosurface to obtain surface reconstruction data of the target.

[0075] Specifically, image grayscale values ​​refer to the grayscale information corresponding to the 3D image data to be processed. Compared to the segmentation mask, the initial 3D image data has grayscale information, meaning the 3D image data to be processed also has grayscale information. For details, please refer to... Figure 5 As shown, the terminal constructs an isosurface using the moving cube interpolation method based on the grayscale value of the image. That is, the terminal selects an appropriate isosurface reconstruction threshold and then performs surface reconstruction on the three-dimensional image data to be processed based on the constructed isosurface to obtain the surface reconstruction data of the target.

[0076] In the above embodiments, the initial three-dimensional image data is used, and the target contour of the segmentation mask is combined with the Marching Cube algorithm in the area around the contour. The gray values ​​of the target structure edges are used to reconstruct the isosurface, resulting in higher precision target surface reconstruction data.

[0077] In one embodiment, an isosurface is constructed using moving cube interpolation based on image grayscale values, including: obtaining the contour grayscale value corresponding to the target contour and the background grayscale value of the background corresponding to the target contour; determining a fixed threshold based on the contour grayscale value and the background grayscale value; and constructing an isosurface using moving cube interpolation based on the fixed threshold.

[0078] In one embodiment, an isosurface is constructed using moving cube interpolation based on image grayscale values, including: acquiring grayscale information of pixels whose distance from the target contour is less than or equal to a preset distance; calculating an adaptive threshold based on the grayscale information; and constructing an isosurface using moving cube interpolation based on the adaptive threshold.

[0079] Specifically, in the two embodiments described above, the main difference lies in the threshold used for isosurface reconstruction: one is a fixed threshold, and the other is an adaptive threshold. The threshold selection depends on the grayscale definition of the edge of the target structure to be reconstructed, and the selection criterion is to choose a value between the contour grayscale value and the background grayscale value. Therefore, on the one hand, a fixed threshold can be used; for example, 150 HU can be selected for the bone structure reconstruction described above. On the other hand, for better stability, an adaptive threshold can be used. The terminal can statistically analyze the grayscale information of points within a certain range near the target contour and extract the median grayscale value as the adaptive threshold. Here, "near the target contour" can refer to the three-dimensional image data to be processed after image dilation processing.

[0080] In the above embodiments, on the one hand, a fixed threshold can be used to construct the isosurface, and on the other hand, an adaptive threshold can be used, thereby ensuring stability.

[0081] In one embodiment, such as Figure 6 As shown, a target surface reconstruction method based on three-dimensional images is provided, which can be applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0082] S602: Obtain surface reconstruction data of the target based on the method in any of the above embodiments.

[0083] Specifically, the method for generating the surface reconstruction data of the target can be found above, and will not be repeated in this embodiment.

[0084] S604: Perform surface reconstruction on the segmented mask to obtain initial reconstruction data.

[0085] Specifically, the initial reconstruction data obtained by surface reconstruction of the segmentation mask can be performed directly on the segmentation mask. This initial reconstruction data has a complete target structure. For details, please refer to [link to relevant documentation]. Figure 4As shown. The algorithms for surface reconstruction of the segmented mask can include, but are not limited to, using the classic Marching Cube algorithm to reconstruct the contour of the mask data, or using the Poisson surface reconstruction algorithm to reconstruct the mesh data using the point cloud formed by all points on the contour, etc.

[0086] The classic Marching Cube algorithm is used as an example for illustration. The terminal selects an appropriate threshold at the contour edge to reconstruct the isosurface, and obtains the initial reconstruction data of the complete target structure with segmentation mask reconstruction.

[0087] Specifically, the terminal sets the pixel values ​​of the target area to 1 and the pixel values ​​of non-target areas to 0. Then, it uses the Marching Cube algorithm with a contour reconstruction threshold of 0.5 and a segmentation mask as input to reconstruct the mesh surface. Taking the femur as an example, the reconstruction results can be found in [reference needed]. Figure 7 The isosurface reconstruction threshold can be any value between 0 and 1.

[0088] Because this segmentation mask uses a discrete representation of the segmentation results—meaning each pixel is either 0 or 1—its precision is limited. Therefore, the initial reconstructed data obtained by directly using the segmentation mask will not be highly accurate. Figure 7 The overall appearance of the first reconstructed data is also poor, showing a stepped artifact caused by the discrete pixel form.

[0089] S606: Obtain the final surface reconstruction data of the target based on the initial reconstruction data and the target's surface reconstruction data.

[0090] Specifically, the initial reconstruction data obtained by the terminal includes the complete target structure, while the target's surface reconstruction data is high-precision. Therefore, the terminal can use the high-precision target surface reconstruction data as the target mesh data and the initial reconstruction data including the complete target structure as the floating mesh data. Then, it can use registration techniques, such as elastic registration, to register the floating mesh data to the target mesh data, thereby obtaining high-precision final surface reconstruction data with a complete target structure. For details, please refer to [link to documentation]. Figure 8 As shown,

[0091] The above-mentioned target surface reconstruction method based on 3D images extracts the target contour from the segmentation mask, and then performs surface reconstruction based on the target contour. It also combines the target surface reconstruction data obtained from the surface reconstruction based on the segmentation mask, so that there are no discontinuities or leaks at the edges of the target structure. The algorithm can fit the edges of the leaked parts by combining the surrounding information, and reconstruct complete and beautiful final surface reconstruction data.

[0092] In one embodiment, obtaining the final surface reconstruction data of the target based on the initial reconstruction data and the surface reconstruction data of the target includes: registering the initial reconstruction data and the surface reconstruction data of the target to obtain the final surface reconstruction data of the target.

[0093] Specifically, in this embodiment, the terminal uses grid or point cloud registration technology to register the high-precision target surface reconstruction data as the target grid data and the initial reconstruction data including the complete target structure as the floating grid data, thereby obtaining the final surface reconstruction data of the high-precision target structure.

[0094] The registration techniques for meshes or point clouds may include, but are not limited to, the CPD (Coherent Point Drift) algorithm, the nonrigid-ICP algorithm, and deep learning-based methods.

[0095] In one embodiment, the final surface reconstruction data of the target is obtained by registering the initial reconstruction data and the surface reconstruction data of the target. This includes: registering the target's surface reconstruction data by iteratively performing viscous transformation and elastic transformation based on the initial reconstruction data and the surface reconstruction data of the target. The degree of viscous transformation and elastic transformation is determined by an attraction function for point pairs that undergo viscous transformation and elastic transformation. The point pairs are the closest corresponding points in the initial and target surface reconstruction data.

[0096] Specifically, in this embodiment, registration is performed using a grid data elastic registration technique and by iteratively executing a viscoelastic transformation. This may include the following steps: First, the terminal finds the nearest corresponding point in the other point cloud for each point in the point clouds corresponding to the two surface reconstruction data. Then, based on a set distance threshold, outlier and non-outlier point pairs are determined. Points with a distance greater than the distance threshold are designated as outlier pairs; otherwise, they are considered non-outlier pairs. Then, the terminal performs viscoelastic and elastic transformations on each point pair until the number of iterations in the above steps meets a preset number of iterations.

[0097] The viscous transformation is defined as displacing each point on the floating grid data or floating point cloud directly in the direction of the corresponding point on the target grid data or target point cloud.

[0098] Elastic transformation is defined as replacing the original coordinate position of each point p in the target grid data or target point cloud with the weighted average of the coordinate positions of its N nearest neighboring points in the same target grid data or target point cloud. This is equivalent to performing a smoothing operation on the position of each point p in the target grid data or target point cloud. Here, the weight of each neighboring point is determined based on the distance from the neighboring point to point p; the closer the distance, the greater the weight. In this example, it is defined as the Gaussian radial basis function of the distance to point p.

[0099] In this embodiment, to ensure better elastic registration, an attraction function is defined for each point pair undergoing viscous and elastic transformations. The attraction value determines the degree of transformation; for point pairs with low attraction, the viscous and elastic transformations are relatively smoother. The specific attraction function is defined by the following formula:

[0100]

[0101] T and F represent the i-th vertex in the target grid data or target point cloud and the j-th vertex in the floating grid data or floating point cloud, respectively. Position represents the coordinates of the vertex, and Normal is the normalized normal vector of the point. The first term f describes the spatial Euclidean distance correlation function between the two points. The larger the distance, the smaller this term is, and the smaller the attraction. Gaussian radial basis function, etc. can be selected. The second term g describes the consistency of the normal vectors of the point pair. The more consistent the normal vector directions are, the larger the dot product result is. The larger g is, the greater the attraction.

[0102] In this example, g uses the following formula:

[0103]

[0104] For two grid data or power sources, an attraction matrix A needs to be defined. M×N A represents the attraction between any pair of points in the target mesh data or target point cloud and the floating mesh data or floating point cloud. M×N The matrix formula is defined as follows:

[0105] A M×N (i,j)=Affinity i,j

[0106] Where M and N represent the number of vertices in the target mesh data or target point cloud and the floating mesh data or floating point cloud, respectively.

[0107] In practical use, the attraction matrix is ​​multiplied by the vertex matrix of the target mesh data or target point cloud to obtain a new target mesh data or target point cloud after weighted summation based on attraction. This new target mesh data or target point cloud is then subjected to the aforementioned elastic registration operation with the floating mesh data or floating point cloud. This registers the floating mesh data or floating point cloud to points on the target mesh data or target point cloud with high attraction to the floating mesh data or floating point cloud. The specific operation is as follows:

[0108] Position_′ 3, =Position_T 3, · M,n

[0109] In the formula, Position_T is a 3-row, M-column target mesh data or target point cloud vertex matrix, where each column represents the 3D coordinates of each vertex; Position_T' is a new target mesh data or target point cloud vertex matrix, 3 rows and N columns, after weighted summation based on the attraction matrix, where each column represents the 3D coordinates of each new vertex. Finally, Position_T' and the vertices of the floating mesh data or floating point cloud are used for elastic registration.

[0110] In the above embodiments, the attraction degree can reflect the outlier degree of a point. For points with high outlier degree (no suitable matching point found), the viscous transformation is weak, but due to the existence of elastic transformation, the transformation of non-outlier points will be transmitted to outlier points. For targets with surface leaks, i.e., missing parts in the reconstructed high-precision target surface reconstruction data, the corresponding positions of the floating mesh data or floating point cloud form outliers because there is no reference. However, through iterative viscoelastic transformation, these outliers will fit a reasonable deformation field based on the movement of surrounding points, so that the floating mesh data or floating point cloud can fit the target boundary of the leaking part well even without a reference.

[0111] Specifically, see Figure 9 As shown, Figure 9The diagram below illustrates the flowchart of a target surface reconstruction method based on 3D images in another embodiment. In this embodiment, initial 3D image data is first imported, and then the terminal performs image segmentation on the target in the initial 3D image data to obtain a segmentation mask. On one hand, the terminal uses the segmentation mask to directly apply the Marching Cube algorithm to the segmentation mask, selecting appropriate thresholds at the contour edges to perform isosurface reconstruction, obtaining initial reconstructed data of the complete target structure with the segmentation mask reconstruction. On the other hand, the processing can be performed serially or in parallel. Using the initial 3D image data and combining the contour information of the segmentation mask, the Marching Cube algorithm is applied to the area surrounding the contour, using appropriate grayscale values ​​of the target structure edges to perform isosurface reconstruction, obtaining more accurate target structure surface reconstruction data.

[0112] Finally, the terminal uses the surface reconstruction data of the high-precision target as the target grid data and the complete initial reconstruction data as the floating grid data. Using the surface elastic registration technique, the complete initial reconstruction data is registered to the surface reconstruction data of the high-precision target to obtain the final surface reconstruction data of the high-precision complete target structure.

[0113] The above-mentioned target surface reconstruction method based on 3D images extracts the target contour from the segmentation mask, and then performs surface reconstruction based on the target contour. It also combines the target surface reconstruction data obtained from the surface reconstruction based on the segmentation mask, so that there are no discontinuities or leaks at the edges of the target structure. The algorithm can fit the edges of the leaked parts by combining the surrounding information, and reconstruct complete and beautiful final surface reconstruction data.

[0114] It should be understood that, although Figure 2 , Figure 6 and Figure 9 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 , Figure 6 and Figure 9 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0115] In one embodiment, such as Figure 10As shown, a target surface reconstruction device based on three-dimensional images is provided, including: a segmentation module 100, a target contour extraction module 200, and a first reconstruction module 300, wherein:

[0116] The segmentation module 100 is used to acquire initial three-dimensional image data and segment the targets in the initial three-dimensional image data to obtain a segmentation mask;

[0117] The target contour extraction module 200 is used to process the segmentation mask to obtain the target contour;

[0118] The first reconstruction module 300 is used to perform surface reconstruction on the initial three-dimensional image data based on the target contour to obtain the surface reconstruction data of the target.

[0119] In one embodiment, the first reconstruction module 300 described above may include:

[0120] The extraction unit is used to extract the three-dimensional image data to be processed from the initial three-dimensional image data based on the target contour.

[0121] The reconstruction unit is used to perform surface reconstruction on the 3D image data to be processed to obtain the surface reconstruction data of the target.

[0122] In one embodiment, the extraction unit is used to perform pixel dilation on the target contour to determine the three-dimensional image data to be processed.

[0123] In one embodiment, the reconstruction unit may include:

[0124] The grayscale value determination subunit is used to obtain the image grayscale value corresponding to the three-dimensional image data to be processed;

[0125] Isosurface construction sub-units are used to construct isosurfaces based on image grayscale values ​​using the moving cube interpolation method;

[0126] The reconstruction sub-unit is used to perform surface reconstruction on the three-dimensional image data to be processed based on the constructed isosurface to obtain the surface reconstruction data of the target.

[0127] In one embodiment, the isosurface construction subunit includes:

[0128] The first grayscale value acquisition subunit is used to acquire the contour grayscale value corresponding to the target contour, and the background grayscale value of the background corresponding to the target contour.

[0129] A fixed threshold determination subunit is used to determine a fixed threshold based on the outline grayscale value and the background grayscale value;

[0130] The first isosurface is constructed using subunits, which are used to construct isosurfaces based on a fixed threshold using the moving cube interpolation method.

[0131] In one embodiment, the aforementioned isosurface construction subunit includes:

[0132] The second grayscale value acquisition subunit is used to acquire grayscale information of pixels whose distance from the target contour is less than or equal to a preset distance;

[0133] The adaptive threshold determination subunit is used to calculate the adaptive threshold based on grayscale information;

[0134] The second isosurface is constructed using the moving cube interpolation method based on an adaptive threshold.

[0135] In one embodiment, such as Figure 11 As shown, a target surface reconstruction device based on three-dimensional images is provided, including: a reconstruction data acquisition module 400, a second reconstruction module 500, and a comprehensive reconstruction module 600, wherein:

[0136] The reconstruction data acquisition module 400 is used to obtain surface reconstruction data of the target based on the apparatus in any of the above embodiments.

[0137] The second reconstruction module 500 is used to perform surface reconstruction on the segmentation mask to obtain initial reconstruction data.

[0138] The integrated reconstruction module 600 is used to obtain the final surface reconstruction data of the target based on the initial reconstruction data and the target's surface reconstruction data.

[0139] In one embodiment, the above-described integrated reconstruction module 600 is used to register the initial reconstruction data and the surface reconstruction data of the target to obtain the final surface reconstruction data of the target.

[0140] Specific limitations regarding the target surface reconstruction device based on 3D images can be found in the limitations of the target surface reconstruction method based on 3D images mentioned above, and will not be repeated here. Each module in the aforementioned target surface reconstruction device based on 3D images can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0141] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a target surface reconstruction method based on 3D images. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0142] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0143] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring initial three-dimensional image data and segmenting the target in the initial three-dimensional image data to obtain a segmentation mask; processing the segmentation mask to obtain a target contour; and performing surface reconstruction on the initial three-dimensional image data based on the target contour to obtain surface reconstruction data of the target.

[0144] In one embodiment, the surface reconstruction of the target based on the target contour of the initial three-dimensional image data, implemented by the processor executing the computer program, includes: extracting the three-dimensional image data to be processed from the initial three-dimensional image data based on the target contour; and performing surface reconstruction on the three-dimensional image data to be processed to obtain the surface reconstruction data of the target.

[0145] In one embodiment, the extraction of three-dimensional image data to be processed from initial three-dimensional image data based on a target contour, implemented by the processor executing a computer program, includes: performing pixel dilation on the target contour to determine the three-dimensional image data to be processed.

[0146] In one embodiment, the surface reconstruction of the target obtained by the processor executing the computer program to perform surface reconstruction on the three-dimensional image data to be processed includes: acquiring the image grayscale value corresponding to the three-dimensional image data to be processed; constructing an isosurface based on the image grayscale value using the moving cube interpolation method; and performing surface reconstruction on the three-dimensional image data to be processed according to the constructed isosurface to obtain the surface reconstruction data of the target.

[0147] In one embodiment, the process of constructing an isosurface based on image grayscale values ​​using moving cube interpolation, implemented by the processor executing a computer program, includes: obtaining the contour grayscale value corresponding to the target contour and the background grayscale value of the background corresponding to the target contour; determining a fixed threshold based on the contour grayscale value and the background grayscale value; and constructing an isosurface using moving cube interpolation based on the fixed threshold.

[0148] In one embodiment, the process of constructing an isosurface based on image grayscale values ​​using the moving cube method when the processor executes a computer program includes: acquiring grayscale information of pixels whose distance from the target contour is less than or equal to a preset distance; calculating an adaptive threshold based on the grayscale information; and constructing an isosurface using the moving cube method based on the adaptive threshold.

[0149] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: obtaining surface reconstruction data of a target based on the method in any of the above embodiments; performing surface reconstruction on a segmented mask to obtain initial reconstruction data; and obtaining final surface reconstruction data of the target based on the initial reconstruction data and the surface reconstruction data of the target.

[0150] In one embodiment, the final surface reconstruction data of a target obtained by the processor executing a computer program based on initial reconstruction data and the surface reconstruction data of the target includes: registering the first reconstruction data and the surface reconstruction data of the target to obtain the surface reconstruction data of the target.

[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps: acquiring initial three-dimensional image data and segmenting the target in the initial three-dimensional image data to obtain a segmentation mask; processing the segmentation mask to obtain a target contour; and performing surface reconstruction on the initial three-dimensional image data based on the target contour to obtain surface reconstruction data of the target.

[0152] In one embodiment, when a computer program is executed by a processor, it performs surface reconstruction on initial three-dimensional image data based on the target contour to obtain surface reconstruction data of the target, including: extracting three-dimensional image data to be processed from the initial three-dimensional image data based on the target contour; and performing surface reconstruction on the three-dimensional image data to be processed to obtain surface reconstruction data of the target.

[0153] In one embodiment, the extraction of three-dimensional image data to be processed from initial three-dimensional image data based on a target contour, implemented when the computer program is executed by a processor, includes: performing pixel dilation on the target contour to determine the three-dimensional image data to be processed.

[0154] In one embodiment, when a computer program is executed by a processor, it performs surface reconstruction on the three-dimensional image data to be processed to obtain the surface reconstruction data of the target, including: acquiring the image grayscale value corresponding to the three-dimensional image data to be processed; constructing an isosurface based on the image grayscale value using the moving cube interpolation method; and performing surface reconstruction on the three-dimensional image data to be processed according to the constructed isosurface to obtain the surface reconstruction data of the target.

[0155] In one embodiment, the process of constructing an isosurface based on image grayscale values ​​using moving cube interpolation, implemented by a computer program executed by a processor, includes: obtaining the contour grayscale value corresponding to the target contour and the background grayscale value of the background corresponding to the target contour; determining a fixed threshold based on the contour grayscale value and the background grayscale value; and constructing an isosurface using moving cube interpolation based on the fixed threshold.

[0156] In one embodiment, the process of constructing an isosurface based on image grayscale values ​​using moving cube interpolation when the computer program is executed by a processor includes: acquiring grayscale information of pixels whose distance from the target contour is less than or equal to a preset distance; calculating an adaptive threshold based on the grayscale information; and constructing an isosurface using moving cube interpolation based on the adaptive threshold.

[0157] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps: obtaining surface reconstruction data of a target based on the method in any of the above embodiments; performing surface reconstruction on a segmented mask to obtain initial reconstruction data; and obtaining final surface reconstruction data of the target based on the initial reconstruction data and the surface reconstruction data of the target.

[0158] In one embodiment, the process of obtaining the final surface reconstruction data of a target based on initial reconstruction data and the surface reconstruction data of the target when the computer program is executed by the processor includes: registering the first reconstruction data and the surface reconstruction data of the target to obtain the surface reconstruction data of the target.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for reconstructing a target surface based on three-dimensional images, characterized by, The method includes: Acquire initial 3D image data, and segment the targets in the initial 3D image data to obtain a segmentation mask; The contour lines of the segmentation mask are extracted using an edge detection algorithm to obtain the target contour. Based on the target contour, extract the three-dimensional image data to be processed from the initial three-dimensional image data, and perform surface reconstruction on the three-dimensional image data to be processed to obtain the surface reconstruction data of the target; Initial reconstruction data is obtained by performing surface reconstruction on the segmentation mask; The final surface reconstruction data of the target is obtained by registration based on the initial reconstruction data and the surface reconstruction data of the target.

2. The method of claim 1, wherein, The step of extracting the three-dimensional image data to be processed from the initial three-dimensional image data based on the target contour includes: Pixel dilation is performed on the target contour to determine the three-dimensional image data to be processed.

3. The method of claim 1, wherein, The process of performing surface reconstruction on the three-dimensional image data to be processed to obtain the surface reconstruction data of the target includes: Obtain the image grayscale value corresponding to the three-dimensional image data to be processed; Based on the image grayscale values, isosurfaces are constructed using the moving cube interpolation method. The surface reconstruction data of the target is obtained by performing surface reconstruction on the three-dimensional image data to be processed based on the constructed isosurface.

4. The method of claim 3, wherein, The step of constructing an isosurface using the moving cube interpolation method based on the image grayscale values ​​includes: Obtain the grayscale value of the target contour and the background grayscale value of the background corresponding to the target contour; A fixed threshold is determined based on the outline grayscale value and the background grayscale value; Based on the fixed threshold, isosurfaces are constructed using the moving cube interpolation method.

5. The method of claim 3, wherein, The step of constructing an isosurface using the moving cube interpolation method based on the image grayscale values ​​includes: Obtain grayscale information of pixels whose distance from the target contour is less than or equal to a preset distance; An adaptive threshold is calculated based on the grayscale information; Based on the adaptive threshold, isosurfaces are constructed using the moving cube interpolation method.

6. The method of claim 1, wherein, The step of registering the initial reconstruction data and the target's surface reconstruction data to obtain the final surface reconstruction data of the target includes: Based on the initial reconstruction data and the target's surface reconstruction data, the target's surface reconstruction data is obtained by iteratively performing viscous transformation and elastic transformation. For point pairs undergoing viscous transformation and elastic transformation, the degree of viscous transformation and elastic transformation is determined by the attraction function. The point pairs are the closest corresponding points in the initial and target surface reconstruction data.

7. The method of claim 6, wherein, The determination of the degree of viscous and elastic transformation of point pairs undergoing viscous and elastic transformations using an attraction function includes: For each pair of points undergoing viscous and elastic transformations, an attraction function is defined to determine the extent of the viscous and elastic transformations.

8. A target surface reconstruction device based on three-dimensional images, characterized in that, The device includes: The segmentation module is used to acquire initial 3D image data and segment the targets in the initial 3D image data to obtain a segmentation mask; The initial reconstruction data target contour extraction module is used to extract the contour lines of the segmentation mask according to the edge detection algorithm to obtain the target contour. The first reconstruction module is used to extract three-dimensional image data to be processed from the initial three-dimensional image data based on the target contour, and to perform surface reconstruction on the three-dimensional image data to be processed to obtain the surface reconstruction data of the target. The second reconstruction module is used to perform surface reconstruction of the segmentation mask to obtain initial reconstruction data; The integrated reconstruction module is used to register the initial reconstruction data and the surface reconstruction data of the target to obtain the final surface reconstruction data of the target.

9. The apparatus of claim 8, wherein, The first reconstruction module includes: An extraction unit is used to perform pixel dilation on the target contour to determine the three-dimensional image data to be processed.

10. The apparatus of claim 8, wherein, The first reconstruction module includes: The grayscale value determination subunit is used to obtain the image grayscale value corresponding to the three-dimensional image data to be processed. An isosurface construction subunit is used to construct isosurfaces based on the image grayscale values ​​using the moving cube interpolation method; The reconstruction subunit is used to perform surface reconstruction on the three-dimensional image data to be processed based on the constructed isosurface to obtain the surface reconstruction data of the target.

11. The apparatus of claim 10, wherein, The isosurface construction subunit includes: The first grayscale value acquisition subunit is used to acquire the contour grayscale value corresponding to the target contour and the background grayscale value corresponding to the target contour. A fixed threshold determination subunit is used to determine a fixed threshold based on the outline grayscale value and the background grayscale value; The first isosurface construction subunit is used to construct isosurfaces using the moving cube interpolation method based on the fixed threshold.

12. The apparatus of claim 10, wherein, The isosurface construction subunit includes: The second grayscale value acquisition subunit is used to acquire grayscale information of pixels whose distance from the target contour is less than or equal to a preset distance; An adaptive threshold determination subunit is used to calculate an adaptive threshold based on the grayscale information; The second isosurface construction subunit is used to construct isosurfaces using the moving cube interpolation method based on the adaptive threshold.

13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

14. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images