Three-dimensional reconstruction method, device and system based on OCT image

Abstract

The invention discloses a three-dimensional reconstruction method, device and system based on an OCT image, and the method comprises the steps: carrying out the feature extraction of a front-end OCT image and a rear-end OCT image of an obtained sub-target object through an ORB algorithm, obtaining a corresponding front-end feature descriptor and a corresponding rear-end feature descriptor, carrying out the matching calculation of the two feature descriptors, obtaining a fused OCT image, and carrying out the reconstruction of the fused OCT image. And then carrying out space coordinate conversion and smooth denoising processing on the fused OCT image to obtain a three-dimensional reconstruction model of the target object. The OCT image is subjected to feature extraction through the ORB algorithm, the accuracy and precision of the extracted feature symbols are improved, it is also ensured that the extracted feature symbols can describe the image features more accurately, and therefore the authenticity and precision of a three-dimensional model obtained through subsequent reconstruction are improved; and smooth denoising processing is carried out on the reconstructed model, so that the quality and accuracy of the reconstructed model are further improved, the noise in the model is removed, and the detail performance of the model is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of OCT image processing, and in particular to a three-dimensional reconstruction method, device and system based on OCT images. Background Art

[0002] With the advancement of science and the improvement of image processing technology, various medical detection methods and display means have become more accurate and perfect, which helps people observe biological tissues and understand material structures. Its development is the result of the combination of multiple disciplines such as physics, mathematics, electronics, computer science and biomedicine. Among various imaging technologies, Optical Coherence Tomography is one of the technologies based on optical imaging. OCT obtains tomography capabilities in the depth direction based on the principle of low-coherence interference. Through scanning, it can reconstruct two-dimensional or three-dimensional images of the internal structure of biological tissues or materials. Its signal contrast comes from the spatial changes in the optical reflection (scattering) characteristics of biological tissues or materials. Therefore, OCT has the advantages of non-contact, non-invasive, fast imaging speed (real-time dynamic imaging), and high detection sensitivity.

[0003] Existing technologies mostly use surface rendering or volume rendering technology to reconstruct slice OCT images in three dimensions. However, surface rendering technology requires the classification of pixels, so there is a problem of low registration accuracy due to classification errors. Volume rendering technology requires processing all voxels, which has high computational complexity and low processing efficiency due to excessive calculations. Summary of the invention

[0004] The present invention provides a three-dimensional reconstruction method, device and system based on OCT images, so as to achieve the technical effect of improving the registration accuracy and model quality of the three-dimensional reconstructed model while reducing the computational complexity.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a three-dimensional reconstruction method based on OCT images, comprising the following steps:

[0006] Acquire a front-end OCT image and a back-end OCT image of the target object, and respectively perform feature extraction on the front-end OCT image and the back-end OCT image using an ORB algorithm to obtain a front-end feature descriptor and a back-end feature descriptor;

[0007] The front-end feature descriptor and the back-end feature descriptor are matched and calculated to obtain a fused OCT image, and the fused OCT image is sequentially subjected to spatial coordinate transformation processing and smoothing and denoising processing to obtain a three-dimensional reconstructed model of the target object.

[0008] The three-dimensional reconstruction method provided by the present invention extracts features from the acquired front-end OCT images and back-end OCT images through the ORB algorithm, and can obtain corresponding front-end feature descriptors and back-end feature descriptors. The present invention extracts features from the front-end OCT images and back-end OCT images through the ORB feature extraction algorithm, thereby improving the accuracy and precision of the extracted feature descriptors, and also ensuring that the extracted feature descriptors can more accurately describe the image features, thereby improving the authenticity and accuracy of the three-dimensional model obtained by subsequent reconstruction.

[0009] After obtaining the front-end signature and the back-end signature, the system can match and calculate the signatures at both ends, and merge the corresponding front-end OCT image and the back-end OCT image into a fused OCT image according to the matching results of the signatures, and perform spatial coordinate transformation and smoothing denoising on the merged fused OCT image to obtain the corresponding processed 3D reconstructed model. More accurate point cloud data can be obtained through more accurate spatial coordinate transformation; and smoothing denoising on the reconstructed model can further improve the quality and accuracy of the reconstructed model, remove noise in the model, and enhance the detail performance of the model.

[0010] As a preferred example, the extracting features of the front-end OCT image and the back-end OCT image by using the ORB algorithm to obtain the front-end feature descriptor and the back-end feature descriptor includes:

[0011] Performing feature point detection on the front-end OCT image and the back-end OCT image respectively, and outputting a front-end feature point set and a back-end feature point set;

[0012] The front-end feature point set and the back-end feature point set are calculated using a binary algorithm to obtain the front-end feature descriptor and the back-end feature descriptor.

[0013] In order to improve the authenticity and accuracy of the generated model, the three-dimensional reconstruction method provided by the present invention, when extracting features from the front-end OCT image and the back-end OCT image through the ORB algorithm, first performs feature point detection on them, outputs the corresponding front-end feature point set and the back-end feature point set, and calculates the feature points in the two feature point sets obtained by the detection through a binary algorithm to obtain their corresponding feature descriptors, which is convenient for subsequent feature matching of the front-end feature descriptors and the back-end feature descriptors. The binary algorithm is used to calculate the feature points, so that the obtained feature descriptors can more intuitively reflect their feature points, which is convenient for subsequent feature point matching.

[0014] As a preferred example, the performing feature point detection on the front-end OCT image and the back-end OCT image respectively and outputting a front-end feature point set and a back-end feature point set includes:

[0015] Performing fast corner point detection on the front-end OCT image and the back-end OCT image respectively to obtain an initial feature point set, and calculating a Harris response value of each initial feature point in the initial feature point set;

[0016] Each of the initial feature points is sorted according to the calculated Harris response value, and the corresponding front-end feature point set and the back-end feature point set are filtered and output from the initial feature point set according to a preset response value threshold.

[0017] In order to further improve the accuracy and effectiveness of feature descriptors, the three-dimensional reconstruction method provided by the present invention first performs fast corner point detection on the OCT image, determines whether the pixel is a corner point by comparing the brightness difference between each pixel point and its surrounding pixels, and integrates the pixel points determined to be corner points into a feature point set for output, thereby improving the efficiency and accuracy of feature point detection and reducing the time complexity of detection.

[0018] After obtaining the initial feature point set, the system will also calculate the Harris response value, by calculating the gray value change of the feature point, that is, the pixel point, that is, calculating the gradient and second-order matrix of the pixel point gray value, and further screening the front-end feature point set corresponding to the front-end image and the back-end feature point set corresponding to the back-end image from the initial feature point set through the preset threshold. The above two feature point detection algorithms not only improve the accuracy of the output feature point set, but also reduce the complexity of subsequent feature point matching calculations.

[0019] As a preferred example, the using of a binary algorithm to calculate the front-end feature point set and the back-end feature point set to obtain the front-end feature descriptor and the back-end feature descriptor includes:

[0020] The front-end feature point set and the back-end feature point set are filtered by a Gaussian filtering algorithm, and a binary value assignment calculation is performed on the filtered feature point set to output a front-end feature descriptor and a back-end feature descriptor;

[0021] The feature angles of the front-end feature point set and the back-end feature point set are calculated, the front-end feature descriptor and the back-end feature descriptor are adjusted according to the feature angles, the adjusted feature descriptors are used as feature descriptors, and the front-end feature descriptors and the back-end feature descriptors are obtained.

[0022] Before calculating the feature point set through the binary algorithm, the system will also filter the feature point set through the Gaussian filtering algorithm to reduce the noise interference in the feature point set, and also improve the accuracy of the feature descriptors obtained by subsequent calculations.

[0023] After obtaining the feature descriptor, the system will also calculate the feature angle of each pixel, that is, the direction of the feature point, and adjust the corresponding descriptor according to the calculated feature angle, that is, rotate and normalize each pixel, so as to achieve rotation compensation for the feature descriptor and then achieve rotation invariance of the feature descriptor.

[0024] As a preferred example, the matching calculation of the front-end feature descriptor and the back-end feature descriptor to obtain a fused OCT image includes:

[0025] Performing matching calculation on the front-end feature descriptor and the back-end feature descriptor by using a Hamming distance algorithm, and performing abnormal screening and elimination on the matching calculation results to obtain a screened image set to be fused;

[0026] The front-end image in the image set to be fused is fused with the corresponding back-end image to obtain the fused OCT image.

[0027] The front-end image and the back-end image are feature matched by calculating the Hamming distance between the front-end feature symbol and the back-end feature symbol, and the matched results are subjected to abnormal screening and elimination processing to remove erroneous matching results or abnormal matching results in the matching, thereby improving the accuracy of the obtained fused OCT image.

[0028] As a preferred example, the step of sequentially performing spatial coordinate transformation processing and smoothing and denoising processing on the fused OCT image to obtain a three-dimensional reconstructed model of the target object includes:

[0029] performing point cloud data calculation, three-dimensional data reconstruction and model surface reconstruction on the fused OCT image in sequence to obtain an initial three-dimensional model;

[0030] The initial three-dimensional model is sequentially smoothed, denoised and enhanced to obtain the three-dimensional reconstructed model of the target object.

[0031] In order to convert the fused OCT image into a three-dimensional reconstructed model, the system also performs point cloud data processing, three-dimensional data reconstruction, and model surface reconstruction on the image, converts the image into a three-dimensional initial model, and performs noise processing and detail enhancement on the three-dimensional initial model through smoothing, denoising, and enhancement processing, so as to improve the clarity of the quality of the reconstructed model.

[0032] As a preferred example, the step of sequentially performing point cloud data calculation, three-dimensional data reconstruction and model surface reconstruction on the fused OCT image to obtain an initial three-dimensional model includes:

[0033] The coordinates of each pixel in the fused OCT image are converted into three-dimensional space coordinates by a circular scanning algorithm to obtain three-dimensional point cloud data;

[0034] Performing volume rendering on the three-dimensional point cloud data by a ray casting method to obtain a three-dimensional data cube;

[0035] The three-dimensional data cube is subjected to model surface reconstruction processing by using a point cloud surface reconstruction algorithm to obtain the initial three-dimensional model.

[0036] The coordinates of each pixel in the fused image are converted into three-dimensional space coordinates through a circular scanning algorithm to generate corresponding three-dimensional point cloud data, thereby improving the quality and accuracy of the generated three-dimensional point cloud data. The three-dimensional point cloud data is volume rendered through the ray casting method, and the 3D image corresponding to the three-dimensional point cloud data is generated by simulating the propagation path of light in the three-dimensional data cube, and the corresponding three-dimensional data cube is constructed. The model surface of the constructed three-dimensional data cube is reconstructed through the point cloud surface reconstruction algorithm to obtain the initial three-dimensional model.

[0037] The fused OCT images are converted into three-dimensional point cloud data and corresponding three-dimensional data cubes through precise circular scanning algorithms and ray casting methods, which improves the quality and accuracy of the obtained point cloud data and data cubes. The surface of the cube is reconstructed through the point cloud surface reconstruction algorithm, which further improves the reconstructed three-dimensional model.

[0038] As a preferred example, the step of sequentially performing smoothing, denoising and enhancing processing on the initial three-dimensional model to obtain the three-dimensional reconstructed model includes:

[0039] The initial three-dimensional model is processed in sequence by a Laplace smoothing algorithm, a median filtering algorithm and a sharpening filtering algorithm to obtain the three-dimensional reconstructed model of the target object.

[0040] The above three smoothing and filtering algorithms can improve the model quality, enhance the detail expression of the model, remove the noise in the model, and further improve the quality and accuracy of the 3D reconstructed model.

[0041] Accordingly, an embodiment of the present invention further provides a three-dimensional reconstruction device based on OCT images, wherein the three-dimensional reconstruction device includes an image feature extraction module and a three-dimensional model reconstruction module;

[0042] The image feature extraction module is used to obtain a front-end OCT image and a back-end OCT image of the target object, and respectively perform feature extraction on the front-end OCT image and the back-end OCT image to obtain a front-end feature descriptor and a back-end feature descriptor;

[0043] The three-dimensional model reconstruction module is used to perform matching calculation on the front-end feature descriptor and the back-end feature descriptor to obtain a fused OCT image, and perform spatial coordinate transformation processing and smoothing denoising processing on the fused OCT image in sequence to obtain a three-dimensional reconstructed model of the target object.

[0044] Correspondingly, an embodiment of the present invention also provides a three-dimensional reconstruction system based on OCT images, characterized in that the three-dimensional reconstruction system includes: a memory, a processor, and a three-dimensional reconstruction program stored in the memory and executable on the processor, and when the three-dimensional reconstruction program is executed by the processor, the steps of a three-dimensional reconstruction method based on OCT images described in any one of the above are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 : A schematic flow chart of an embodiment of a three-dimensional reconstruction method based on OCT images provided by the present invention;

[0046] Figure 2 : A schematic structural diagram of an embodiment of a three-dimensional reconstruction device based on OCT images provided by the present invention;

[0047] Figure 3 : A flow chart of an embodiment of a method for reconstructing a three-dimensional eyeball model based on OCT images provided by the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Embodiment 1

[0050] Please refer to Figure 1 , is a flow chart of an embodiment of a three-dimensional reconstruction method based on OCT images provided by the present invention, including steps 101 to 102, each of which is specifically as follows:

[0051] Step 101: Acquire a front-end OCT image and a back-end OCT image of a target object, and perform feature extraction on the front-end OCT image and the back-end OCT image respectively by using an ORB algorithm to obtain a front-end feature descriptor and a back-end feature descriptor.

[0052] The three-dimensional reconstruction method provided by the embodiment of the present invention extracts features from the acquired front-end OCT image and the back-end OCT image through the ORB algorithm, and can obtain the corresponding front-end feature descriptor and back-end feature descriptor. The present invention extracts features from the front-end OCT image and the back-end OCT image through the ORB feature extraction algorithm, thereby improving the accuracy and precision of the extracted feature descriptors, and also ensuring that the extracted feature descriptors can more accurately describe the image features, thereby improving the authenticity and accuracy of the three-dimensional model obtained by subsequent reconstruction.

[0053] As a specific example provided in this embodiment, see Figure 3 , Figure 3 A schematic diagram of a flow chart of an embodiment of a method for reconstructing a three-dimensional eyeball model based on OCT images provided by the present invention, as shown in FIG. Figure 3 As shown, Figure 3 The eyeball shown is the target object described in this embodiment. Figure 3 The anterior segment OCT image shown is the front end OCT image described in this embodiment. Figure 3 The posterior segment OCT image shown is the posterior end OCT image described in this embodiment.

[0054] Depend on Figure 3 It can be seen that the system will first collect the length of the eyeball, that is, collect and obtain the length of the eyeball for subsequent reconstruction of the three-dimensional eyeball model. For the anterior segment OCT image, that is, the front end OCT image described in this embodiment, the system obtains the OCT image data of the anterior segment by performing an OCT scan on the front of the eye. Similarly, for the posterior segment OCT image, that is, the posterior end OCT image described in this embodiment, the system also obtains the OCT image data of the posterior end of the eye by performing an OCT scan on the posterior end of the eye. After obtaining the anterior segment OCT image and the posterior end OCT image, the system will perform feature detection and description on the above two images through the ORB algorithm.

[0055] Specifically, in this embodiment, the front-end OCT image and the back-end OCT image are respectively subjected to feature extraction by the ORB algorithm to obtain the front-end feature descriptor and the back-end feature descriptor, including:

[0056] Performing feature point detection on the front-end OCT image and the back-end OCT image respectively, and outputting a front-end feature point set and a back-end feature point set;

[0057] The front-end feature point set and the back-end feature point set are calculated using a binary algorithm to obtain the front-end feature descriptor and the back-end feature descriptor.

[0058] In order to improve the authenticity and accuracy of the generated model, the three-dimensional reconstruction method provided in the embodiment of the present invention first performs feature point detection on the front-end OCT image and the back-end OCT image when extracting features through the ORB algorithm, outputs the corresponding front-end feature point set and the back-end feature point set, and calculates the feature points in the two feature point sets obtained by the detection through a binary algorithm to obtain the corresponding feature descriptors, which is convenient for subsequent feature matching of the front-end feature descriptors and the back-end feature descriptors. The binary algorithm is used to calculate the feature points, so that the acquired feature descriptors can more intuitively reflect their feature points, which is convenient for subsequent feature point matching.

[0059] Furthermore, the present embodiment performs feature point detection on the front-end OCT image and the back-end OCT image respectively, and outputs a front-end feature point set and a back-end feature point set, including:

[0060] Performing fast corner point detection on the front-end OCT image and the back-end OCT image respectively to obtain an initial feature point set, and calculating a Harris response value of each initial feature point in the initial feature point set;

[0061] Each of the initial feature points is sorted according to the calculated Harris response value, and the corresponding front-end feature point set and the back-end feature point set are filtered and output from the initial feature point set according to a preset response value threshold.

[0062] In order to further improve the accuracy and effectiveness of feature descriptors, the three-dimensional reconstruction method provided in the embodiment of the present invention first performs fast corner point detection on the OCT image, determines whether each pixel is a corner point by comparing the brightness difference between each pixel and its surrounding pixels, and integrates the pixel points determined to be corner points into a feature point set for output, thereby improving the efficiency and accuracy of feature point detection and reducing the time complexity of detection.

[0063] After obtaining the initial feature point set, the system will also calculate the Harris response value, by calculating the gray value change of the feature point, that is, the pixel point, that is, calculating the gradient and second-order matrix of the pixel point gray value, and further screening the front-end feature point set corresponding to the front-end image and the back-end feature point set corresponding to the back-end image from the initial feature point set through the preset threshold. The above two feature point detection algorithms not only improve the accuracy of the output feature point set, but also reduce the complexity of subsequent feature point matching calculations.

[0064] In this embodiment, the system first performs fast corner point detection on the image, that is, the FAST (Features from Accelerated Segment Test) algorithm detects feature points in the image, and performs fast corner point detection on each pixel point p in the image, and determines whether the pixel point is a corner point by comparing the brightness difference between each pixel point and other pixel points around it, that is, the remaining 16 pixel points on the circle where the pixel point is located.

[0065] If it is judged to be a corner point, it can be extracted as an initial feature point and added to the initial feature point set. The system will also calculate the Harris response value H(p) for all initial feature points in the initial feature point set. The specific calculation formula is as follows:

[0066] H(p)=Harris(p)

[0067] Each initial feature point is sorted according to the calculated Harris response value, and N initial feature points with response values ​​higher than the preset response value threshold are selected as feature points in the feature point set, while the remaining initial feature points with response values ​​lower than the response value threshold will not be added to the feature point set as feature points.

[0068] Furthermore, the method of using a binary algorithm to calculate the front-end feature point set and the back-end feature point set to obtain the front-end feature descriptor and the back-end feature descriptor in this embodiment includes:

[0069] The front-end feature point set and the back-end feature point set are filtered by a Gaussian filtering algorithm, and a binary value assignment calculation is performed on the filtered feature point set to output a front-end feature descriptor and a back-end feature descriptor;

[0070] The feature angles of the front-end feature point set and the back-end feature point set are calculated, the front-end feature descriptor and the back-end feature descriptor are adjusted according to the feature angles, the adjusted feature descriptors are used as feature descriptors, and the front-end feature descriptors and the back-end feature descriptors are obtained.

[0071] Before calculating the feature point set through the binary algorithm, the system will also filter the feature point set through the Gaussian filtering algorithm to reduce the noise interference in the feature point set, and also improve the accuracy of the feature descriptors obtained by subsequent calculations.

[0072] After obtaining the feature descriptor, the system will also calculate the feature angle of each pixel, that is, the direction of the feature point, and adjust the corresponding descriptor according to the calculated feature angle, that is, rotate and normalize each pixel, so as to achieve rotation compensation for the feature descriptor and then achieve rotation invariance of the feature descriptor.

[0073] In this embodiment, before the system performs binary calculation on the feature points in the feature point set, it will also filter them through Gaussian filtering to reduce noise interference in the feature points. The embodiment of the present invention does not further limit the specific parameter settings of Gaussian filtering, but the preferred parameter settings of this embodiment are: variance is 2, and Gaussian window is 9x9.

[0074] After Gaussian filtering, the system will perform binary calculations on the feature points in the feature point set through a binary algorithm. The specific steps are: take the feature point as the center, take a neighborhood window of SxS, and randomly select a pair (i.e., two) of feature points in the window, compare the sizes of the pixels of the two selected points, and perform binary assignment as shown in the following formula:

[0075]

[0076] After completing the above binary assignment, repeat the above steps, i.e., randomly select N pairs of random points from the selected neighborhood window, and repeat pixel comparison and binary assignment for each pair of selected random points until a corresponding binary code is generated. The generated binary code is the description of the feature point by the binary algorithm, i.e., the feature descriptor described in this embodiment. The embodiment of the present invention does not limit the number of random point pairs, but in this embodiment, N=256 is preferred.

[0077] After generating the feature descriptor, the system will also calculate the feature angle of each feature point, that is, the direction of each feature point. The calculation formula is as follows:

[0078]

[0079] Where, the direction is θ, I(p) is the grayscale value of pixel p, and x and y are the coordinates relative to the corner point. After calculating the characteristic angle of each feature point, the feature descriptor will be adjusted according to the calculated characteristic angle. Specifically, the feature descriptor is rotated and compensated by the calculated characteristic angle, that is, rotation normalization is performed to rotate the image to the direction θ of the feature point, so as to achieve the rotation invariance of the obtained feature symbol.

[0080] Step 102: performing matching calculation on the front-end feature descriptor and the back-end feature descriptor to obtain a fused OCT image, and performing spatial coordinate transformation processing and smoothing denoising processing on the fused OCT image in sequence to obtain a three-dimensional reconstructed model of the target object.

[0081] After obtaining the front-end signature and the back-end signature, the system can match and calculate the signatures at both ends, and merge the corresponding front-end OCT image and the back-end OCT image into a fused OCT image according to the matching results of the signatures, and perform spatial coordinate transformation and smoothing denoising on the merged fused OCT image to obtain the corresponding processed 3D reconstructed model. More accurate point cloud data can be obtained through more accurate spatial coordinate transformation; and smoothing denoising on the reconstructed model can further improve the quality and accuracy of the reconstructed model, remove noise in the model, and enhance the detail performance of the model.

[0082] Specifically, the matching calculation of the front-end feature descriptor and the back-end feature descriptor to obtain the fused OCT image in this embodiment includes:

[0083] Performing matching calculation on the front-end feature descriptor and the back-end feature descriptor by using a Hamming distance algorithm, and performing abnormal screening and elimination on the matching calculation results to obtain a screened image set to be fused;

[0084] The front-end image in the image set to be fused is fused with the corresponding back-end image to obtain the fused OCT image.

[0085] The front-end image and the back-end image are feature matched by calculating the Hamming distance between the front-end feature symbol and the back-end feature symbol, and the matched results are subjected to abnormal screening and elimination processing to remove erroneous matching results or abnormal matching results in the matching, thereby improving the accuracy of the obtained fused OCT image.

[0086] As another example of this embodiment, Figure 3 As shown, the system uses the BFMatcher algorithm to match feature points of the anterior eye segment image and the posterior eye segment image, and fuses the matched anterior eye segment image and posterior eye segment image to generate a complete eyeball image.

[0087] In this embodiment, the system performs feature point matching on the two feature points by calculating the Hamming distance H(d1, d2) between the two feature points. For the two feature descriptors d1 and d2, the matching formula is as follows:

[0088]

[0089] Through the above calculation formula, each front-end feature descriptor can be matched with each back-end feature and descriptor. By traversing all front-end feature descriptors and subsequent feature descriptors, the Hamming distance between each front-end feature descriptor and each back-end feature descriptor can be obtained. By filtering and eliminating the calculated Hamming distance, the feature matching result of the front-end feature descriptor and the back-end feature descriptor can be obtained. Through the above steps, the ORB algorithm can realize the effective extraction, description and matching of OCT image feature points.

[0090] This embodiment does not limit the specific method of screening the matching results, but this embodiment preferably screens the matching results through the KNN (K nearest neighbor) algorithm combined with a ratio test to remove erroneous matching results or abnormal matching results.

[0091] Specifically, the present embodiment sequentially performs spatial coordinate transformation processing and smoothing and denoising processing on the fused OCT image to obtain a three-dimensional reconstructed model of the target object, including:

[0092] performing point cloud data calculation, three-dimensional data reconstruction and model surface reconstruction on the fused OCT image in sequence to obtain an initial three-dimensional model;

[0093] The initial three-dimensional model is sequentially smoothed, denoised and enhanced to obtain the three-dimensional reconstructed model of the target object.

[0094] In order to convert the fused OCT image into a three-dimensional reconstructed model, the system also performs point cloud data processing, three-dimensional data reconstruction, and model surface reconstruction on the image, converts the image into a three-dimensional initial model, and performs noise processing and detail enhancement on the three-dimensional initial model through smoothing, denoising, and enhancement processing, so as to improve the clarity of the quality of the reconstructed model.

[0095] Furthermore, the present embodiment sequentially performs point cloud data calculation, three-dimensional data reconstruction and model surface reconstruction on the fused OCT image to obtain an initial three-dimensional model, including:

[0096] The coordinates of each pixel in the fused OCT image are converted into three-dimensional space coordinates by a circular scanning algorithm to obtain three-dimensional point cloud data;

[0097] Performing volume rendering on the three-dimensional point cloud data by a ray casting method to obtain a three-dimensional data cube;

[0098] The three-dimensional data cube is subjected to model surface reconstruction processing by using a point cloud surface reconstruction algorithm to obtain the initial three-dimensional model.

[0099] The coordinates of each pixel in the fused image are converted into three-dimensional space coordinates through a circular scanning algorithm to generate corresponding three-dimensional point cloud data, thereby improving the quality and accuracy of the generated three-dimensional point cloud data. The three-dimensional point cloud data is volume rendered through the ray casting method, and the 3D image corresponding to the three-dimensional point cloud data is generated by simulating the propagation path of light in the three-dimensional data cube, and the corresponding three-dimensional data cube is constructed. The model surface of the constructed three-dimensional data cube is reconstructed through the point cloud surface reconstruction algorithm to obtain the initial three-dimensional model.

[0100] The fused OCT images are converted into three-dimensional point cloud data and corresponding three-dimensional data cubes through precise circular scanning algorithms and ray casting methods, which improves the quality and accuracy of the obtained point cloud data and data cubes. The surface of the cube is reconstructed through the point cloud surface reconstruction algorithm, which further improves the reconstructed three-dimensional model.

[0101] In this embodiment, the coordinates of the fused OCT image are converted into three-dimensional space coordinates. The image coordinates (u, v) can be converted into real three-dimensional coordinates (X, Y, Z) through the camera's intrinsic parameter matrix K. The specific formula is as follows:

[0102]

[0103] In addition to the above-mentioned method, the embodiment of the present invention also provides another method for constructing an initial three-dimensional model by performing point cloud computing, three-dimensional data reconstruction, and model surface reconstruction on the image.

[0104] The method for calculating point cloud data for the fused OCT image is as follows: for the pixel points in the fused OCT image generated after automatic feature matching, the distance output by the C-scan scanning algorithm, i.e., the cyclic scanning algorithm described in this embodiment, is converted into three-dimensional spatial coordinates to generate corresponding three-dimensional point cloud data. That is, according to the scanning spacing and position of each 2D slice, i.e., the fused OCT image described in this embodiment, the three-dimensional coordinates of each pixel point are calculated, and the specific coordinate conversion formula is as follows:

[0105] P=RX+t

[0106] Where R is the rotation matrix, t is the translation vector, and X is the image coordinate.

[0107] The specific steps of volume rendering of point cloud data by ray casting are: by simulating the propagation path of light in the three-dimensional data cube, a 3D image corresponding to the point cloud data is generated, that is, by stacking all 2D images in sequence to construct a three-dimensional data cube corresponding to the three-dimensional point cloud data. The specific calculation formula is as follows:

[0108]

[0109] Where C(r) is the accumulated color of the light in direction r, T(r(t)) is the transmittance of the voxel, and μ(r(s)) is the absorption coefficient of the voxel.

[0110] Furthermore, the specific steps of reconstructing the model surface of the three-dimensional data cube by the point cloud surface reconstruction algorithm are as follows: the system uses the Marching Cubes algorithm to reconstruct the surface, and the extracted 2D contour data, that is, the three-dimensional data cube described in this embodiment, is input into the Marching Cubes algorithm to generate a 3D surface model. When the MarchingCubes algorithm reconstructs the model surface, each voxel cube in the three-dimensional data cube is also classified at a fixed point, and the corresponding triangle topology structure is selected according to the classification result, and the triangle vertex position of each triangle topology structure is calculated by interpolation.

[0111] As another example of this embodiment, Figure 3 As shown, after obtaining the fused image, the system will perform coordinate conversion processing on the fused image, that is, converting the coordinates of the feature points in the image into actual three-dimensional space coordinates.

[0112] Furthermore, the present embodiment sequentially performs smoothing, denoising and enhancing on the initial three-dimensional model to obtain the three-dimensional reconstructed model, including:

[0113] The initial three-dimensional model is processed in sequence by a Laplace smoothing algorithm, a median filtering algorithm and a sharpening filtering algorithm to obtain the three-dimensional reconstructed model of the target object.

[0114] The above three smoothing and filtering algorithms can improve the model quality, enhance the detail expression of the model, remove the noise in the model, and further improve the quality and accuracy of the 3D reconstructed model.

[0115] In this embodiment, the system sequentially processes the initial three-dimensional model using a Laplace smoothing algorithm, a median filtering algorithm, and a sharpening filtering algorithm, sequentially enhancing the detail performance of the model, removing noise interference of the model, and improving the quality of the model.

[0116] Among them, the system uses the Laplace smoothing algorithm to smooth the model. Specifically: the Laplace smoothing algorithm is used to reduce the noise on the surface of the 3D model. Since the Laplace smoothing algorithm processes the model based on the second-order differential operator of the 3D model, for the new position Vi′ of each vertex Vi in the model, the Laplace smoothing algorithm uses the following formula:

[0117]

[0118] Among them, Vi is the original position of vertex i; Vj is the original position of the neighboring vertices of vertex i; N(i) is the set of neighboring vertices of vertex i; λ is the smoothing factor, which usually takes a value between [0,1]. Among them, the gradient Divergence Used to represent the Laplace operator, the expression is as follows:

[0119]

[0120] For vertices in three-dimensional space, the Laplace operator can be discretized as:

[0121]

[0122] Through the above formula, the model can be smoothed using the Laplace smoothing algorithm.

[0123] Furthermore, the system uses the median filtering algorithm to denoise the model. Specifically, the isolated noise points in the model are removed by using the median filtering technique, that is, the noise is removed by replacing the value of each vertex with the median of its neighboring vertex values. Therefore, for the vertex Vi in the model, the corresponding median filtering formula is as follows:

[0124] V i ′=median{V j |j∈N(i)}

[0125] Among them, Vi is the original position of vertex i; N(i) is the set of neighbor vertices of vertex i; median represents the median operation.

[0126] Furthermore, the system uses a sharpening filter algorithm to perform detail enhancement on the model. Specifically, the sharpening filter technology is used to enhance the detail performance of the 3D model. Since the sharpening filter technology improves the details by enhancing the high-frequency components in the image, the corresponding sharpening filter expression is:

[0127] V′ i =V i +α(V i -G(V i ))

[0128] Among them, Vi is the original position of vertex i; G(Vi) is the Gaussian blurred version of vertex i; α is the sharpening strength factor, which is used to control the degree of sharpening.

[0129] As another example provided by this embodiment, Figure 3As shown, after obtaining the three-dimensional space coordinates, the system converts the converted three-dimensional coordinate point set into point cloud data, i.e., a 3D model, and smoothes the generated 3D model to reduce noise and improve model quality, and generates the final three-dimensional eyeball model after the above processing.

[0130] In summary, the 3D reconstruction method provided by the embodiment of the present invention utilizes the ORB feature extraction and matching algorithm to achieve high-precision image registration and ensure accurate alignment between different images; through precise spatial coordinate transformation and ray casting, high-quality 3D point cloud data and reconstructed images are generated; through the Laplace smoothing algorithm, median filtering algorithm and sharpening filtering algorithm, the quality of the 3D model is improved, the detail performance of the model is enhanced, and noise interference in the model is removed. In addition, the reconstruction method for the model in the embodiment of the present invention is a set of fully designed automated image processing pipelines, which realizes the automation of the entire process from image acquisition, preprocessing, registration, reconstruction to final visual display, and reduces manual intervention and human errors through standardized data formats and processing steps, thereby improving the quality and accuracy of the reconstructed model.

[0131] In order to better illustrate the working principle and process flow of the OCT image-based three-dimensional reconstruction method, device and system of the present invention, reference may be made to, but is not limited to, the relevant records above.

[0132] Accordingly, see Figure 2 , Figure 2 The structure diagram of an embodiment of a three-dimensional reconstruction device based on OCT images provided by the present invention is shown in FIG. Figure 2 As shown, the three-dimensional reconstruction device includes an image feature extraction module 201 and a three-dimensional model reconstruction module 202.

[0133] The image feature extraction module 201 is used to obtain the front-end OCT image and the back-end OCT image of the target object, and respectively extract features from the front-end OCT image and the back-end OCT image to obtain the front-end feature descriptor and the back-end feature descriptor.

[0134] Furthermore, the image feature extraction module 201 extracts features from the front-end OCT image and the back-end OCT image respectively through the ORB algorithm to obtain a front-end feature descriptor and a back-end feature descriptor, including:

[0135] Feature point detection is performed on the front-end OCT image and the back-end OCT image respectively, and a front-end feature point set and a back-end feature point set are output; the front-end feature point set and the back-end feature point set are calculated using a binary algorithm to obtain the front-end feature descriptor and the back-end feature descriptor.

[0136] Furthermore, the image feature extraction module 201 performs feature point detection on the front-end OCT image and the back-end OCT image respectively, and outputs a front-end feature point set and a back-end feature point set, including:

[0137] Perform fast corner point detection on the front-end OCT image and the back-end OCT image respectively to obtain an initial feature point set, and calculate the Harris response value of each initial feature point in the initial feature point set; sort each initial feature point according to the calculated Harris response value, and filter and output the corresponding front-end feature point set and the back-end feature point set from the initial feature point set according to a preset response value threshold.

[0138] Furthermore, the image feature extraction module 201 uses a binary algorithm to calculate the front-end feature point set and the back-end feature point set to obtain the front-end feature descriptor and the back-end feature descriptor, including:

[0139] The front-end feature point set and the back-end feature point set are filtered by a Gaussian filtering algorithm, and a binary assignment calculation is performed on the filtered feature point set to output a front-end feature descriptor and a back-end feature descriptor; the feature angles of the front-end feature point set and the back-end feature point set are calculated, the front-end feature descriptor and the back-end feature descriptor are adjusted according to the feature angles, the adjusted feature descriptors are used as feature descriptors, and the front-end feature descriptors and the back-end feature descriptors are obtained.

[0140] The three-dimensional model reconstruction module 202 is used to perform matching calculation on the front-end feature descriptor and the back-end feature descriptor to obtain a fused OCT image, and perform spatial coordinate transformation processing and smoothing denoising processing on the fused OCT image in sequence to obtain a three-dimensional reconstructed model of the target object.

[0141] Furthermore, the three-dimensional model reconstruction module 202 performs matching calculation on the front-end feature descriptor and the back-end feature descriptor to obtain a fused OCT image, including:

[0142] The front-end feature descriptor and the back-end feature descriptor are matched and calculated by a Hamming distance algorithm, and the matching calculation results are screened and eliminated for abnormalities to obtain a screened image set to be fused; the front-end image in the image set to be fused is fused with the corresponding back-end image to obtain the fused OCT image.

[0143] Furthermore, the three-dimensional model reconstruction module 202 sequentially performs spatial coordinate conversion processing and smoothing and denoising processing on the fused OCT image to obtain a three-dimensional reconstructed model of the target object, including:

[0144] The fused OCT image is sequentially subjected to point cloud data calculation, three-dimensional data reconstruction and model surface reconstruction to obtain an initial three-dimensional model; the initial three-dimensional model is sequentially subjected to smoothing, denoising and enhancement processing to obtain the three-dimensional reconstructed model of the target object.

[0145] Furthermore, the three-dimensional model reconstruction module 202 sequentially performs point cloud data calculation, three-dimensional data reconstruction and model surface reconstruction on the fused OCT image to obtain an initial three-dimensional model, including:

[0146] The coordinates of each pixel point in the fused OCT image are converted into three-dimensional space coordinates through a circular scanning algorithm to obtain three-dimensional point cloud data; the three-dimensional point cloud data is volume rendered through a ray casting method to obtain a three-dimensional data cube; the three-dimensional data cube is modeled and reconstructed through a point cloud surface reconstruction algorithm to obtain the initial three-dimensional model.

[0147] Furthermore, the 3D model reconstruction module 202 sequentially performs smoothing, denoising and enhancement processing on the initial 3D model to obtain the 3D reconstructed model, including:

[0148] The initial three-dimensional model is processed in sequence by a Laplace smoothing algorithm, a median filtering algorithm and a sharpening filtering algorithm to obtain the three-dimensional reconstructed model of the target object.

[0149] Correspondingly, an embodiment of the present invention also provides a three-dimensional reconstruction system based on OCT images, characterized in that the three-dimensional reconstruction system includes: a memory, a processor, and a three-dimensional reconstruction program stored in the memory and executable on the processor, and when the three-dimensional reconstruction program is executed by the processor, the steps of a three-dimensional reconstruction method based on OCT images described in any one of the above are implemented.

[0150] In summary, the embodiments of the present invention provide a three-dimensional reconstruction method, device and system based on OCT images, which extract features from the front-end OCT images and the back-end OCT images of the sub-target object through the ORB algorithm, obtain the corresponding front-end feature descriptors and back-end feature descriptors, and perform matching calculations on the above two feature descriptors to obtain a fused OCT image, and then perform spatial coordinate transformation and smoothing denoising on the fused OCT image to obtain a three-dimensional reconstructed model of the target object. The feature extraction of OCT images by the ORB algorithm improves the accuracy and precision of the extracted feature symbols, and also ensures that the extracted feature symbols can more accurately describe the image features, thereby improving the authenticity and accuracy of the three-dimensional model obtained by subsequent reconstruction; the smoothing denoising process is performed on the reconstructed model, which further improves the quality and accuracy of the reconstructed model, removes the noise in the model, and enhances the detail performance of the model.

[0151] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A three-dimensional reconstruction method based on OCT images, characterized in that: The following steps are involved: Acquire a front-end OCT image and a back-end OCT image of the target object, and respectively perform feature extraction on the front-end OCT image and the back-end OCT image using an ORB algorithm to obtain a front-end feature descriptor and a back-end feature descriptor; The front-end feature descriptor and the back-end feature descriptor are matched and calculated to obtain a fused OCT image, and the fused OCT image is sequentially subjected to spatial coordinate transformation processing and smoothing and denoising processing to obtain a three-dimensional reconstructed model of the target object.

2. A three-dimensional reconstruction method based on OCT images as claimed in claim 1, characterized in that: The method of extracting features from the front-end OCT image and the back-end OCT image by using the ORB algorithm to obtain a front-end feature descriptor and a back-end feature descriptor includes: Performing feature point detection on the front-end OCT image and the back-end OCT image respectively, and outputting a front-end feature point set and a back-end feature point set; The front-end feature point set and the back-end feature point set are calculated using a binary algorithm to obtain the front-end feature descriptor and the back-end feature descriptor.

3. A three-dimensional reconstruction method based on OCT images as claimed in claim 2, characterized in that: The performing feature point detection on the front-end OCT image and the back-end OCT image respectively and outputting a front-end feature point set and a back-end feature point set comprises: Performing fast corner point detection on the front-end OCT image and the back-end OCT image respectively to obtain an initial feature point set, and calculating a Harris response value of each initial feature point in the initial feature point set; Each of the initial feature points is sorted according to the calculated Harris response value, and the corresponding front-end feature point set and the back-end feature point set are filtered and output from the initial feature point set according to a preset response value threshold.

4. A three-dimensional reconstruction method based on OCT images as claimed in claim 2, characterized in that: The using a binary algorithm to calculate the front-end feature point set and the back-end feature point set to obtain the front-end feature descriptor and the back-end feature descriptor includes: The front-end feature point set and the back-end feature point set are filtered by a Gaussian filtering algorithm, and a binary value assignment calculation is performed on the filtered feature point set to output a front-end feature descriptor and a back-end feature descriptor; The feature angles of the front-end feature point set and the back-end feature point set are calculated, the front-end feature descriptor and the back-end feature descriptor are adjusted according to the feature angles, the adjusted feature descriptors are used as feature descriptors, and the front-end feature descriptors and the back-end feature descriptors are obtained.

5. The three-dimensional reconstruction method based on OCT images according to claim 1, characterized in that: The performing matching calculation on the front-end feature descriptor and the back-end feature descriptor to obtain a fused OCT image includes: Performing matching calculation on the front-end feature descriptor and the back-end feature descriptor by using a Hamming distance algorithm, and performing abnormal screening and elimination on the matching calculation results to obtain a screened image set to be fused; The front-end image in the image set to be fused is fused with the corresponding back-end image to obtain the fused OCT image.

6. The three-dimensional reconstruction method based on OCT images according to claim 1, characterized in that: The step of sequentially performing spatial coordinate conversion processing and smoothing and denoising processing on the fused OCT image to obtain a three-dimensional reconstructed model of the target object includes: performing point cloud data calculation, three-dimensional data reconstruction and model surface reconstruction on the fused OCT image in sequence to obtain an initial three-dimensional model; The initial three-dimensional model is sequentially smoothed, denoised and enhanced to obtain the three-dimensional reconstructed model of the target object.

7. A three-dimensional reconstruction method based on OCT images as claimed in claim 6, characterized in that: The step of sequentially performing point cloud data calculation, three-dimensional data reconstruction and model surface reconstruction on the fused OCT image to obtain an initial three-dimensional model includes: The coordinates of each pixel in the fused OCT image are converted into three-dimensional space coordinates by a circular scanning algorithm to obtain three-dimensional point cloud data; Performing volume rendering on the three-dimensional point cloud data by a ray casting method to obtain a three-dimensional data cube; The three-dimensional data cube is subjected to model surface reconstruction processing by using a point cloud surface reconstruction algorithm to obtain the initial three-dimensional model.

8. The three-dimensional reconstruction method based on OCT images according to claim 6, characterized in that: The step of sequentially performing smoothing, denoising and enhancing processing on the initial three-dimensional model to obtain the three-dimensional reconstructed model includes: The initial three-dimensional model is processed in sequence by a Laplace smoothing algorithm, a median filtering algorithm and a sharpening filtering algorithm to obtain the three-dimensional reconstructed model of the target object.

9. A three-dimensional reconstruction device based on OCT images, characterized in that: The three-dimensional reconstruction device includes an image feature extraction module and a three-dimensional model reconstruction module; The image feature extraction module is used to obtain a front-end OCT image and a back-end OCT image of the target object, and respectively perform feature extraction on the front-end OCT image and the back-end OCT image to obtain a front-end feature descriptor and a back-end feature descriptor; The three-dimensional model reconstruction module is used to perform matching calculation on the front-end feature descriptor and the back-end feature descriptor to obtain a fused OCT image, and perform spatial coordinate transformation processing and smoothing denoising processing on the fused OCT image in sequence to obtain a three-dimensional reconstructed model of the target object.

10. A three-dimensional reconstruction system based on OCT images, characterized in that: The three-dimensional reconstruction system includes: a memory, a processor, and a three-dimensional reconstruction program stored in the memory and executable on the processor. When the three-dimensional reconstruction program is executed by the processor, the steps of a three-dimensional reconstruction method based on OCT images as described in any one of claims 1 to 8 are implemented.

Images