A method and system for reconstructing corneal 3D model based on optical image
Through optical scanning equipment, the multi-angle corneal images are collected and processed, and the depth of key feature points is extracted and estimated, which solves the problem of inaccurate estimation of depth information in corneal images, and improves the construction accuracy and reconstruction effect of the corneal 3D model.
Patent Information
- Application Number
- CN202410881817.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-03
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-07-03
AI Technical Summary
In the prior art, the accuracy of estimation of depth information in corneal images is poor, resulting in low accuracy of construction of corneal 3D models, affecting the reconstruction effect.
Multi-angle corneal images are collected through optical scanning equipment, denoising pre-processing and data dimensionality reduction, key feature points are extracted, similarity analysis and depth estimation are performed, depth maps are generated, and three-dimensional model reconstruction is realized.
It improves the accuracy of corneal depth estimation, improves the construction accuracy of corneal 3D model, and solves the problem of model error and dislocation.
Smart Images

Figure CN118429546B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of optical image reconstruction, and in particular to a method and system for reconstructing a corneal 3D model based on optical images. Background Art
[0002] Reconstructing a 3D corneal model refers to extracting the key feature points of the cornea from the optical images of the cornea acquired from multiple angles to generate a three-dimensional model that can accurately reflect the corneal morphology, structure and texture.
[0003] At present, in the existing corneal reconstruction process, depth information is an indispensable part of building a three-dimensional model. Without accurate depth estimation, when fusing corneal images into a three-dimensional model, it is impossible to accurately determine the position of each feature point in the three-dimensional space, which may cause errors in the shape and structure of the reconstructed corneal 3D model, and the accuracy of the model is greatly reduced. When stereo stitching multi-angle images, without depth estimation, it is difficult to ensure the consistency of the position of the same feature points in different images in the three-dimensional space, and it is impossible to accurately align the feature points in different images, resulting in problems such as misalignment and overlap in the stitched model. Therefore, a method is needed to solve the above problems.
[0004] In summary, the prior art has a technical problem in that the accuracy of estimating the depth information in the corneal image is poor, resulting in low accuracy in constructing the 3D model, which further affects the corneal reconstruction effect. Summary of the invention
[0005] The purpose of this application is to provide a method and system for reconstructing a corneal 3D model based on optical images, so as to solve the technical problem in the prior art that the accuracy of constructing a 3D model is low due to the poor estimation accuracy of depth information in corneal images, which further affects the corneal reconstruction effect.
[0006] In view of the above problems, the present application provides a method and system for reconstructing a corneal 3D model based on optical images.
[0007] In a first aspect, the present application provides a method for reconstructing a 3D corneal model based on optical images, wherein the method is implemented by a system for reconstructing a 3D corneal model based on optical images, wherein the method comprises: S1: acquiring a set of multi-angle corneal images through an optical scanning device, performing denoising preprocessing on the multi-angle corneal image set, obtaining a set of available multi-angle corneal images, performing data dimensionality reduction based on the available multi-angle corneal image set, and obtaining a standard multi-angle corneal image set; S2: extracting key feature points based on the available multi-angle corneal image set to obtain a multi-angle corneal image. A key feature point set, performing similarity analysis based on the multi-angle corneal image key feature point set to obtain a matching key feature point set; S3: performing depth estimation on each feature point in the multi-angle corneal image key feature point set based on the matching key feature point set to obtain a key feature point depth value set, and generating a key feature point depth map based on the key feature point depth value set; S4: performing stereo stitching fusion and three-dimensional model reconstruction on the available multi-angle corneal image set according to the matching key feature point set and the key feature point depth map to obtain a stereoscopic three-dimensional panoramic model of the cornea.
[0008] In a second aspect, the present application also provides a system for reconstructing a corneal 3D model based on optical images, which is used to execute a method for reconstructing a corneal 3D model based on optical images as described in the first aspect, wherein the system includes: a standard multi-angle corneal image set acquisition module, the standard multi-angle corneal image set acquisition module is used to acquire a multi-angle corneal image set through an optical scanning device, perform denoising preprocessing on the multi-angle corneal image set, obtain a usable multi-angle corneal image set, perform data dimensionality reduction based on the usable multi-angle corneal image set, and obtain a standard multi-angle corneal image set; a matching key feature point set acquisition module, the matching key feature point set acquisition module is used to extract key feature points based on the usable multi-angle corneal image set to obtain a multi-angle A corneal image key feature point set, performing similarity analysis based on the multi-angle corneal image key feature point set to obtain a matching key feature point set; a key feature point depth map generation module, the key feature point depth map generation module is used to perform depth estimation on each feature point in the multi-angle corneal image key feature point set based on the matching key feature point set to obtain a key feature point depth value set, and generate a key feature point depth map based on the key feature point depth value set; a corneal stereoscopic three-dimensional panoramic model acquisition module, the corneal stereoscopic three-dimensional panoramic model acquisition module is used to perform stereo stitching fusion and three-dimensional model reconstruction on the available multi-angle corneal image set according to the matching key feature point set and the key feature point depth map to obtain a corneal stereoscopic three-dimensional panoramic model.
[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages:
[0010] A multi-angle corneal image set is acquired by collecting through an optical scanning device, and the multi-angle corneal image set is pre-processed for denoising to obtain a usable multi-angle corneal image set. Data dimension reduction is performed based on the usable multi-angle corneal image set to obtain a standard multi-angle corneal image set; key feature points are extracted based on the usable multi-angle corneal image set to obtain a multi-angle corneal image key feature point set, and similarity analysis is performed based on the multi-angle corneal image key feature point set to obtain a matching key feature point set; depth estimation is performed on each feature point in the multi-angle corneal image key feature point set based on the matching key feature point set to obtain a key feature point depth value set, and a key feature point depth map is generated based on the key feature point depth value set; stereo stitching and fusion and three-dimensional model reconstruction are performed on the usable multi-angle corneal image set according to the matching key feature point set and the key feature point depth map to obtain a stereoscopic three-dimensional panoramic model of the cornea, thereby achieving the technical goal of improving the accuracy of corneal depth estimation and the technical effect of improving the accuracy of corneal model construction.
[0011] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented according to the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically cited below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0013] Figure 1 A schematic diagram of a method for reconstructing a 3D corneal model based on optical images in this application;
[0014] Figure 2 This is a schematic diagram of the structure of a system for reconstructing a corneal 3D model based on optical images in the present application.
[0015] Description of reference numerals:
[0016] A module for obtaining a set of standard multi-angle corneal images 11, a module for obtaining a set of matching key feature points 12, a module for generating a depth map of key feature points 13, and a module for obtaining a stereoscopic three-dimensional panoramic model of the cornea 14. DETAILED DESCRIPTION
[0017] The present application provides a method and system for reconstructing a corneal 3D model based on optical images, which solves the technical problem in the prior art that the accuracy of estimating the depth information in the corneal image is poor, resulting in low accuracy in constructing the 3D model, which further affects the effect of corneal reconstruction. The technical goal of improving the accuracy of corneal depth estimation is achieved, and the technical effect of improving the accuracy of corneal model construction is achieved.
[0018] Below, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them. Embodiment 1
[0019] Please refer to the attached Figure 1 The present application provides a method for reconstructing a 3D corneal model based on optical images, wherein the method is applied to a system for reconstructing a 3D corneal model based on optical images, and the method specifically comprises the following steps:
[0020] S1: acquiring a multi-angle corneal image set by an optical scanning device, performing denoising preprocessing on the multi-angle corneal image set to acquire a usable multi-angle corneal image set, performing data dimension reduction based on the usable multi-angle corneal image set to acquire a standard multi-angle corneal image set;
[0021] Specifically, a multi-angle corneal image set is acquired through an optical scanning device, the edge features of the multi-angle corneal image set are detected, a circular fitting is performed on the corneal area, and then pixel denoising is performed, pixel points that do not conform to the fitted circle are removed, and the image is smoothed, etc., to obtain a usable multi-angle corneal image set with improved quality. The pixel value features in the neighborhood of each pixel point are extracted, and the similarity between the current pixel point and each other pixel point in the neighborhood is calculated. The dimensionality reduction processing of the usable multi-angle corneal image set is performed as the corneal pixel similarity, and the usable multi-angle corneal images or parts of the images with low similarity are removed, so as to obtain a standard multi-angle corneal image set.
[0022] S2: extracting key feature points based on the available multi-angle corneal image set to obtain a multi-angle corneal image key feature point set, and performing similarity analysis based on the multi-angle corneal image key feature point set to obtain a matching key feature point set;
[0023] Specifically, key feature points are extracted based on the available multi-angle corneal image set, such as the corneal vertex, corneal edge, etc., to obtain a multi-angle corneal image key feature point set. Each multi-angle corneal image key feature point is converted into a high-dimensional descriptor, such as a vector or a matrix, and the similarity of each key feature point in the corneal image key feature point descriptor set is calculated. If the descriptor of each key feature point is regarded as a vector, the cosine similarity between multiple vectors is calculated and stored in the key feature point similarity set. Different similarity matching benchmarks are assigned according to the corneal image characteristic information, and the key feature points with similarity lower than the similarity matching benchmark are removed from the key feature point similarity set, and only the key feature points with sufficiently high similarity are retained to obtain a matching key feature point set.
[0024] S3: performing depth estimation on each feature point in the key feature point set of the multi-angle corneal image based on the matching key feature point set to obtain a key feature point depth value set, and generating a key feature point depth map according to the key feature point depth value set;
[0025] Specifically, the depth of each key feature point in the multi-angle corneal image is estimated using the results of stereo matching, and the position of each key feature point in three-dimensional space, that is, the distance of each key feature point from the optical scanning device, is determined as the three-dimensional spatial position of each key feature point. The disparity value of each key feature point is converted into a depth value to generate a depth map. The depth map is a two-dimensional image of the same size as the input image, and each pixel value represents the depth information corresponding to the pixel position.
[0026] S4: According to the matching key feature point set and the key feature point depth map, stereo stitching and fusion and three-dimensional model reconstruction are performed on the available multi-angle corneal image set to obtain a stereoscopic three-dimensional panoramic model of the cornea.
[0027] Specifically, the correspondence between corneal images at different angles is determined by matching the key feature point set and the key feature point depth map. By matching the feature points, the relative transformation between different images, such as rotation and translation, is calculated to ensure correct alignment in three-dimensional space. Combined with the key feature point depth map, depth information is assigned to each matched feature point to obtain the actual position of the feature point in three-dimensional space. Image fusion techniques, such as weighted averaging and multi-band fusion, are used to fuse the transformed aligned images together. Color correction and brightness adjustment are performed to ensure that the fused images are visually coherent. Using the matched key feature points and depth information, point cloud data of the cornea is generated. Point cloud data is a set of discrete points of the cornea in three-dimensional space, reflecting the shape and structure of the cornea. The surface information of the cornea is extracted from the point cloud data to generate a stereoscopic three-dimensional panoramic model of the cornea.
[0028] The method for reconstructing a corneal 3D model based on optical images is applied to a system for reconstructing a corneal 3D model based on optical images, which can achieve the technical goal of improving the accuracy of corneal depth estimation and achieve the technical effect of improving the accuracy of corneal model construction.
[0029] Furthermore, the present application also includes the following steps:
[0030] S11: extracting and identifying edge features of the multi-angle corneal image set by using the Canny operator to obtain a corneal edge recognition information set;
[0031] S12: establishing a circular fitting rule according to the corneal edge recognition information set, and determining a circular fitting parameter based on the circular fitting rule;
[0032] S13: randomly selecting three points in the multi-angle corneal image set, connecting them to obtain a fitting circle as a circular optimal solution, and iteratively fitting the circular optimal solution using the circular fitting parameters to obtain a circular fitting result;
[0033] S14: performing data denoising preprocessing on the multi-angle corneal image set based on the circular fitting result to obtain the usable multi-angle corneal image set.
[0034] Specifically, the Canny operator is an edge detection algorithm that can detect strong edges in an image without being affected by noise. The Canny operator is used to detect edge features of a multi-angle corneal image set, and the edge features are identified to obtain a corneal edge recognition information set.
[0035] Then, since the cornea is usually circular or approximately circular, the RANSAC algorithm is used to perform circular fitting on the corneal area, and the corneal edge information is further processed to perform pixel denoising. The circular fitting rules are established based on the corneal edge recognition information set, including the maximum number of iterations, distance threshold, fitting direction, fitting range, etc., and then the parameters for circular fitting are determined.
[0036] Next, three points are randomly selected in the multi-angle corneal image set for connection to determine the initial fitting circle as the optimal solution for the circle. The optimal circle is iteratively optimized according to the circle fitting parameters. Among them, the number of point cloud data with an iteration distance less than the distance threshold is compared. If the number of the second fitting circles is greater than the initial number of fitting circles, the second fitting circle is replaced as the optimal solution for the circle. The distance threshold is obtained by a technician in this field according to the actual situation. After multiple iterations, the optimal fitting circle is found to represent the actual shape of the cornea, and the circular fitting result is obtained.
[0037] Next, the multi-angle corneal image set is preprocessed with data denoising according to the circle fitting result. Denoising may include removing pixels that do not conform to the fitted circle, smoothing the image, and other operations. After data denoising preprocessing, a multi-angle corneal image with improved quality is obtained, which is easier to be used by subsequent algorithms or applications, and a usable multi-angle corneal image set is output.
[0038] The corneal image is processed by using the Canny operator and circle fitting to improve the image quality and usability.
[0039] Furthermore, the present application also includes the following steps:
[0040] Performing data similarity calculation on each image information in the available multi-angle corneal image set to obtain corneal pixel similarity;
[0041] A similarity probability distribution is constructed according to the corneal pixel similarity, and the similarity probability distribution is specifically: ,in, is the divergence of similarity between the available multi-angle corneal image and the standard multi-angle corneal image, Pixels in the corneal image available at multiple angles With pixels The similarity probability of is the pixel point in the standard multi-angle corneal image With pixels The similarity probability of is a measure of the closeness of the similarity distribution, let The tendency to 0 is used as a constraint condition for dimensionality reduction;
[0042] The available multi-angle corneal image set is subjected to dimensionality reduction processing based on the similarity probability distribution to obtain the standard multi-angle corneal image set.
[0043] Specifically, each pixel point of each image information in the available multi-angle corneal image set is traversed and accessed, and the pixel value features in the neighborhood of the current pixel point are extracted. The similarity between the current pixel point and each other pixel point in the neighborhood is calculated to form a similarity matrix as the corneal pixel similarity.
[0044] Then, a similarity probability distribution is constructed based on the corneal pixel similarity to reflect the similarity relationship between each pixel point and to achieve the subsequent probability distribution dimensionality reduction. Based on the historical impact records, standard multi-angle corneal images are extracted. In the similarity probability distribution, if the pixel points in the multi-angle corneal image are available, With pixels The similarity probability of the pixel point in the standard multi-angle corneal image With pixels The closer the similarity probability is, the closer the divergence of the similarity between the available multi-angle corneal image and the standard multi-angle corneal image is to 0, and the higher the corneal pixel similarity is, otherwise, the lower it is. The approach to 0 is used as a constraint condition for dimensionality reduction. The closer KL approaches 0, the closer the available multi-angle corneal image is to the standard multi-angle corneal image. Conversely, the greater the difference is.
[0045] Next, the available multi-angle corneal image set is subjected to dimensionality reduction processing based on the similarity probability distribution, and the available multi-angle corneal images or parts of the images that have low similarity with the standard multi-angle corneal images are removed, so as to obtain a set that is closer to the standard multi-angle corneal image, namely, the standard multi-angle corneal image set.
[0046] By acquiring a set of standard multi-angle corneal images for subsequent image processing, analysis, and comparison, the data quality is higher and more representative.
[0047] Furthermore, the present application also includes the following steps:
[0048] S21: Acquire pixel description feature factors, wherein the pixel description feature factors include grayscale, texture, and spatial relationship, and perform feature description on the multi-angle corneal image key feature point set based on the pixel description feature factors to obtain a corneal image key feature point description subset;
[0049] S22: using a cosine similarity algorithm to calculate the similarity of each key feature point in the corneal image key feature point descriptor set to obtain a key feature point similarity set;
[0050] S23: Setting a similarity matching benchmark according to the corneal image characteristic information, and performing screening and registration analysis on the key feature point similarity set based on the similarity matching benchmark to obtain the matching key feature point set.
[0051] Specifically, pixel description feature factors are obtained to describe pixels, and the pixel description feature factors include grayscale, texture, and spatial relationship. Grayscale is used to characterize the brightness or darkness of a pixel. Texture is used to characterize the local pattern or structure around a pixel. Spatial relationship is used to characterize the relative position or arrangement between pixels. Based on the pixel description feature factors, a set of key feature points of a multi-angle corneal image is characterized to obtain a set of descriptors of key feature points of the corneal image. Each key feature point of the multi-angle corneal image is converted into a high-dimensional descriptor, such as a vector or a matrix. The descriptor can contain detailed information of the feature point for subsequent similarity calculation and matching.
[0052] Then, after obtaining the corneal image key feature point descriptor set, the cosine similarity algorithm is used to calculate the similarity of each key feature point in the corneal image key feature point descriptor set. Cosine similarity is a similarity measurement method that evaluates the similarity between two vectors by calculating the cosine value of the angle between them. If the descriptor of each key feature point is regarded as a vector, the cosine similarity between multiple vectors is calculated, and the similarity results are stored in the key feature point similarity set.
[0053] Next, a similarity matching benchmark is set according to the characteristic information of the corneal image, that is, according to the application scenario of the image and the density of each key feature point. For example, the higher the accuracy requirement of the application scenario, the higher the similarity matching benchmark, and vice versa. Images with higher density of key feature points may require a larger similarity matching benchmark to improve the accuracy of matching, and vice versa. Based on the similarity matching benchmark, the key feature point similarity set is screened and registered for analysis, and then the key feature points with similarity lower than the similarity matching benchmark are removed, and only the key feature points with sufficiently high similarity are retained. During the registration analysis process, the screened key feature points are aligned or matched to obtain a set of matching key feature points for subsequent image processing or analysis tasks.
[0054] By performing feature description, similarity calculation and screening and registration analysis on the key feature points of multi-angle corneal images, the accuracy of obtaining the matching key feature point set can be further improved.
[0055] Furthermore, the present application also includes the following steps:
[0056] S31: selecting a reference image from the available multi-angle corneal image set based on the matching key feature point set to obtain reference corneal image information and a set of other corneal image information;
[0057] S32: constructing a reference feature three-dimensional coordinate system according to the reference key feature point set of the reference corneal image information and the key feature point depth map;
[0058] S33: Stereoscopically stitching and fusing the reference corneal image information and the remaining corneal image information sets based on the reference feature three-dimensional coordinate system to obtain corneal feature point fused image information;
[0059] S34: enhancing the clarity of the overlapping image area of the corneal feature point fusion image information to obtain corneal enhanced fusion image information, and reconstructing a three-dimensional model based on the corneal enhanced fusion image information to obtain the corneal stereoscopic three-dimensional panoramic model.
[0060] Specifically, the available multi-angle corneal image corresponding to the matched key feature point with the highest similarity is determined as the reference image by matching the key feature point set, and the reference corneal image information is obtained. The remaining images in the available multi-angle corneal image set are classified into the remaining corneal image information set.
[0061] Then, a reference feature three-dimensional coordinate system is constructed using a set of reference key feature points such as the corneal vertex and corneal edge of the reference corneal image information and a depth map of the key feature points, which serves as a reference for subsequent three-dimensional model reconstruction.
[0062] Next, the reference feature three-dimensional coordinate system is used to perform posture association mapping on the remaining key feature point sets of the other corneal image information sets, so as to determine the posture of each remaining corneal image relative to the reference corneal image information, obtain the necessary information for converting each remaining corneal image information to the reference feature three-dimensional coordinate system of the reference corneal image information, and determine the remaining feature point fusion conversion feature matrix information. Based on the remaining feature point fusion conversion feature matrix information, the reference corneal image information and the remaining corneal image information sets are stereo stitched and fused to generate corneal feature point fusion image information.
[0063] Next, the overlapping area of the two images in the corneal feature point fusion image information is gridded, the clarity of the two images is compared in the grid area, and the image area with higher clarity is selected for clarity enhancement to obtain corneal enhanced fusion image information. A three-dimensional model is reconstructed based on the corneal enhanced fusion image information to obtain the stereoscopic three-dimensional panoramic model of the cornea. For example, three-dimensional information is extracted from the corneal enhanced fusion image information, and a model representing the three-dimensional shape and structure of the cornea is constructed to obtain a stereoscopic three-dimensional panoramic model of the cornea.
[0064] By obtaining a stereoscopic three-dimensional panoramic model of the cornea, it can be used for further medical analysis or visualization.
[0065] Furthermore, the present application also includes the following steps:
[0066] Performing posture association mapping on the remaining key feature point sets of the remaining corneal image information sets based on the reference feature three-dimensional coordinate system to obtain fusion conversion type information of the remaining feature points;
[0067] Determine the fusion conversion feature matrix information of the remaining feature points according to the fusion conversion type information of the remaining feature points;
[0068] The reference corneal image information and the remaining corneal image information sets are stereoscopically spliced and fused based on the remaining feature point fusion conversion feature matrix information to obtain the corneal feature point fusion image information.
[0069] Specifically, the benchmark feature three-dimensional coordinate system is used to perform posture association mapping on the remaining key feature point sets of other corneal image information sets, such as the corneal vertex, corneal edge, etc., and the position and direction of the key feature point of each remaining corneal image in the benchmark feature three-dimensional coordinate system are calculated, so as to determine the posture of each remaining corneal image relative to the benchmark corneal image information, and obtain the remaining feature point fusion conversion type information to ensure that in the subsequent stereo stitching and fusion process, the key feature points of all images can be correctly aligned and matched.
[0070] Then, according to the remaining feature point fusion conversion type information, the remaining feature point fusion conversion feature matrix information is determined. The remaining feature point fusion conversion feature matrix information contains the necessary information for converting each remaining corneal image information to the reference feature three-dimensional coordinate system of the reference corneal image information, such as rotation, translation, symmetry, etc. Different conversion types are selected, and the conversion matrix of the feature point coordinates is also different, ensuring that the key feature points of each remaining corneal image information can be accurately matched and aligned during stereo stitching and fusion.
[0071] Next, the reference corneal image information and the remaining corneal image information sets are stereoscopically stitched and fused based on the remaining feature point fusion transformation feature matrix information. Among them, rotation, translation and other transformations are performed to accurately align the remaining corneal image information sets with the reference corneal image information at key feature points. Then the transformed image is fused with the reference corneal image information to generate corneal feature point fusion image information containing all image information.
[0072] By acquiring corneal feature points and fusing image information, a comprehensive and accurate corneal view is provided, which facilitates subsequent three-dimensional model reconstruction and analysis.
[0073] Furthermore, the present application also includes the following steps:
[0074] Divide the overlapping image area of the corneal feature point fusion image information into grids to obtain a set of overlapping image grid areas;
[0075] The clarity of the overlapping image grid area set is evaluated by using a clarity multi-dimensional index to obtain a fused image grid area clarity set;
[0076] The grid areas in the fusion image grid area clarity set that do not meet the preset clarity threshold are subjected to clarity comparison and enhancement to determine corneal enhanced fusion image information.
[0077] Specifically, during the splicing and fusion, the overlapping area of the two images in the corneal feature point fusion image information is gridded. In the fused image, the overlapping area may have different clarity due to factors such as the viewing angle and lighting between different images. The clarity of each grid area is evaluated, that is, the clarity of the two images is compared in the grid area, and the image area with higher clarity is selected for clarity enhancement to obtain a set of overlapping image grid areas.
[0078] Then, the clarity of each overlapping image grid area set is evaluated by the clarity multidimensional index. Among them, the clarity multidimensional index may include contrast, sharpness, noise level and other aspects, which are used to reflect the clarity quality of the image. By evaluating the clarity of each grid area, the clarity set of the fused image grid area is obtained.
[0079] Next, after obtaining the clarity set of the fused image grid area, check whether the clarity of each grid area meets the preset clarity threshold. Obtaining a grid area with a clarity lower than the preset threshold indicates that the image quality of the area is poor and clarity enhancement is required. Using methods such as denoising, sharpening, and contrast enhancement, each grid area that does not meet the clarity threshold is processed separately to ensure that the image quality of other areas is not damaged while the clarity is enhanced. The fused image that has been processed with clarity enhancement is determined to be the cornea enhanced fusion image information.
[0080] By obtaining corneal enhanced fusion image information, higher image quality and more accurate corneal feature point information can be obtained.
[0081] In summary, the method for reconstructing a corneal 3D model based on optical images provided in this application has the following technical effects:
[0082] A multi-angle corneal image set is acquired by collecting through an optical scanning device, and the multi-angle corneal image set is pre-processed for denoising to obtain a usable multi-angle corneal image set. Data dimension reduction is performed based on the usable multi-angle corneal image set to obtain a standard multi-angle corneal image set; key feature points are extracted based on the usable multi-angle corneal image set to obtain a multi-angle corneal image key feature point set, and similarity analysis is performed based on the multi-angle corneal image key feature point set to obtain a matching key feature point set; depth estimation is performed on each feature point in the multi-angle corneal image key feature point set based on the matching key feature point set to obtain a key feature point depth value set, and a key feature point depth map is generated based on the key feature point depth value set; stereo stitching and fusion and three-dimensional model reconstruction are performed on the usable multi-angle corneal image set according to the matching key feature point set and the key feature point depth map to obtain a stereoscopic three-dimensional panoramic model of the cornea, thereby achieving the technical goal of improving the accuracy of corneal depth estimation and the technical effect of improving the accuracy of corneal model construction. Embodiment 2
[0083] Based on the same inventive concept as the method for reconstructing a 3D corneal model based on optical images in the aforementioned embodiment, the present application also provides a system for reconstructing a 3D corneal model based on optical images. Figure 2 , the system comprising:
[0084] A standard multi-angle corneal image set acquisition module 11, the standard multi-angle corneal image set acquisition module 11 is used to acquire a multi-angle corneal image set through an optical scanning device, perform denoising preprocessing on the multi-angle corneal image set, acquire a usable multi-angle corneal image set, perform data dimension reduction based on the usable multi-angle corneal image set, and acquire a standard multi-angle corneal image set;
[0085] A matching key feature point set acquisition module 12, wherein the matching key feature point set acquisition module 12 is used to extract key feature points based on the available multi-angle corneal image set to obtain a multi-angle corneal image key feature point set, and perform similarity analysis based on the multi-angle corneal image key feature point set to obtain a matching key feature point set;
[0086] A key feature point depth map generation module 13, the key feature point depth map generation module 13 is used to perform depth estimation on each feature point in the multi-angle corneal image key feature point set based on the matching key feature point set, obtain a key feature point depth value set, and generate a key feature point depth map according to the key feature point depth value set;
[0087] The cornea stereoscopic three-dimensional panoramic model acquisition module 14 is used to stereoscopically stitch and fuse the available multi-angle corneal image set and reconstruct the three-dimensional model according to the matching key feature point set and the key feature point depth map to obtain the cornea stereoscopic three-dimensional panoramic model.
[0088] Furthermore, the standard multi-angle corneal image set acquisition module 11 in the system is also used for:
[0089] Performing edge feature extraction and identification on the multi-angle corneal image set by using the Canny operator to obtain a corneal edge recognition information set;
[0090] Establishing a circular fitting rule according to the corneal edge recognition information set, and determining a circular fitting parameter based on the circular fitting rule;
[0091] Randomly selecting three points in the multi-angle corneal image set, connecting them to obtain a fitting circle as a circular optimal solution, and iteratively fitting the circular optimal solution using the circular fitting parameters to obtain a circular fitting result;
[0092] The multi-angle corneal image set is subjected to data denoising preprocessing based on the circular fitting result to obtain the available multi-angle corneal image set.
[0093] Furthermore, the standard multi-angle corneal image set acquisition module 11 in the system is also used for:
[0094] Performing data similarity calculation on each image information in the available multi-angle corneal image set to obtain corneal pixel similarity;
[0095] A similarity probability distribution is constructed according to the corneal pixel similarity, and the similarity probability distribution is specifically: ,in, is the divergence of similarity between the available multi-angle corneal image and the standard multi-angle corneal image, Pixels in the corneal image available at multiple angles With pixels The similarity probability of is the pixel point in the standard multi-angle corneal image With pixels The similarity probability of is a measure of the closeness of the similarity distribution, let The tendency to 0 is used as a constraint condition for dimensionality reduction;
[0096] The available multi-angle corneal image set is subjected to dimensionality reduction processing based on the similarity probability distribution to obtain the standard multi-angle corneal image set.
[0097] Furthermore, the matching key feature point set obtaining module 12 in the system is also used for:
[0098] Acquire pixel description feature factors, wherein the pixel description feature factors include grayscale, texture, and spatial relationship, and perform feature description on the multi-angle corneal image key feature point set based on the pixel description feature factors to obtain a corneal image key feature point description subset;
[0099] Using a cosine similarity algorithm to calculate the similarity of each key feature point in the corneal image key feature point descriptor set, to obtain a key feature point similarity set;
[0100] A similarity matching benchmark is set according to the corneal image characteristic information, and the key feature point similarity set is screened and registered based on the similarity matching benchmark to obtain the matching key feature point set.
[0101] Furthermore, the key feature point depth map generating module 13 in the system is also used for:
[0102] Based on the set of matching key feature points, a reference image is selected from the available multi-angle corneal image set to obtain reference corneal image information and a set of other corneal image information;
[0103] Constructing a reference feature three-dimensional coordinate system according to a reference key feature point set of the reference corneal image information and a depth map of the key feature points;
[0104] Based on the reference feature three-dimensional coordinate system, the reference corneal image information and the remaining corneal image information sets are stereoscopically stitched and fused to obtain corneal feature point fused image information;
[0105] The overlapping image area of the corneal feature point fusion image information is enhanced in clarity to obtain corneal enhanced fusion image information, and a three-dimensional model is reconstructed based on the corneal enhanced fusion image information to obtain the corneal stereoscopic three-dimensional panoramic model.
[0106] Furthermore, the key feature point depth map generating module 13 in the system is also used for:
[0107] Performing posture association mapping on the remaining key feature point sets of the remaining corneal image information sets based on the reference feature three-dimensional coordinate system to obtain fusion conversion type information of the remaining feature points;
[0108] Determine the fusion conversion feature matrix information of the remaining feature points according to the fusion conversion type information of the remaining feature points;
[0109] The reference corneal image information and the remaining corneal image information sets are stereoscopically spliced and fused based on the remaining feature point fusion conversion feature matrix information to obtain the corneal feature point fusion image information.
[0110] Furthermore, the key feature point depth map generating module 13 in the system is also used for:
[0111] Divide the overlapping image area of the corneal feature point fusion image information into grids to obtain a set of overlapping image grid areas;
[0112] The clarity of the overlapping image grid area set is evaluated by using a clarity multi-dimensional index to obtain a fused image grid area clarity set;
[0113] The grid areas in the fusion image grid area clarity set that do not meet the preset clarity threshold are subjected to clarity comparison and enhancement to determine corneal enhanced fusion image information.
[0114] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The method and specific examples of the aforementioned embodiment 1 for reconstructing a 3D corneal model based on optical images are also applicable to the system for reconstructing a 3D corneal model based on optical images in this embodiment. Through the aforementioned detailed description of the method for reconstructing a 3D corneal model based on optical images, those skilled in the art can clearly understand the system for reconstructing a 3D corneal model based on optical images in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.
[0115] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
[0116] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is also intended to include these modifications and variations.
Claims
1. A method for reconstructing a corneal 3D model based on optical images, characterized in that: The method comprises: S1: acquiring a multi-angle corneal image set by an optical scanning device, performing denoising preprocessing on the multi-angle corneal image set to acquire a usable multi-angle corneal image set, performing data dimension reduction based on the usable multi-angle corneal image set to acquire a standard multi-angle corneal image set; S2: extracting key feature points based on the available multi-angle corneal image set to obtain a multi-angle corneal image key feature point set, and performing similarity analysis based on the multi-angle corneal image key feature point set to obtain a matching key feature point set; S3: performing depth estimation on each feature point in the key feature point set of the multi-angle corneal image based on the matching key feature point set to obtain a key feature point depth value set, and generating a key feature point depth map according to the key feature point depth value set; S4: performing stereo stitching fusion and three-dimensional model reconstruction on the available multi-angle corneal image set according to the matching key feature point set and the key feature point depth map to obtain a three-dimensional panoramic model of the cornea; Wherein, obtaining a set of available multi-angle corneal images in S1 includes: S11: extracting and identifying edge features of the multi-angle corneal image set by using the Canny operator to obtain a corneal edge recognition information set; S12: establishing a circular fitting rule according to the corneal edge recognition information set, and determining a circular fitting parameter based on the circular fitting rule; S13: randomly selecting three points in the multi-angle corneal image set, connecting them to obtain a fitting circle as a circular optimal solution, and iteratively fitting the circular optimal solution using the circular fitting parameters to obtain a circular fitting result; S14: performing data denoising preprocessing on the multi-angle corneal image set based on the circular fitting result to obtain the usable multi-angle corneal image set.
2. The method according to claim 1, characterized in that The standard multi-angle corneal image set obtained in S1 includes: Performing data similarity calculation on each image information in the available multi-angle corneal image set to obtain corneal pixel similarity; A similarity probability distribution is constructed according to the corneal pixel similarity, and the similarity probability distribution is specifically: in, is the divergence of similarity between the available multi-angle corneal image and the standard multi-angle corneal image, Pixels in the corneal image available at multiple angles With pixels The similarity probability of is the pixel point in the standard multi-angle corneal image With pixels The similarity probability of is a measure of the closeness of the similarity distribution, let The tendency to 0 is used as a constraint condition for dimensionality reduction; The available multi-angle corneal image set is subjected to dimensionality reduction processing based on the similarity probability distribution to obtain the standard multi-angle corneal image set.
3. The method according to claim 1, characterized in that The set of matching key feature points obtained in S2 includes: S21: Acquire pixel description feature factors, wherein the pixel description feature factors include grayscale, texture, and spatial relationship, and perform feature description on the multi-angle corneal image key feature point set based on the pixel description feature factors to obtain a corneal image key feature point description subset; S22: using a cosine similarity algorithm to calculate the similarity of each key feature point in the corneal image key feature point descriptor set to obtain a key feature point similarity set; S23: Setting a similarity matching benchmark according to the corneal image characteristic information, and performing screening and registration analysis on the key feature point similarity set based on the similarity matching benchmark to obtain the matching key feature point set.
4. The method according to claim 1, characterized in that The key feature point depth value set obtained in S3 includes: S31: selecting a reference image from the available multi-angle corneal image set based on the matching key feature point set to obtain reference corneal image information and a set of other corneal image information; S32: constructing a reference feature three-dimensional coordinate system according to the reference key feature point set of the reference corneal image information and the key feature point depth map; S33: Stereoscopically stitching and fusing the reference corneal image information and the remaining corneal image information sets based on the reference feature three-dimensional coordinate system to obtain corneal feature point fused image information; S34: enhancing the clarity of the overlapping image area of the corneal feature point fusion image information to obtain corneal enhanced fusion image information, and reconstructing a three-dimensional model based on the corneal enhanced fusion image information to obtain the corneal stereoscopic three-dimensional panoramic model.
5. The method according to claim 4, characterized in that The step S33 of obtaining corneal feature point fusion image information includes: Performing posture association mapping on the remaining key feature point sets of the remaining corneal image information sets based on the reference feature three-dimensional coordinate system to obtain fusion conversion type information of the remaining feature points; Determine the fusion conversion feature matrix information of the remaining feature points according to the fusion conversion type information of the remaining feature points; The reference corneal image information and the remaining corneal image information sets are stereoscopically spliced and fused based on the remaining feature point fusion conversion feature matrix information to obtain the corneal feature point fusion image information.
6. The method according to claim 4, characterized in that The cornea enhanced fusion image information is obtained in S34, including: Divide the overlapping image area of the corneal feature point fusion image information into grids to obtain a set of overlapping image grid areas; The clarity of the overlapping image grid area set is evaluated by using a clarity multi-dimensional index to obtain a fused image grid area clarity set; The grid areas in the fusion image grid area clarity set that do not meet the preset clarity threshold are subjected to clarity comparison and enhancement to determine corneal enhanced fusion image information.
7. A system for reconstructing a 3D model of the cornea based on optical images, characterized in that: For implementing the steps of the method according to any one of claims 1 to 6, the system comprises: A standard multi-angle corneal image set acquisition module, the standard multi-angle corneal image set acquisition module is used to acquire a multi-angle corneal image set through an optical scanning device, perform denoising preprocessing on the multi-angle corneal image set, acquire a usable multi-angle corneal image set, perform data dimension reduction based on the usable multi-angle corneal image set, and acquire a standard multi-angle corneal image set; A matching key feature point set acquisition module, the matching key feature point set acquisition module is used to extract key feature points based on the available multi-angle corneal image set to obtain a multi-angle corneal image key feature point set, and perform similarity analysis based on the multi-angle corneal image key feature point set to obtain a matching key feature point set; A key feature point depth map generation module, the key feature point depth map generation module is used to perform depth estimation on each feature point in the multi-angle corneal image key feature point set based on the matching key feature point set, obtain a key feature point depth value set, and generate a key feature point depth map according to the key feature point depth value set; A corneal stereoscopic three-dimensional panoramic model acquisition module is used to perform stereoscopic stitching and fusion and three-dimensional model reconstruction on the available multi-angle corneal image set according to the matching key feature point set and the key feature point depth map to obtain a corneal stereoscopic three-dimensional panoramic model.
Citation Information
Patent Citations
Method and device for detecting retinopathy of premature infant
CN114862760A
Orthokeratology lens optimization method and system based on 3D retina myopia out-of-focus plane
CN116597072A
Endoscopic image observation method and system based on 3D imaging
CN116912407A
Eye movement tracking method and device, electronic equipment and computer storage medium
CN117409038A
