Inter-frame registration and three-dimensional drawing method and system based on anterior segment OCT sequence image

Abstract

The invention relates to the technical field of medical image processing, in particular to an anterior segment OCT sequence image-based inter-frame registration and three-dimensional drawing method and system, and the method comprises the steps: constructing an anterior segment structure segmentation model, taking an image as input, and outputting the segmentation results of the front and rear surfaces of a cornea; collecting an image and inputting the image into the model, and extracting a cornea contour point set based on a segmentation result output by the model; calculating a horizontal offset amount of the image by using the image registration module, and performing horizontal registration based on the horizontal offset amount; extracting a middle column from the horizontally registered image, combining to form a new image, inputting the new image into the model, generating a longitudinal reference curve based on a segmentation result newly output by the model, calculating a vertical offset of the horizontally registered image, and executing vertical registration based on the vertical offset; and based on the registered image, executing three-dimensional visual drawing in a mode of combining volume rendering and surface rendering to generate a three-dimensional model. According to the invention, the registration precision and the model can be correlated with each other, and the visual accuracy and the structure reduction degree of the three-dimensional model are improved.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method and system for inter-frame registration and three-dimensional rendering based on anterior segment OCT sequence images. Background Technology

[0002] Optical coherence tomography (OCT) technology, with its advantages of being non-invasive, high-resolution, and fast, has become a core imaging method for anterior segment structure detection and morphological analysis, and is widely used in clinical examinations and surgical planning of tissues such as the cornea, iris, and lens. Traditional OCT imaging mainly uses single-frame tomographic images, which can only present the two-dimensional cross-sectional structure of local tissues and cannot fully reflect the overall morphological characteristics of the anterior segment. With the increasing clinical needs, how to achieve three-dimensional reconstruction and visualization of the anterior segment based on multi-frame OCT image sequences has gradually become an important development direction in the field of ophthalmic imaging. By performing three-dimensional rendering of OCT sequence images, it is possible to transform from local structural observation to overall spatial morphological analysis, enabling doctors to intuitively understand the changes in the curvature of the anterior and posterior corneal surfaces, the morphological structure of the iris, and the spatial positional relationship of the lens, providing more valuable visual information for refractive surgery design, lesion localization, and preoperative assessment.

[0003] However, in actual OCT image acquisition, involuntary eye movements (including blinking, slight tremors, and gaze deviation) can cause spatial misalignment, translation, or rotation differences between adjacent frames, disrupting the spatial continuity of the image sequence. If these unregistered OCT sequences are used directly for 3D reconstruction, the superposition error will be amplified during the rendering process, causing distortions such as corneal surface distortion, iris boundary breakage, and lens position shift. This results in the generated 3D model failing to accurately reflect the anterior segment's anatomical morphology, thus affecting the accuracy of clinical diagnosis and surgical planning. To eliminate these effects, researchers have proposed various image registration techniques, including cross-correlation methods based on grayscale matching, SIFT / SURF algorithms based on feature points, and registration models based on deep learning that have emerged in recent years. While the first two traditional methods can correct image shifts to some extent, their stability and accuracy are clearly insufficient when dealing with low contrast, strong speckle noise, and complex curved surface structures in OCT images. Although deep learning methods have strong feature extraction and nonlinear mapping capabilities, they are mostly limited to local inter-frame registration and fail to constrain image consistency at the global spatial level. Therefore, error accumulation problems still occur when the sequence length is large or the curvature of the structure changes significantly.

[0004] On the other hand, existing 3D rendering methods mainly include volume rendering and surface rendering. Volume rendering can directly reconstruct the overall structure in grayscale based on OCT volume data, but its key anatomical interfaces are blurred, making it difficult to accurately distinguish tissue boundaries. Surface rendering, while achieving a clear external shape display through surface meshing, cannot reflect internal tissue layers and is highly sensitive to registration errors. If the input sequence of images is not strictly aligned, the reconstructed model will exhibit significant distortion. Therefore, in existing technologies, image registration and 3D rendering are always interdependent and continuous processes: the accuracy of registration directly determines the realism of the 3D rendering, while the geometric consistency of the rendering result, in turn, reflects the registration quality. If the two are processed separately, error propagation and reference drift between data interfaces will significantly reduce the overall modeling accuracy, making it impossible for the system to achieve high-fidelity spatial reconstruction of the anterior segment structure.

[0005] In summary, while existing methods can achieve registration or 3D rendering of OCT images separately, there is a lack of effective integration between the two. Insufficient registration accuracy can lead to misalignment or distortion in the subsequent 3D model, while the rendering process cannot correct the morphological distortion caused by inter-frame offsets, resulting in a disjointed structure and unrealistic details in the reconstructed result. Due to the lack of a holistic solution that balances registration accuracy and rendering realism in the same process, the currently obtained 3D models still fail to accurately reflect the true spatial structure of the anterior segment, limiting their application in clinical diagnosis and surgical planning. Summary of the Invention

[0006] This application provides a method and system for inter-frame registration and 3D rendering based on anterior segment OCT sequence images, which can correlate registration accuracy with model realism, improving the visual accuracy and structural fidelity of the 3D model. This application provides the following technical solutions: In a first aspect, this application provides a method for inter-frame registration and 3D rendering based on anterior segment OCT sequence images, the method comprising: An anterior segment structure segmentation model is constructed, which takes an anterior segment OCT image as input and outputs segmentation results including the anterior and posterior surfaces of the cornea; Anterior segment OCT sequence images are acquired and preprocessed. The preprocessed anterior segment OCT sequence images are then input into the anterior segment structure segmentation model. Based on the segmentation results of the anterior and posterior corneal surfaces output by the model, a corneal contour point set is extracted. The horizontal offset of the sequence image frames is calculated using the corneal contour point set, and horizontal registration of the anterior segment OCT sequence image is performed based on the horizontal offset; The middle column of the horizontally registered anterior segment OCT sequence image is extracted, merged to form a new image, and the new image is input into the anterior segment structure segmentation model. Based on the segmentation results of the anterior and posterior corneal surfaces newly output by the model, a longitudinal baseline curve is generated. The vertical offset of the horizontally registered sequence of image frames is calculated using the longitudinal reference curve, and vertical registration of the anterior segment OCT sequence of images is performed based on the vertical offset. Based on the registered anterior segment OCT sequence images, a three-dimensional visualization rendering is performed by combining volume rendering and surface rendering to generate a three-dimensional morphological model of the anterior segment.

[0007] In one specific implementation, the acquisition and preprocessing of anterior segment OCT sequence images, the input of the preprocessed anterior segment OCT sequence images into the anterior segment structure segmentation model, and the extraction of corneal contour point sets based on the segmentation results of the anterior and posterior corneal surfaces output by the model include: The size of each OCT image frame is checked, and its deviation from the reference size is calculated as follows: ; in, and The first The height and width of the frame image, and Use the reference dimensions; if discrepancies exist, adjust the image to the model's input dimensions using interpolation. The image is subjected to grayscale normalization, and the processed image is input into the anterior segment structure segmentation model. The anterior segment structure segmentation model outputs a segmentation mask image of the anterior and posterior surfaces of the cornea. The contour point set is extracted from the mask image as follows: ; Among them, each point pair This corresponds to a pixel coordinate point on the anterior or posterior surface boundary of the cornea in the mask image. This represents the number of pixels identified on the corneal boundary; after extracting the contour point set, a smoothing filter is applied to the point set.

[0008] In one specific implementation, calculating the horizontal offset of the sequence image frames using the corneal contour point set, and performing horizontal registration of the anterior segment OCT sequence images based on the horizontal offset, includes: Determine the first from the corneal contour point set Horizontal coordinates of the corneal vertex of the frame image And select the corneal vertex coordinates of the reference image frame. As an alignment reference, calculate the... The horizontal offset of the frame relative to the reference image frame is shown below: ; After calculating the horizontal offset, the anterior segment OCT sequence image is translated horizontally. Pixels are used to complete the correction.

[0009] In one specific implementation, the step of extracting the middle column from the horizontally registered anterior segment OCT sequence image, merging them to form a new image, and inputting the new image into the anterior segment structure segmentation model, and generating a longitudinal reference curve based on the segmentation results of the anterior and posterior corneal surfaces newly output by the model, includes: Define the original sequence image as ,in For frame index, For row coordinates, If we use column coordinates, then the merged new image can be represented as: ; in, Define the image width; convert the image... Input the anterior segment structure segmentation model to obtain segmentation mask images of the anterior and posterior corneal surfaces, and extract the point set of the anterior corneal surface from them. and for point sets Performing a quadratic polynomial fitting yields the following longitudinal baseline curve: ; in, , , These are the fitting coefficients obtained using the least squares method.

[0010] In one specific implementation, the step of calculating the vertical offset of the horizontally registered sequence of image frames using the longitudinal reference curve, and performing vertical registration of the anterior segment OCT sequence image based on the vertical offset, includes: Based on the longitudinal reference curve Calculate the first The actual position of the corneal vertex in the vertical direction of the frame image Its vertical offset can be expressed as: ; Based on this offset, the first Frame image translated in the vertical direction Pixel calibration complete.

[0011] In one specific implementation, the step of generating a three-dimensional morphological model of the anterior segment by performing three-dimensional visualization rendering based on the registered anterior segment OCT sequence image through a combination of volume rendering and surface rendering includes: The registered anterior segment OCT sequence images are reconstructed into three-dimensional volume data according to the frame index direction and the physical spacing is corrected to establish the correspondence between two-dimensional pixel coordinates and actual spatial coordinates. Based on 3D volume data, volume rendering parameters are configured and rendering mapping is constructed. GPU acceleration and segmented opacity transfer function are used to realize volume visualization rendering of the anterior segment tissue hierarchy. A three-dimensional mesh surface model is constructed from the set of multi-structure contour points output from the anterior segment structure segmentation model. The surface rendering results are then fused with the volume rendering data to form a complete three-dimensional structural representation. An interactive multi-faceted observation and perspective control mechanism is established in the 3D rendering results, and the anterior segment structure is visualized from multiple angles through slice translation, rotation and scaling operations.

[0012] In a specific feasible implementation, the establishment of an interactive multi-faceted observation and perspective control mechanism in the 3D rendering results, and the realization of multi-angle visualization of the anterior segment structure through slice translation, rotation, and scaling operations, includes: In the plotting results, construct X / Y / Z triorthogonal slice windows, corresponding to the transverse, sagittal, and coronal planes respectively; the slice positions are updated according to the formula: in, This is the initial slice position. For the interaction step size, This is the interaction direction vector.

[0013] Secondly, this application provides an inter-frame registration and 3D rendering system based on anterior segment OCT sequence images, employing the following technical solution: An inter-frame registration and 3D rendering system based on anterior segment OCT sequence images, comprising: The model building module is used to build an anterior segment structure segmentation model. The anterior segment structure segmentation model takes an anterior segment OCT image as input and outputs segmentation results including the anterior and posterior surfaces of the cornea. The image acquisition module is used to acquire and preprocess anterior segment OCT sequence images, input the preprocessed anterior segment OCT sequence images into the anterior segment structure segmentation model, and extract the corneal contour point set based on the segmentation results of the anterior and posterior corneal surfaces output by the model; The horizontal registration module is used to calculate the horizontal offset of the sequence image frames using the corneal contour point set, and to perform horizontal registration of the anterior segment OCT sequence images based on the horizontal offset. The benchmark generation module is used to extract the middle column of the horizontally registered anterior segment OCT sequence image, merge them to form a new image, and input the new image into the anterior segment structure segmentation model. Based on the segmentation results of the anterior and posterior corneal surfaces newly output by the model, a longitudinal benchmark curve is generated. The vertical registration module is used to calculate the vertical offset of the horizontally registered sequence of image frames using the longitudinal reference curve, and to perform vertical registration of the anterior segment OCT sequence image based on the vertical offset. The 3D rendering module is used to perform 3D visualization rendering based on the registered anterior segment OCT sequence images, using a combination of volume rendering and surface rendering to generate a 3D morphological model of the anterior segment.

[0014] Thirdly, this application provides an electronic device, the device including a processor and a memory; the memory stores a program, the program being loaded and executed by the processor to implement a method for inter-frame registration and three-dimensional rendering based on anterior segment OCT sequence images as described in the first aspect.

[0015] Fourthly, this application provides a computer-readable storage medium storing a program that, when executed by a processor, is used to implement a method for inter-frame registration and three-dimensional rendering based on anterior segment OCT sequence images as described in the first aspect.

[0016] In summary, the beneficial effects of this application include at least the following: (1) By employing the U-Net++ deep learning model optimized for the features of anterior segment OCT grayscale images, automated segmentation of key anatomical structures such as the anterior and posterior surfaces of the cornea and iris was achieved. The model's structural design incorporates a lightweight encoder and a spatial-channel attention mechanism, resulting in higher accuracy and stability in boundary recognition and morphological preservation. This automated segmentation process avoids the subjective biases and inefficiencies inherent in traditional manual drawing or semi-automatic segmentation methods, significantly improving the consistency and repeatability of image structure recognition. Simultaneously, the segmentation output mask map and contour point set provide clear and stable anatomical benchmarks for inter-frame registration, enabling subsequent registration operations to be completed under real structural constraints, thus ensuring the accuracy of inter-frame position correction and the overall reliability of the model.

[0017] (2) In the inter-frame registration stage, a unified registration benchmark is established based on the fitting curve of the anterior corneal surface, and horizontal and vertical dual-dimensional registration logic is designed to simultaneously correct the lateral and vertical offsets caused by micro-movements of the eyeball. This registration strategy, combined with a multi-threaded parallel processing mechanism, realizes the rapid calculation of inter-frame offsets and batch translation correction, significantly improving the efficiency of registration operations and eliminating the dependence on manual intervention. By accurately correcting the spatial misalignment between sequential images, the generated three-dimensional volume data maintains global consistency in the coordinate system, avoiding three-dimensional reconstruction distortion caused by inter-frame misalignment or distortion, thus establishing a stable geometric foundation for high-precision three-dimensional rendering of the anterior segment structure.

[0018] (3) In terms of overall design, an integrated automated pipeline was constructed, and interactive visualization functions such as triorthogonal slice linkage, adjustable rendering parameters, slice switching, and viewpoint saving were introduced. This integrated solution not only realizes the complete reconstruction of the three-dimensional structure of the anterior segment, but also significantly improves the problems of incomplete lesion identification and ambiguous positioning in traditional two-dimensional image reading. Doctors can simultaneously observe the lesion area through multi-angle slices, and combine adjustable lighting and transparency settings to intuitively analyze the corneal, iris, and lens structures, thereby achieving rapid lesion positioning and accurate interpretation.

[0019] First, an anterior segment structure segmentation model is constructed to achieve automatic identification and segmentation of the anterior and posterior corneal surfaces in OCT images. Then, by extracting corneal contour point sets, the inter-frame offsets in the horizontal and vertical directions of the image sequence are calculated and corrected to achieve spatial alignment of the image sequence. After completing inter-frame registration, volume rendering and surface rendering techniques are combined to perform 3D visualization processing on the registered image sequence, generating a complete 3D morphological model of the anterior segment. By introducing a registration benchmark based on the segmentation results, inter-frame alignment is ensured to be referenced to anatomical structures, fundamentally improving spatial consistency. Simultaneously, 3D rendering is performed within a unified framework, linking registration accuracy with model realism. The resulting 3D morphological model possesses a continuous, stable, and anatomically consistent spatial structure, improving not only the model's visual accuracy and structural fidelity but also its usability and reliability in clinical diagnosis and surgical planning.

[0020] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the inter-frame registration and 3D rendering method based on anterior segment OCT sequence images in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of a three-dimensional morphological model of the anterior segment in an embodiment of this application.

[0023] Figure 3 This is a structural block diagram of the inter-frame registration and 3D rendering system based on anterior segment OCT sequence images in the embodiments of this application.

[0024] Figure 4 This is a block diagram of an electronic device based on inter-frame registration and 3D rendering of anterior segment OCT sequence images in an embodiment of this application. Detailed Implementation

[0025] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0026] Optionally, this application uses the inter-frame registration and three-dimensional rendering method based on anterior segment OCT sequence images provided in various embodiments as an example for application in an electronic device. The electronic device is a terminal or server. The terminal can be a computer, tablet computer, etc. This embodiment does not limit the type of electronic device.

[0027] Reference Figure 1 This is a flowchart illustrating a method for inter-frame registration and 3D rendering based on anterior segment OCT sequence images provided in an embodiment of this application. The method includes at least the following steps: Step S101: Construct an anterior segment structure segmentation model. The anterior segment structure segmentation model takes the anterior segment OCT image as input and outputs the segmentation results including the anterior and posterior surfaces of the cornea.

[0028] In step S101, an anterior segment structure segmentation model is constructed to perform pixel-level segmentation of key anatomical structures such as the cornea, iris, and lens in the anterior segment OCT image. This step uses the segmentation model to perform semantic parsing and region recognition on the input OCT image, so that the spatial boundaries of the anterior and posterior surfaces of the cornea, the anterior and posterior surfaces of the iris, and the anterior surface of the lens are accurately marked in the image, resulting in a segmentation result that accurately reflects the hierarchical relationship of the anterior segment tissues.

[0029] Specifically, the anterior segment structure segmentation model employs an improved U-Net++ network architecture. The encoder uses a lightweight tf_efficientnet_lite0 backbone network to balance feature extraction capability and computational efficiency, adapting to the feature representation requirements of grayscale single-channel OCT images. The decoder uses a depthwise separable convolutional structure to reduce the number of model parameters, and embeds an SCSE spatial-channel attention module in the feature fusion layer to enhance the recognition of detailed features such as corneal edges, iris contours, and the anterior surface of the lens. The model output layer has five segmentation channels, including background, anterior and posterior corneal surfaces, anterior and posterior iris surfaces, and the anterior surface of the lens.

[0030] In implementation, the model training uses a joint loss function. ,in, and These are all weighting coefficients, used to control the proportion of different loss terms in the overall optimization process. This is a similarity loss function based on overlap, used to improve the segmentation accuracy of small target regions. Weighted cross-entropy loss is used to balance the sample ratio of different classes. By assigning higher weights to rare classes, the model maintains stable segmentation performance when identifying structures with small areas (such as the anterior surface of the lens). The AdamW optimizer is used during training to achieve adaptive parameter updates and weight decay control, preventing overfitting due to gradient accumulation. Mixed-precision training is employed to reduce memory usage and improve computational efficiency, while a gradient clipping threshold is set during backpropagation to limit numerical instability caused by excessive gradients. An early stopping mechanism is implemented during training; training terminates when the validation set accuracy fails to improve over several consecutive iterations, ensuring model convergence stability while avoiding performance degradation. Validation on multiple sample sets shows that the model achieves an average Dice coefficient of over 0.95 in segmentation tasks of key structures such as the cornea, iris, and lens, indicating high consistency in boundary positions and structural morphology.

[0031] The trained anterior segment structure segmentation model can output multi-structure segmentation results from input OCT tomographic images. The output includes, but is not limited to, segmentation mask images of the anterior and posterior surfaces of the cornea, iris, and lens. The segmentation mask image of the anterior and posterior cornea is used to extract corneal contour point sets, while the segmentation results of the iris and lens regions provide morphological information of the corresponding structures. Through the segmentation output of this model, clear, continuous anterior segment structure data with defined spatial boundaries can be obtained at the image level.

[0032] Step S102: Acquire anterior segment OCT sequence images and preprocess them. Input the preprocessed anterior segment OCT sequence images into the anterior segment structure segmentation model. Extract the corneal contour point set based on the segmentation results of the anterior and posterior corneal surfaces output by the model.

[0033] In step S102, the acquired anterior segment OCT sequence images are read, preprocessed, and the segmentation results of the anterior and posterior corneal surfaces are generated using the anterior segment structure segmentation model constructed in step S101. This step ensures that each frame of the image meets the input requirements of the segmentation model by standardizing image size and grayscale standards, and simultaneously performs pixel-level annotation of the corneal structure to obtain the spatial boundary information of the anterior and posterior corneal surfaces. Through this processing, the corneal contour of each frame of the image is accurately identified at a uniform scale, providing basic structural data for the horizontal and vertical correction of the image.

[0034] Specifically, the size of each OCT image frame is first checked, and the deviation from the reference size is calculated as follows: ; in, and The first The height and width of the frame image, and The image is used as a reference size. If discrepancies exist, the image is adjusted to the model's input size via interpolation. The image is then normalized to eliminate the influence of imaging differences, and the processed image is input into the trained anterior segment structure segmentation model. The anterior segment structure segmentation model outputs a segmentation mask image of the anterior and posterior corneal surfaces. The contour point set is extracted from the mask image as follows: ; Among them, each point pair This corresponds to a pixel coordinate point on the anterior or posterior surface boundary of the cornea in the mask image. This represents the number of pixels identified on the corneal boundary. After extracting the contour point set, a smoothing filter is applied to the point set to eliminate noise and boundary jitter, ensuring a continuous and stable contour. This process yields a clear, continuous corneal contour point set with defined spatial boundaries for each frame, which can be directly used for spatial alignment operations, restoring spatial consistency between frames in a sequence of images.

[0035] Step S103: Calculate the horizontal offset of the sequence image frames using the corneal contour point set, and perform horizontal registration of the anterior segment OCT sequence images based on the horizontal offset.

[0036] In step S103, based on the corneal contour point set obtained in step S102, the horizontal positional offset of each frame in the sequence is calculated, and the anterior segment OCT sequence images are translated accordingly. This step corrects inter-frame misalignment caused by micro-movements of the eyeball or device jitter by accurately measuring the horizontal displacement of the corneal apex, thus ensuring a consistent spatial positional relationship in the horizontal direction of the sequence images.

[0037] Specifically, the first step is to determine the first corneal contour point set. Horizontal coordinates of the corneal vertex of the frame image And select the corneal vertex coordinates of the reference image frame. As an alignment reference, calculate the... The horizontal offset of the frame relative to the reference image frame is shown below: ; After calculating the horizontal offset, the anterior segment OCT sequence image is translated horizontally. Pixel calibration is completed. Furthermore, to improve computational efficiency, the image sequence is divided into multiple batches, and a multi-threaded parallel processing mode is used to implement batch translation operations. During registration, the integrity of pixel data is maintained to prevent missing boundary information or morphological distortion, ensuring the corneal contour is continuous and stable in the translated image. After this step, the sequence images are consistently aligned horizontally, the corneal structure maintains a uniform horizontal position across frames, and the overall spatial reference of the sequence is effectively established, thus ensuring the accurate restoration of horizontal geometric relationships between frames.

[0038] Step S104: Extract the middle column from the horizontally registered anterior segment OCT sequence image, merge them to form a new image, and input the new image into the anterior segment structure segmentation model. Generate a longitudinal baseline curve based on the segmentation results of the anterior and posterior corneal surfaces newly output by the model.

[0039] In step S104, for the anterior segment OCT sequence image that has been horizontally registered in step S103, column pixel data located at the middle position of the image width are extracted from each frame image, and the middle columns of each frame are sequentially stitched together according to the frame index order to form a new two-dimensional image. This step extracts longitudinal structural information from the central region of the cornea, displaying the overall changes in corneal morphology across a sequence of images in a single image, thus providing input data for fitting the longitudinal baseline.

[0040] Specifically, the original sequence image is defined as ,in For frame index, For row coordinates, If we use column coordinates, then the merged new image can be represented as: ; in, The image width. (This refers to the image's width.) The anterior segment structure segmentation model constructed in step S101 is input for segmentation inference to obtain segmentation mask images of the anterior and posterior corneal surfaces, and the point set of the anterior corneal surface is extracted from them. and for point sets Performing a quadratic polynomial fitting yields the following longitudinal baseline curve: ; in, , , The fitting coefficients, obtained using the least squares method, describe the continuous variation trend of the anterior corneal surface in the longitudinal direction. The fitted longitudinal reference curve smoothly represents the overall morphology of the corneal apex, enabling a continuous representation of the geometric features of the anterior corneal surface in the longitudinal direction. After this step, the resulting longitudinal reference curve accurately reflects the overall longitudinal morphology of the anterior corneal surface of the anterior segment, providing a stable geometric reference for vertical registration calculations and ensuring the spatial consistency of the image sequence in the longitudinal direction.

[0041] Step S105: Calculate the vertical offset of the horizontally registered sequence image frames using the longitudinal reference curve, and perform vertical registration of the anterior segment OCT sequence image based on the vertical offset.

[0042] In step S105, the vertical reference curve generated in step S104 is used to perform vertical registration processing on the horizontally registered anterior segment OCT sequence images. This step measures the positional deviation of the corneal vertex in each frame image relative to the reference curve in the vertical direction, and uses this as a basis for translation to achieve uniform alignment of the image sequence in the vertical direction, thereby eliminating vertical misalignment caused by slight eye movements or acquisition jitter.

[0043] Specifically, based on the longitudinal baseline curve Calculate the first The actual position of the corneal vertex in the vertical direction of the frame image Its vertical offset can be expressed as: ; Based on this offset, the first Frame image translated in the vertical direction Pixel calibration is completed. To improve processing efficiency, the image sequence is divided into multiple batches, and a multi-threaded parallel approach is used to perform batch vertical registration. During registration, pixel data integrity is maintained to prevent edge information truncation or structural deformation, ensuring the continuity and geometric stability of the corneal contour in the vertical direction. After this step, the image sequence is precisely aligned in the vertical direction, the corneal structure maintains a consistent vertical position across frames, the overall spatial coordinate system of the sequence is unified, and the longitudinal geometric relationships between frames are accurately restored.

[0044] Step S106: Based on the registered anterior segment OCT sequence image, perform three-dimensional visualization rendering by combining volume rendering and surface rendering to generate a three-dimensional morphological model of the anterior segment.

[0045] In step S106, based on the horizontally and vertically registered anterior segment OCT sequence images, three-dimensional volume data is constructed and three-dimensional visualization is performed. This step uses a combination of volume rendering and surface rendering to reconstruct the registered two-dimensional tomographic image sequence into a three-dimensional model with realistic spatial proportions and structural details, so that key anatomical structures of the anterior segment such as the cornea, iris, and lens are fully presented in three-dimensional space.

[0046] Step S106 includes at least the following sub-steps: Step S1061: Reassemble the registered anterior segment OCT sequence image into three-dimensional volume data according to the frame index direction and perform physical spacing correction to establish the correspondence between two-dimensional pixel coordinates and actual spatial coordinates.

[0047] In step S1061, the data format is first converted to reassemble the registered OCT sequence image dataset into three-dimensional volume data. ,in , For single-frame pixel coordinates, This is the frame index. The mapping from pixel coordinates to physical coordinates is achieved through physical spacing correction, as shown below: ; in, , , These represent the physical distances (usually in millimeters) between adjacent pixels in the x, y, and z directions, respectively. They are used to map discrete pixel coordinates to continuous spatial physical coordinates, thereby ensuring the accuracy of the 3D reconstruction scale.

[0048] Step S1062: Configure volume rendering parameters and construct rendering mapping based on 3D volume data, and use GPU acceleration and segmented opacity transfer function to realize volume visualization rendering of the anterior segment organization hierarchy.

[0049] In step S1062, during volume rendering, a volume rendering core mapper is used and GPU-accelerated rendering is enabled to balance rendering efficiency and detail fidelity, thus meeting the real-time visualization needs of large-scale OCT data. Based on the OCT grayscale distribution characteristics, a segmented opacity transfer function is constructed, setting different opacity coefficients for different grayscale ranges of tissues such as the cornea, iris, and lens, achieving layered transparency between background noise and key tissues, thereby highlighting tissue boundaries and suppressing speckle interference. Rendering parameters include the light projection direction along the Z-axis, the adaptive sampling step size, and the anti-aliasing sampling mechanism, balancing rendering accuracy and real-time performance. During rendering, a light source is added, configuring ambient light and diffuse light intensity, with the light source pointing towards the volume data center to enhance the three-dimensional depth of the structure.

[0050] Step S1063: Construct a three-dimensional mesh surface model from the set of multi-structure contour points output from the anterior segment structure segmentation model, and fuse the surface rendering results with the volume rendering data to form a complete three-dimensional structure representation.

[0051] In step S1063, surface rendering information is overlaid on the volume rendering results. Three-dimensional contour point sets of the anterior and posterior corneal surfaces, anterior and posterior iris surfaces, and anterior lens surface are extracted from the anterior segment structure segmentation model output. Each point set contains spatial coordinates corrected for physical spacing. Triangulation is performed on these point sets to generate a closed three-dimensional curved surface mesh model, and adaptive meshing is used in the corneal surface region to ensure accurate restoration of high-curvature details. The surface rendering results are fused with the volume rendering data to achieve synchronous presentation of the internal tissue grayscale distribution and the external structural contours, forming a complete three-dimensional structural model.

[0052] Step S1064: Establish an interactive multi-faceted observation and perspective control mechanism in the 3D rendering results, and realize multi-angle visualization of the anterior segment structure through slice translation, rotation and scaling operations.

[0053] In step S1064, to enhance visualization interactivity, an X / Y / Z triorthogonal slice window is constructed in the rendering result, corresponding to the transverse, sagittal, and coronal planes, respectively. The slice positions are updated according to the formula: in, This is the initial slice position. For the interaction step size, This is the interaction direction vector. The system supports moving a single slice by dragging with the mouse, while keeping the other two slices fixed; it supports 3D view rotation in 1° increments and scaling with a scaling factor of 0.1, achieving a smooth interactive experience through camera parameter control. Users can save frequently used viewing angles; a typical viewing angle is a 45° downward tilt of the corneal module, which can fully display the corneal structure. It also supports dynamic adjustment of opacity mapping parameters and one-click restoration of preset viewing angles to improve image reading efficiency.

[0054] After this processing step, the registered anterior segment OCT sequence images are converted into a three-dimensional visualization model with spatial depth and structural hierarchy. This model accurately reflects the spatial distribution and morphological characteristics of structures such as the cornea, iris, and lens in a unified coordinate system, achieving a complete reconstruction from two-dimensional tomographic data to real three-dimensional structures, and possessing good clinical observation, diagnostic, and assessment value.

[0055] In summary, firstly, a pixel-level segmentation model is used to segment key anatomical structures such as the cornea, iris, and lens in OCT images, obtaining a corneal contour point set and spatial segmentation information for other structures. Then, the horizontal and vertical offsets are calculated using the corneal contour point set and the longitudinal reference curve, respectively, to accurately register the entire image sequence. This ensures that the spatial positional relationships of each frame are consistent in the horizontal and vertical directions, eliminating inter-frame misalignment caused by eye movements or device jitter. The image sequence after inter-frame registration not only maintains the continuity and spatial consistency of each structure but also provides a precise geometric basis for subsequent 3D rendering, ensuring the accuracy and stability of the structural positions during the rendering process.

[0056] In the 3D visualization stage, this solution employs a hybrid approach combining volumetric and surface rendering. It maps 2D image sequences into 3D volumetric data through physical spacing correction and utilizes a piecewise opacity transfer function and ray casting sampling strategy to achieve a clear 3D representation with distinct grayscale levels and sharp details. Simultaneously, by extracting the 3D contour point set from the segmentation results and performing meshing, the 3D surfaces of the cornea, iris, and lens are accurately reconstructed to reproduce their anatomical morphology. This integrated solution effectively solves the problem of registration and rendering disconnect in existing technologies, ensuring that the 3D model possesses both inter-frame spatial consistency and realistically reflects the anatomical details of the anterior segment. This overcomes defects such as discontinuous model structure, morphological distortion, and unrealistic details, making the generated 3D reconstruction results more reliable and valuable for clinical diagnosis, preoperative assessment, and surgical planning.

[0057] An example is provided below: The method of this application can be directly applied to the intelligent analysis workflow of anterior segment OCT clinical images. During the examination, ophthalmologists can use a conventional anterior segment OCT device to scan the patient's target eye (e.g., the right eye) in an anterior segment region scanning mode, obtaining multiple OCT tomographic images covering the cornea, iris, and lens. After acquisition, the image sequence is stored in a standardized format and automatically imported into the diagnostic software integrating this method. After data import, the software can automatically perform the entire process without manual intervention, including image size verification and normalization preprocessing, mask inference of the anterior segment structure segmentation model, multi-threaded horizontal and vertical registration operations to eliminate inter-frame misalignment caused by eyeball micro-movements, physical spacing correction, and hybrid 3D rendering. After processing, a high-fidelity 3D visualization result corresponding to the patient's actual anterior segment anatomy can be generated.

[0058] Reference Figure 2This diagram illustrates a use case of a three-dimensional anterior segment morphological model in an embodiment of this application. During the three-dimensional morphological analysis stage, doctors can use the software's interactive visualization function to rotate, scale, and observe the three-dimensional model from multiple angles. They can simultaneously view the cross-sectional, sagittal, and coronal structural distribution using a triorthogonal slice view. By adjusting rendering parameters, corneal transparency can be achieved to observe the internal morphological characteristics of the iris and lens, thereby providing a direct analysis of key physiological parameters such as corneal curvature, iris thickness, and anterior chamber depth, offering quantitative evidence for disease diagnosis, preoperative assessment, and postoperative monitoring. The processed OCT sequence images, registration results, three-dimensional reconstruction data, and system-generated diagnostic reports can be automatically linked and archived into the electronic medical record system, achieving integrated management of patient image data and structured diagnostic results.

[0059] Figure 3 This is a structural block diagram of an inter-frame registration and 3D rendering system based on anterior segment OCT sequence images provided in one embodiment of this application. The system includes at least the following modules: The model building module is used to build an anterior segment structure segmentation model. The anterior segment structure segmentation model takes an anterior segment OCT image as input and outputs segmentation results including the anterior and posterior surfaces of the cornea. The image acquisition module is used to acquire and preprocess anterior segment OCT sequence images, input the preprocessed anterior segment OCT sequence images into the anterior segment structure segmentation model, and extract the corneal contour point set based on the segmentation results of the anterior and posterior corneal surfaces output by the model; The horizontal registration module is used to calculate the horizontal offset of the sequence of image frames using the corneal contour point set, and to perform horizontal registration of the anterior segment OCT sequence images based on the horizontal offset. The baseline generation module is used to extract the middle column of the horizontally registered anterior segment OCT sequence image, merge them to form a new image, and input the new image into the anterior segment structure segmentation model. Based on the segmentation results of the anterior and posterior corneal surfaces newly output by the model, a longitudinal baseline curve is generated. The vertical registration module is used to calculate the vertical offset of the horizontally registered sequence of image frames using the vertical reference curve, and to perform vertical registration of the anterior segment OCT sequence image based on the vertical offset. The 3D rendering module is used to perform 3D visualization rendering based on the registered anterior segment OCT sequence images, using a combination of volume rendering and surface rendering to generate a 3D morphological model of the anterior segment.

[0060] For relevant details, please refer to the above method implementation examples.

[0061] Figure 4 This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 401 and a memory 402.

[0062] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0063] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 402 is used to store at least one instruction, which is executed by the processor 401 to implement the inter-frame registration and 3D rendering method based on anterior segment OCT sequence images provided in the method embodiments of this application.

[0064] In some embodiments, the electronic device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 401, memory 402, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to: radio frequency circuits, touch displays, audio circuits, and power supplies.

[0065] Of course, electronic devices may also include fewer or more components, and this embodiment does not limit this.

[0066] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the inter-frame registration and 3D rendering method based on anterior segment OCT sequence images in the above method embodiments.

[0067] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program, which is loaded and executed by a processor to implement the inter-frame registration and 3D rendering method based on anterior segment OCT sequence images of the above method embodiments.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for inter-frame registration and three-dimensional rendering based on anterior segment OCT sequence images, characterized in that, The method comprises: constructing an anterior segment structure segmentation model, which takes an anterior segment OCT image as input and outputs segmentation results of corneal anterior and posterior surfaces; collecting an anterior segment OCT sequence image and pre-processing the same, inputting the pre-processed anterior segment OCT sequence image into the anterior segment structure segmentation model, and extracting a corneal contour point set based on segmentation results of corneal anterior and posterior surfaces output by the model; calculating a horizontal offset of a sequence image frame by using the corneal contour point set, and performing horizontal registration of the anterior segment OCT sequence image based on the horizontal offset; merging intermediate columns of the horizontally registered anterior segment OCT sequence image to form a new image, inputting the new image into the anterior segment structure segmentation model, and generating a longitudinal reference curve based on segmentation results of corneal anterior and posterior surfaces newly output by the model; calculating a vertical offset of a horizontally registered sequence image frame by using the longitudinal reference curve, and performing vertical registration of the anterior segment OCT sequence image based on the vertical offset; performing three-dimensional visualization rendering in a manner combining volume rendering and surface rendering based on the registered anterior segment OCT sequence image, and generating an anterior segment three-dimensional morphology model.

2. The frame registration and three-dimensional rendering method based on anterior segment OCT sequence images according to claim 1, characterized in that, The collecting an anterior segment OCT sequence image and pre-processing the same, inputting the pre-processed anterior segment OCT sequence image into the anterior segment structure segmentation model, and extracting a corneal contour point set based on segmentation results of corneal anterior and posterior surfaces output by the model comprises: performing size checking on each OCT image, and calculating a deviation of the same from a reference size as follows: ； wherein, and are the height and width of the frame image, respectively, are the reference dimensions; if there is a difference the image is adjusted to the input dimensions of the model by interpolation.​​ performing gray scale normalization on the image, inputting the processed image into the anterior segment structure segmentation model, and outputting a segmentation mask image of corneal anterior and posterior surfaces by the anterior segment structure segmentation model, and extracting a contour point set from the mask image as follows: ； wherein each pair of points corresponding to a pixel coordinate point on the boundary of the anterior or posterior surface of the cornea, is the number of pixel points identified on the boundary of the cornea; after the contour point set is extracted, a smoothing filtering process is performed on the point set. 3.The frame registration and three-dimensional rendering method based on anterior segment OCT sequence images of claim 1, wherein, The calculating a horizontal offset of a sequence image frame by using the corneal contour point set, and performing horizontal registration of the anterior segment OCT sequence image based on the horizontal offset comprises: determining a first corneal vertex horizontal coordinate of the frame image and selecting a corneal vertex coordinate of the reference image frame as an alignment reference, calculating a first horizontal offset of the frame relative to the reference image frame is given by ； After the horizontal offset is calculated, the anterior segment OCT sequence image is translated along the horizontal direction pixels to complete the correction.

4. The method of claim 1, wherein, The merging intermediate columns of the horizontally registered anterior segment OCT sequence image to form a new image, inputting the new image into the anterior segment structure segmentation model, and generating a longitudinal reference curve based on segmentation results of corneal anterior and posterior surfaces newly output by the model comprises: The original sequence images are defined as wherein is a frame index, is a row coordinate, is a column coordinate, the merged new image can be represented as: ； wherein, is the image width; the image The anterior segment structure segmentation model is inputted to obtain a segmentation mask image of the corneal anterior and posterior surfaces, and a point set of the corneal anterior surface is extracted from the segmentation mask image The point set is A quadratic polynomial fitting is performed to obtain a longitudinal reference curve as follows: ； wherein , , are the fitting coefficients obtained by the least square method.

5. The method of frame registration and 3D rendering based on anterior segment OCT sequence images according to claim 4, wherein, The calculating a vertical offset of a horizontally registered sequence image frame by using the longitudinal reference curve, and performing vertical registration of the anterior segment OCT sequence image based on the vertical offset comprises: According to the longitudinal reference curve Calculating the The actual position of the frame image corneal vertex in the longitudinal direction The vertical offset of which can be expressed as: ； According to the offset, the first frame image is translated in the vertical direction pixels are corrected.

6. The method of frame registration and 3D rendering based on anterior segment OCT sequence images according to claim 1, wherein, The performing three-dimensional visualization rendering in a manner combining volume rendering and surface rendering based on the registered anterior segment OCT sequence image, and generating an anterior segment three-dimensional morphology model comprises: reorganizing the registered anterior segment OCT sequence image into three-dimensional volume data in a frame index direction and performing physical interval correction to establish a corresponding relationship between two-dimensional pixel coordinates and actual spatial coordinates; performing volume rendering parameter configuration and rendering mapping construction based on the three-dimensional volume data, and realizing volume visualization rendering of anterior segment tissue levels by using GPU acceleration and segmented opacity transfer functions. A three-dimensional mesh surface model is constructed from the multiple structure contour points output from the anterior segment structure segmentation model, and the surface rendering result is fused with the volume rendering volume data to form a complete three-dimensional structure representation; An interactive multi-slice observation and view control mechanism is established in the three-dimensional rendering result, and multi-angle visual display of the anterior segment structure is realized through slice translation, rotation and scaling operations.

7. The method of claim 6, wherein, The interactive multi-slice observation and view control mechanism in the three-dimensional rendering result includes: An X / Y / Z three-orthogonal slice window is constructed in the rendering result, corresponding to the transverse, sagittal and coronal planes respectively; the update of the slice position follows the formula: wherein, is an initial slice position, is an interaction step size, is an interaction direction vector.

8. An inter-frame registration and three-dimensional mapping system based on anterior segment OCT sequence images, characterized in that, It includes: A model construction module is configured to construct an anterior segment structure segmentation model, which takes an anterior segment OCT image as input and outputs a segmentation result including corneal anterior and posterior surfaces; An image acquisition module is configured to acquire an anterior segment OCT sequence image and pre-process it, input the pre-processed anterior segment OCT sequence image into the anterior segment structure segmentation model, and extract a corneal contour point set based on the segmentation result of the corneal anterior and posterior surfaces output by the model; A horizontal registration module is configured to calculate a horizontal offset of the sequence image frames using the corneal contour point set, and perform horizontal registration of the anterior segment OCT sequence image based on the horizontal offset; A reference generation module is configured to merge the intermediate columns of the horizontally registered anterior segment OCT sequence image to form a new image, input the new image into the anterior segment structure segmentation model, and generate a longitudinal reference curve based on the segmentation result of the corneal anterior and posterior surfaces newly output by the model; A vertical registration module is configured to calculate a vertical offset of the horizontally registered sequence image frames using the longitudinal reference curve, and perform vertical registration of the anterior segment OCT sequence image based on the vertical offset; A three-dimensional rendering module is configured to perform three-dimensional visualization rendering by combining volume rendering and surface rendering based on the registered anterior segment OCT sequence image, and generate an anterior segment three-dimensional morphological model.

9. An electronic device, comprising: The device includes a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement the frame-to-frame registration and three-dimensional rendering method based on an anterior segment OCT sequence image according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program, which is executed by the processor to implement the frame-to-frame registration and three-dimensional rendering method based on an anterior segment OCT sequence image according to any one of claims 1 to 7.

Images