A corneal topography alignment method based on placido ring image and laser ranging fusion

Abstract

The application provides a corneal topographic alignment method based on Placido ring image and laser ranging fusion, belongs to the technical field of image processing of ophthalmic diagnostic equipment, and utilizes a single auxiliary sensor-free main camera to collect an RGB original image; a centering branch process and a focusing branch process are synchronously and in parallel executed on the same frame of RGB original image to solve XY axis centering deviation and Z axis focusing deviation; wherein the centering branch process adopts two sets of DCSA-Unet models with consistent architecture to sequentially complete coarse positioning of a pupil and fine positioning of a Placido inner circle; horizontal centering deviation threshold value, vertical centering deviation threshold value and focusing deviation threshold value are synchronously checked, and when the three deviations are all less than the corresponding preset threshold values, corneal image shooting is triggered; if any one of the deviations exceeds the threshold value, the position of the main camera is adjusted to re-collect the RGB original image. Thus, only by relying on single camera homologous image collection, three-axis full-automatic closed-loop alignment can be realized through a double-branch parallel algorithm combined with a lightweight DCSA-Unet network.

Description

Technical Field

[0001] This application belongs to the field of image processing technology for ophthalmic diagnostic equipment, and specifically relates to a corneal topography alignment method based on the fusion of Placido ring image and laser ranging. Background Technology

[0002] Corneal topography relies on acquiring the ring-shaped reflection image projected onto the corneal surface by the Placido disc to reconstruct the three-dimensional curvature of the cornea. It is a core device for clinical corneal refractive and keratoconus screening. Precise centering and focusing are required before imaging: centering requires the center of the Placido ring to coincide with the camera's optical axis, and focusing requires the device to maintain a fixed standard working distance from the cornea. The device has a small field of view (25°) and a depth of field of only 2mm, allowing for a focusing error of ≤0.1mm. Even a slight deviation can cause imaging failure; therefore, fully automatic centering and focusing are core performance indicators of the device.

[0003] Existing autofocus solutions are mainly divided into three categories, each with its own inherent flaws: Manual / semi-automatic adjustment scheme: Relies on the operator to observe the crosshairs on the screen and manually move the device. Focusing depends entirely on human experience. It has a low degree of automation, takes a long time to check, has a large human error, and requires a high level of professionalism from the operator.

[0004] The multi-sensor collaborative automatic solution involves adding an independent infrared auxiliary camera, a ToF / ultrasonic ranging sensor, and a dedicated infrared illumination source. The main camera is only responsible for final imaging. Placido ring center detection uses Canny edge detection combined with traditional image processing algorithms such as Hough circle transform and least squares circle fitting. This solution's hardware stacking results in high overall cost and complex internal structure. Multiple sensors require repeated spatial calibration, leading to continuous accumulation of calibration errors and reduced positioning accuracy.

[0005] Deep learning-assisted post-processing solution: Existing deep learning models are only used for ring image segmentation and lesion identification after image acquisition, and are not integrated into the closed-loop control process of device motion, so they cannot achieve real-time automatic centering and focus adjustment. Summary of the Invention

[0006] In view of this, the purpose of this application is to provide a corneal topographic mapping alignment method based on the fusion of Placido ring image and laser ranging, which can achieve three-axis fully automatic closed-loop alignment by relying on a single camera to acquire images from the same source and by combining a dual-branch parallel algorithm with a lightweight DCSA-Unet network.

[0007] The system uses a single sensorless main camera to capture raw RGB images that simultaneously include Placido ring reflections, human eye pupils, and red laser reflection spots. The centering branch process and the focus branch process are executed synchronously and in parallel on the same frame of RGB original image. The centering branch process uses two DCSA-Unet models with the same architecture to complete the coarse pupil localization, the fine Placido inner circle localization and solve the XY axis centering deviation in sequence. The focus branch process sequentially performs red light channel extraction, narrow ROI cropping, adaptive threshold segmentation, morphological denoising, multi-constraint contour filtering, multi-priority spot selection and solve the Z axis focus deviation. The system synchronously verifies the horizontal centering deviation threshold, the vertical centering deviation threshold, and the focus deviation threshold. When all three deviations are less than the corresponding preset thresholds, corneal image capture is triggered. If any deviation exceeds the threshold, the position of the main camera is adjusted and the original RGB image is re-acquired.

[0008] In some embodiments, the central branching process includes the following steps: The original image is preprocessed and scaled before being fed into the first DCSA-Unet model to complete pupil semantic segmentation. The pupil centroid is solved by connected component filtering to obtain the pupil center coordinates (pupilCenter). Using pullilCenter as a reference, crop and scale the local ROI, and feed it into the second DCSA-Unet model with the same architecture as the first DCSA-Unet model to segment the inner circle of Placido. After contour filtering, restore the local coordinates to the global coordinates of the original image to obtain the global inner circle center coordinates inner_center. The horizontal and vertical two-dimensional offsets relative to the image center are calculated based on inner_center to obtain the XY axis centering deviation.

[0009] In some embodiments, after contour filtering, the local coordinates are restored to the original image's global coordinates to obtain the global inner circle center coordinates inner_center, including the following steps: The obtained Placido inner circle segmentation results are filtered according to the area ratio of connected regions to obtain the effective inner circle connected regions. Find the centroid of the effective inner circle connected domain as the local inner circle center point, and calculate the distance from the local inner circle center to the edge of the effective inner circle connected domain to obtain the inner circle radius; The coordinates of the inner circle center corresponding to the local ROI image are mapped back to the global coordinate system of the original image to obtain the global inner circle center coordinates inner_center.

[0010] In some embodiments, the middle branch process is also configured with a fault tolerance and degradation mechanism: During coarse pupil localization, if no effective pupil connected region is detected, the geometric center of the entire image is used directly to replace pupilCenter as the local ROI cropping reference; during fine Placido inner circle localization, if no effective Placido inner circle contour is segmented, the current frame image is discarded.

[0011] In some embodiments, the focusing branch process includes the following steps: The red channel is extracted by channel separation of the original image. Based on the physical constraint that the laser spot only moves horizontally and its vertical position is fixed, a narrow horizontal strip of the region of interest is cropped to obtain a locally processed image of the laser spot. The local image of the light spot is binarized using Gaussian weighted adaptive thresholding, and an erosion operation is performed using a rectangular erosion kernel to obtain a binary image of the light spot. All connected contours are extracted from the binary image of the light spot. After filtering by multiple constraints, a set of candidate contours is obtained. Then, the optimal center coordinates of the light spot are obtained by selecting the candidate contours by applying multiple priorities. The horizontal offset relative to the center of the original image is calculated based on the optimal spot center coordinates, and this horizontal offset is used as the Z-axis focusing deviation.

[0012] In some embodiments, the process of obtaining a candidate contour set after multi-constraint contour filtering, and then obtaining the optimal spot center coordinates by applying multi-priority spot selection to the candidate contour set, includes the following steps: The extracted connected contours are subjected to multiple constraints, including contour area, number of contour points, mean gray value of region, and aspect ratio of large contour, to remove stray light spots and invalid reflections, thus obtaining a set of candidate contours. A two-level competition strategy is used to screen the candidate contour set for a unique main spot; wherein, the first priority is to select the spot with a larger contour area and a position further to the right; when there is no spot that meets the first priority, the spot with a larger contour area and a position closer to the center of the image is selected, and the optimal spot center coordinates are output.

[0013] In some embodiments, the DCSA-Unet model uses MobileNetV2 inverse residual depthwise separable convolution to build an encoder and integrates an SPPM spatial pyramid pooling module.

[0014] In some embodiments, a corneal topography alignment device based on Placido ring image and laser ranging fusion is also provided, the device comprising: The image acquisition module is used to acquire RGB raw images containing Placido ring reflection, human eye pupil, and red laser reflection spot using a single main camera without auxiliary sensors. The image processing module is used to synchronously and in parallel execute a centering branch process and a focusing branch process on the same frame of RGB raw image. The centering branch process uses two DCSA-Unet models with the same architecture to sequentially complete coarse pupil localization, fine Placido inner circle localization, and solve the XY axis centering deviation. The focusing branch process sequentially executes red light channel extraction, narrow ROI cropping, adaptive threshold segmentation, morphological denoising, multi-constraint contour filtering, multi-priority spot selection, and then solves the Z-axis focusing deviation. The verification execution module is used to synchronously verify the horizontal centering deviation threshold, the vertical centering deviation threshold, and the focus deviation threshold. When all three deviations are less than the corresponding preset thresholds, corneal image capture is triggered. If any deviation exceeds the threshold, the position of the main camera is adjusted and the original RGB image is re-acquired.

[0015] In some embodiments, an electronic device is also provided, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the corneal topography alignment method based on Placido ring image and laser ranging fusion as described above are performed.

[0016] In some embodiments, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, performs the steps of the corneal topography alignment method based on Placido ring image and laser ranging fusion as described above.

[0017] The corneal topography alignment method described in this application, based on Placido ring image and laser ranging fusion, completes all imaging acquisition using only a single main camera. It achieves coarse pupil localization and fine Placido inner circle localization through a lightweight dual-cascaded DCSA-Unet deep learning model, and is equipped with laser spot extraction and optimization algorithms. The centering and focus deviations of a single frame image are calculated in parallel, and the motion mechanism is driven in real time to complete the joint alignment, achieving low-cost, highly robust, high-speed fully automatic shooting. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1A flowchart of the corneal topography alignment method based on Placido ring image and laser ranging fusion described in the embodiments of this application is shown; Figure 2 A flowchart of the branching process in an embodiment of this application is shown; Figure 3 A schematic diagram of the structure of the DCSA-Unet model according to an embodiment of this application is shown; Figure 4 A flowchart of the focusing branch in an embodiment of this application is shown; Figure 5 This illustration shows a schematic diagram of the corneal topography alignment device based on Placido ring image and laser ranging fusion as described in an embodiment of this application. Figure 6 A schematic diagram of the structure of the electronic device described in an embodiment of this application is shown. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0021] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0023] In view of the technical problems raised in the background, this application provides a corneal topographic mapping alignment method based on the fusion of Placido ring image and laser ranging, which can achieve three-axis fully automatic closed-loop alignment by relying on a single camera to acquire images from the same source and by combining a dual-branch parallel algorithm with a lightweight DCSA-Unet network.

[0024] See the instruction manual appendix Figure 1 This application provides a corneal topography alignment method based on Placido ring image and laser ranging fusion, the method comprising the following steps: S1. Use a single main camera without auxiliary sensors to acquire RGB raw images that simultaneously include Placido ring reflection, human eye pupil, and red laser reflection spot; S2. The centering branch process and the focus branch process are executed synchronously and in parallel on the same frame of RGB original image. The centering branch process uses two DCSA-Unet models with the same architecture to complete the coarse pupil localization, the fine Placido inner circle localization and solve the XY axis centering deviation in sequence. The focus branch process executes the red light channel extraction, narrow ROI cropping, adaptive threshold segmentation, morphological denoising, multi-constraint contour filtering, multi-priority spot selection and solve the Z axis focus deviation in sequence. S3. Synchronously verify the horizontal centering deviation threshold, vertical centering deviation threshold, and focus deviation threshold. When all three deviations are less than the corresponding preset thresholds, trigger corneal image capture. If any deviation exceeds the threshold, adjust the position of the main camera and re-capture the original RGB image.

[0025] Specifically, step S1 mainly involves acquiring a single-frame RGB image that simultaneously carries three key visual features: pupil, Placido ring reflection, and laser reflection spot. This provides a unified data source for subsequent centering and focusing operations. Notably, only a single main camera is used for acquisition, eliminating the need for multi-sensor spatial calibration and removing alignment errors caused by multiple devices at the source. Furthermore, a single image frame contains all the features required for planar centering and depth focusing, eliminating the need for multiple image captures and improving overall detection speed.

[0026] Step S2 mainly involves performing two independent algorithm operations in parallel based on the same frame of acquired image, outputting the plane XY centering deviation and the depth Z-axis focusing deviation respectively, and quantifying the two-dimensional offset of the current human eye relative to the camera optical axis and the working distance error.

[0027] The centering branch uses a two-level DCSA-Unet segmentation model with identical architecture to accurately locate the center of the Placido inner ring and calculates the horizontal and vertical plane offsets to correct the centering position of the human eye within the camera frame. It also incorporates a two-level fault-tolerant degradation mechanism to address clinical interference scenarios such as closed eyes, eyelash obstruction, and significant defocusing, improving the algorithm's robustness. The focusing branch extracts the laser reflection spot through red light channel extraction, morphological processing, and a spot selection algorithm, calculating the Z-axis working distance deviation to correct the front-to-back shooting distance between the device and the cornea, ensuring image clarity. Parallel synchronous computation, rather than sequential processing, allows for the simultaneous output of both types of deviations, significantly reducing the processing time per frame and meeting the real-time automatic alignment requirements in clinical settings.

[0028] In one embodiment, see the appendix to the specification. Figure 2 The central branching process includes the following steps: S201. After preprocessing and scaling the original image, it is fed into the first DCSA-Unet model to complete the pupil semantic segmentation. The pupil centroid is solved by connected component filtering to obtain the pupil center coordinates (pupilCenter). S202. Using pullilCenter as the reference, crop and scale the local ROI, and feed it into the second DCSA-Unet model with the same architecture as the first DCSA-Unet model to segment the inner circle of Placido. After contour filtering, restore the local coordinates to the global coordinates of the original image to obtain the global inner circle center coordinates inner_center. S203. Based on inner_center, calculate its horizontal and vertical two-dimensional offset relative to the image center to obtain the XY axis centering deviation.

[0029] In step S201, the raw image acquired by the main camera is input into the first deep learning model (Model A). This model adopts the DCSA-Unet architecture, with an input size of 64×64 pixels and an output of a pupil region segmentation mask. Specific processing flow: Original image preprocessing: Convert the three-channel RGB original image to a single-channel grayscale image, and crop a 2000×2000 pixel area from the center of the 5020×4060 original image; Constructing a 64×64 pupil input image: A 2000×2000 grayscale image is scaled down to 64×64. Pupil localization is a coarse localization stage; the core objective is simply to obtain the pupil center position, without requiring precise boundary segmentation. Extensive experimental verification shows that reducing the input size from 128×128 to 64×64 results in a pupil center localization error change of less than 2 pixels and a reduction in model inference computation of approximately 35%. Balancing localization accuracy and overall real-time performance, 64×64 is chosen as the input size for model A. DCSA-Unet pupil segmentation: Input a [64,64,1] grayscale image into the model to complete inference, and output the binary segmentation result of the pupil region; Connected component filtering: Non-pupil clutter such as eyelashes and light spots may exist within the segmentation mask. Filtering is completed by statistically analyzing the area ratio of each connected region, retaining only the regions that match the pupil size characteristics. Calculate the pupil center: Find the centroid of the only valid connected region after filtering, and use it as the coordinates of the pupil center; Output pullilCenter.

[0030] If no valid pupil connected region is detected in this step, the first-level fault tolerance degradation logic is triggered, and the geometric center of the entire original image is directly used to replace pupilCenter as the ROI cropping benchmark.

[0031] In step S202, a 200×200 pixel local ROI region is cropped centered on the pupilCenter (or the geometric center of the image after first-level downgrading) and fed into the second deep learning model (Model B). Model B uses the exact same DCSA-Unet network architecture as Model A, only the training dataset is different. The training samples for Model B are Placido disk inner circle segmentation images, and the input size is uniformly set to 128×128 pixels. Specific processing flow: Constructing a ROI using pupilCenter: Using the pupil centroid as a reference, cropping the image within a 200×200 rectangle as the local processing screen; ROI scaling to 128×128: ​​The 200×200 locally cropped image is scaled to 128×128 to balance inference speed and contour accuracy. Placido inner circle localization is a fine-tuning step; the deviation of the circle's center coordinates directly determines the device's planar alignment accuracy. Compared to coarse-tuning pupil segmentation, this step needs to completely preserve the details of the annular edge. Experiments show that a 128×128 input size effectively avoids edge blurring caused by scaling and completely restores the inner circle's contour features; therefore, model B uses a 128×128 input specification. DCSA-Unet Inner Circle Segmentation: Input a grayscale image of [128,128,1] into the model for inference, and output a Placido inner circle binary segmentation mask; Inner circle contour extraction and filtering: The model output mask contains non-inner circle interference reflective areas such as eyelashes and corneal spots. The filtering is completed according to the area ratio of connected regions, and only a single continuous effective inner circle connected region without large area defects is retained. Inner circle center fitting: Find the centroid of the effective inner circle connected region obtained by screening, and use it as the center point of the inner circle in the local ROI coordinate system; Inner circle radius estimation: Calculate the pixel distance from the local inner circle center to the edge of the connected domain of the effective inner circle to obtain the inner circle radius; Coordinate restoration: Map and restore the coordinates of the inner circle center point under the 200×200 local ROI coordinate system to the original 2000×2000 large image global coordinate system; Output the global inner circle center, inner_center.

[0032] If, after performing the first-level degradation with the image center as the cropping reference, this step still cannot segment an effective Placido inner circle contour, then the second-level degradation is triggered, directly discarding the current entire frame image and waiting for the next frame of the original image to completely run the centering branch process again.

[0033] In step S203, the two-dimensional offset of the global center of the Placido inner circle relative to the geometric center of the original image is calculated using the following formula: inner_circle_center_diff.y = inner_center.y - image.rows / 2 inner_circle_center_diff.x = inner_center.x - image.cols / 2 Wherein, inner_center.y is the vertical pixel coordinate of the global inner circle center inner_center in the original large image coordinate system; inner_center.x is the horizontal pixel coordinate of the global inner circle center inner_center in the original large image coordinate system; image.rows is the total number of rows of pixels in the original RGB image, representing the total vertical height of the image; image.cols is the total number of columns of pixels in the original RGB image, representing the total horizontal width of the image; inner_circle_center_diff.y corresponds to the vertical offset Δy, representing the vertical difference between the Placido inner circle center and the image center; inner_circle_center_diff.x corresponds to the horizontal offset Δx, representing the horizontal difference between the Placido inner circle center and the image center.

[0034] The horizontal offset Δx and the vertical offset Δy together constitute the XY axis alignment deviation, which serves as the closed-loop control input for the three-axis motion mechanism. By adjusting the position of the main camera plane and continuously bringing the XY axis alignment deviation close to zero, the center of the inner circle of the Placido coincides with the center of the camera's optical axis, and the plane alignment is completed.

[0035] Additionally, see the instruction manual appendix. Figure 3This application addresses the characteristics of pupils and the inner circle of Placido, which are regular geometric targets with limited texture information and a limited range of scale variations. It employs a lightweight encoder based on the MobileNetV2 inverse residual structure to replace the traditional U-Net encoder. The inverse residual structure utilizes depthwise separable convolution to decompose standard convolutions into depthwise and pointwise convolutions, significantly reducing the number of parameters and computational cost while maintaining feature representation capabilities. Furthermore, it combines the SPPM spatial pyramid pooling module to obtain multi-scale global contextual information, enabling the network to accurately segment the pupil and the inner circle of Placido with fewer parameters. Compared to the traditional U-Net structure, the model in this application reduces the number of parameters by more than 95% and the computational cost by more than 90%, meeting the requirement of single-frame inference within 50ms during real-time automatic centering in corneal topography. Table 1 shows the parameter table of the DCSA-Unet model in this application, and Table 2 shows the parameter table of DCSA-Unet compared to the traditional U-Net.

[0036]

[0037] Table 1

[0038] Table 2 Compared to the classic U-Net network, this application addresses the need for regular geometric target segmentation in automatic corneal topography centering tasks by employing a lightweight encoder based on an inverse residual structure and a SPPM multi-scale pooling module instead of the traditional high-channel convolution stacking structure. With an input size of 128×128, the total number of parameters in this application's model is approximately 0.196M, only 0.7% of U-Net's; the total computational cost is approximately 45.6MFLOPs, only 0.4% of U-Net's. While maintaining the segmentation accuracy of the pupil and Placido inner circle, it significantly reduces model storage overhead and inference time, making it more suitable for real-time automatic centering scenarios in corneal topography.

[0039] See the instruction manual appendix Figure 4 The central branching process includes the following steps: M201. Perform channel separation on the original image to extract the red channel. Based on the physical constraint that the laser spot only moves horizontally and its vertical position is fixed, extract a narrow horizontal strip of the region of interest to obtain a locally processed image of the spot. M202. The local image of the light spot is binarized using Gaussian weighted adaptive thresholding, and an erosion operation is performed using a rectangular erosion kernel to obtain a binary image of the light spot. M203. Extract all connected contours from the binary image of the light spot, and obtain a set of candidate contours after filtering by multiple constraints. Then, select the optimal light spot center coordinates by applying multiple priority light spot selection to the set of candidate contours. M204. Calculate the horizontal offset of the spot relative to the center of the original image based on the optimal spot center coordinates, and use the horizontal offset as the Z-axis focusing deviation.

[0040] Step M201 primarily involves extracting the red channel and cropping the narrow ROI. Specifically, the original RGB image is split into red, green, and blue channels, and the red channel image is extracted separately. The red laser spot has the highest brightness response in the red channel, which can be significantly distinguished from the uniformly distributed Placido ring reflections across multiple channels, facilitating subsequent spot segmentation and extraction. When cropping the narrow ROI, a horizontally narrow ROI region with a width of 400 pixels and a height of 100 pixels is cropped from the center of the entire original image as a dedicated image for spot processing. This cropping method relies on the inherent physical constraints of the device's optical path: the red laser reflection spot only shifts horizontally along the image's horizontal direction with changes in the device's working distance, while the vertical coordinates remain fixed. This large vertical cropping directly removes invalid background pixels, significantly reducing the computational load of subsequent image processing.

[0041] Step M202 primarily involves adaptive thresholding and morphological denoising. Specifically, it calls the Gaussian weighted adaptive thresholding algorithm (ADAPTIVE_THRESH_GAUSSIAN_C) to perform binary image segmentation, with fixed configuration parameters: a neighborhood block size of 155 pixels and a correction constant C=7. This adaptive thresholding scheme can adapt to differences in corneal reflectivity and tear film reflectivity among different patients, and can stably separate weaker laser spots even against the complex background of high-intensity ring reflections in the Placido ring. During morphological denoising, a 7×7 rectangular erosion kernel is used to perform a single erosion operation on the binary image, removing the outlines of tiny noise points formed by dust and minor debris, while completely preserving the connected regions of the laser spot itself without breakage or damage.

[0042] Step M203 mainly involves performing multi-constraint contour filtering and multi-priority spot selection. Specifically, it traverses the binary image to extract all closed connected contours, and filters out invalid reflections and noisy contours layer by layer through four hard constraints, retaining only the candidate contour set that conforms to the morphological characteristics of the laser spot. The filtering constraint criteria are as follows: Contour area constraint: pixel area range of 300~9000, filtering out extremely small noise and large diffuse reflection areas of the cornea; Contour point count constraint: The number of pixels at the contour boundary is 10~180, eliminating incomplete shapes and small, scattered patches; Regional grayscale mean constraint: The average grayscale value inside the contour is ≥60 to ensure that the extracted target is a bright and realistic reflective spot; Aspect Ratio Constraint: Only applies to large outlines with a pixel area greater than 600. The aspect ratio is controlled in the range of 0.7 to 2.0 to ensure that the light spot outline is close to a circle or ellipse and to eliminate long strip-shaped stray reflections.

[0043] A two-level competition strategy is used to determine the unique optimal target spot for all candidate contours after screening. The first priority prioritizes spots with larger contour areas and a lateral coordinate biased towards the right side of the image, suitable for scenarios where primary reflective spots and secondary stray spots coexist, locking in the right-side primary spot formed by corneal vertex reflection. When no candidate spot meets the first priority condition, the second priority is activated: the contour with a lateral coordinate closer to the image center and a contour area not significantly smaller than other candidate spots is selected as the optimal spot, suitable for standard situations. The lateral offset of the spot is represented by a signed numerical value; a positive value indicates the spot is located to the right of the image cropping center, and a negative value indicates the spot is located to the left of the cropping center.

[0044] Step M204 primarily involves calculating the Z-axis focusing deviation. Specifically, based on the selected optimal spot center coordinates, the horizontal offset dx is calculated using the following formula: dx = best_center.x - original_image.cols / 2 Where, best_center.x is the horizontal pixel coordinate of the optimal laser spot center in the original complete image coordinate system; original_image.cols is the total number of columns of pixels in the original RGB image, and original_image.cols / 2 is the horizontal midpoint pixel coordinate of the original image; dx represents the horizontal offset of the laser spot relative to the horizontal center of the image, and this value is directly used as the closed-loop control input of the Z-axis (working distance adjustment direction) of the XYZ three-axis motion mechanism; the forward and backward movement direction of the device is determined based on the positive or negative sign of dx, and the movement amplitude is determined based on the absolute value of dx, and the main camera movement distance is continuously adjusted so that dx approaches zero infinitely; when dx=0, the spot formed by the red laser after reflection by the specular apex of the human cornea falls precisely at the center of the optical axis of the main camera. At this time, the distance between the main camera and the cornea is the standard imaging working distance, and the imaging clarity is optimal.

[0045] Step S3 mainly involves uniformly verifying the three-axis deviation output in step S2 to determine whether the current main camera meets the standard shooting conditions. If the standard is not met, the three-axis mechanism is driven to adjust the spatial position of the main camera. If the standard is met, corneal image capture is triggered, forming an automated closed-loop control that eliminates the need for manual adjustment of the equipment.

[0046] Specifically, the system simultaneously reads three sets of deviations: Δx, Δy, and dx, and compares them with preset horizontal, vertical, and focus deviation thresholds for determination. Shooting criteria: Horizontal centering deviation, vertical centering deviation, and Z-axis focus deviation are all less than their respective preset thresholds. At this point, the cornea is centered, the imaging distance matches the standard, the processor outputs a shooting trigger signal, the main camera acquires a high-resolution corneal topography map, and the single automatic alignment process ends. In other words: |inner_circle_center_diff.y| <threshold_y |inner_circle_center_diff.x| <threshold_x |dx| <threshold_focus Where threshold_y is the preset threshold for vertical centering; threshold_x is the preset threshold for horizontal centering; and threshold_focus is the preset threshold for focus.

[0047] Unmet adjustment logic: If any deviation exceeds the threshold, the direction of movement of the three axes is determined based on the sign of each deviation, and the displacement amplitude is determined based on the absolute value of the deviation. The output control signal drives the XYZ three-axis translation stage to drive the main camera to adjust synchronously, continuously making the XY and Z axis deviations approach zero. After the displacement is completed, it automatically jumps to step S1 to re-acquire images, and repeats steps S2 and S3 iterative calculations until all deviations meet the threshold requirements.

[0048] This application provides a corneal topography alignment method based on Placido ring image and laser ranging fusion, which has the following significant advantages compared with the prior art: The hardware is extremely simplified, resulting in significantly reduced costs. This application uses only a single small FOV main camera to complete all image acquisition tasks, eliminating the need for auxiliary cameras, infrared illumination, structured light projectors, or ultrasonic ranging modules. Compared to existing three-sensor solutions, hardware costs are reduced by approximately 40% to 60%, size is reduced, and assembly and calibration processes are simplified. The three functions reuse the same physical image source, eliminating the accumulation of spatial calibration errors from multiple sensors.

[0049] Deep learning-based end-to-end localization significantly improves robustness. Compared to Hough transform and least squares fitting, the dual-model cascade scheme in this application has the following advantages: the model's training data includes eyelash / eyelid occlusion samples, yet it can still effectively locate the image during inference; the model learns semantic-level pupil and inner circle concepts, rather than edge geometric features, thus maintaining localization capabilities even in out-of-focus or blurred scenes; and the training data covers various lighting conditions, resulting in stable model output.

[0050] Centering and focusing are processed in parallel with real-time closed-loop control. Existing solutions typically execute this sequentially (centering first, then focusing). This application outputs both centering and focusing deviations simultaneously per frame, with both degrees of freedom adjusted synchronously. The total processing time per frame is ≤50ms (20fps), meeting the requirements for real-time closed-loop control. The patient only needs to place their chin on the chin rest, and the device automatically tracks and locks onto the optimal shooting position, reducing the single positioning time from 8-12 seconds in existing solutions to 3-4 seconds.

[0051] The laser spot was stably extracted under Placido ring interference. Through a three-step process—red channel separation, vertical narrow ROI cropping, and Gaussian adaptive thresholding—the laser spot was effectively separated from the strong background of the Placido ring. The large blockSize (155) design of the adaptive threshold ensured that the threshold was based on local region statistics containing the spot, rather than global statistics, adapting to differences in corneal reflectivity among different patients. Multi-condition contour screening (area + number of points + grayscale + aspect ratio) further excluded secondary spots and stray reflections.

[0052] Dissolving ambiguity between primary and secondary light spots. Laser light may produce primary and secondary reflection spots on the corneal surface. This application employs a signed horizontal offset and right-biased priority competition strategy to stably select the primary light spot (usually located on the right) when both left and right light spots coexist, avoiding focus misjudgment.

[0053] A dual-model reuse architecture ensures efficient deployment. The two-stage models in the mid-module employ the same DCSA-Unet architecture, differing only in training data and input size. This design allows for shared inference engines and memory pools, reducing model deployment resource consumption and facilitating 50ms real-time inference on embedded platforms.

[0054] Based on the same inventive concept, this application also provides a corneal topography alignment device based on Placido ring image and laser ranging fusion. Since the principle of the device in this application is similar to the corneal topography alignment method based on Placido ring image and laser ranging fusion described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0055] As per the instruction manual Figure 5 As shown in the illustration, this application also provides a corneal topography alignment device based on Placido ring image and laser ranging fusion, the device comprising: Image acquisition module 501 is used to acquire RGB raw images containing Placido ring reflection, human eye pupil and red laser reflection spot using a single main camera without auxiliary sensors; Image processing module 502 is used to synchronously and in parallel execute a centering branch process and a focusing branch process on the same frame of RGB original image; the centering branch process uses two DCSA-Unet models with the same architecture to sequentially complete coarse pupil localization, fine Placido inner circle localization and solve the XY axis centering deviation; the focusing branch process sequentially executes red light channel extraction, narrow ROI cropping, adaptive threshold segmentation, morphological denoising, multi-constraint contour filtering, multi-priority spot selection and then solves the Z axis focusing deviation. The verification execution module 503 is used to synchronously verify the horizontal centering deviation threshold, the vertical centering deviation threshold, and the focus deviation threshold. When all three deviations are less than the corresponding preset thresholds, corneal image capture is triggered. If any deviation exceeds the threshold, the position of the main camera is adjusted and the original RGB image is re-acquired.

[0056] The corneal topography alignment device based on Placido ring image and laser ranging fusion described in this application uses only a single main camera to complete all image acquisition. It achieves coarse pupil localization and fine Placido inner circle localization through a lightweight dual-cascaded DCSA-Unet deep learning model. It is equipped with a laser spot extraction and optimization algorithm, and calculates the centering and focus deviation in parallel for a single frame image. It drives the motion mechanism in real time to complete the joint alignment, realizing low-cost, highly robust, high-speed fully automatic shooting.

[0057] Based on the same concept of the present invention, as shown in the appendix to the specification. Figure 6 As shown in the figure, an embodiment of this application provides the structure of an electronic device 600, which includes: at least one processor 601, at least one network interface 604 or other user interface 603, memory 605, and at least one communication bus 602. The communication bus 602 is used to realize the connection and communication between these components. The electronic device 600 may optionally include a user interface 603, including a display (e.g., touch screen, LCD, CRT, holographic imaging, or projector, etc.), a keyboard, or a clicking device (e.g., mouse, trackball, touchpad, or touch screen, etc.).

[0058] Memory 605 may include read-only memory and random access memory, and provides instructions and data to processor 601. A portion of memory 605 may also include non-volatile random access memory (NVRAM).

[0059] In some implementations, memory 605 stores executable modules or data structures, or subsets thereof, or extended sets thereof: The 6051 operating system contains various system programs used to implement various basic business functions and handle hardware-based tasks. Application module 6052 contains various applications, such as launcher, media player, and browser, to implement various application functions.

[0060] In this embodiment of the application, by calling the program or instructions stored in the memory 605, the processor 601 is used to execute steps such as a corneal topography alignment method based on Placido ring image and laser ranging fusion.

[0061] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs steps such as those in a corneal topography alignment method based on Placido ring image and laser ranging fusion.

[0062] Specifically, the storage medium can be a general-purpose storage medium, such as a portable disk or hard disk. When the computer program on the storage medium is run, it can acquire images from a single camera and achieve three-axis fully automatic closed-loop alignment by combining a dual-branch parallel algorithm with a lightweight DCSA-Unet network.

[0063] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, and the indirect coupling or communication connection of the apparatus or units may be electrical, mechanical, or other forms.

[0064] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0066] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] Finally, it should be noted that the above embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A corneal topography alignment method based on Placido ring image and laser ranging fusion, characterized in that, The method includes the following steps: The system uses a single sensorless main camera to capture raw RGB images that simultaneously include Placido ring reflections, human eye pupils, and red laser reflection spots. The centering branch process and the focus branch process are executed synchronously and in parallel on the same frame of RGB original image. The centering branch process uses two DCSA-Unet models with the same architecture to complete the coarse pupil localization, the fine Placido inner circle localization and solve the XY axis centering deviation in sequence. The focus branch process sequentially performs red light channel extraction, narrow ROI cropping, adaptive threshold segmentation, morphological denoising, multi-constraint contour filtering, multi-priority spot selection and solve the Z axis focus deviation. The system synchronously verifies the horizontal centering deviation threshold, the vertical centering deviation threshold, and the focus deviation threshold. When all three deviations are less than the corresponding preset thresholds, corneal image capture is triggered. If any deviation exceeds the threshold, the position of the main camera is adjusted and the original RGB image is re-acquired.

2. The corneal topography alignment method based on Placido ring image and laser ranging fusion according to claim 1, characterized in that, The central branching process includes the following steps: The original image is preprocessed and scaled before being fed into the first DCSA-Unet model to complete pupil semantic segmentation. The pupil centroid is solved by connected component filtering to obtain the pupil center coordinates (pupilCenter). Using pullilCenter as a reference, crop and scale the local ROI, and feed it into the second DCSA-Unet model with the same architecture as the first DCSA-Unet model to segment the inner circle of Placido. After contour filtering, restore the local coordinates to the global coordinates of the original image to obtain the global inner circle center coordinates inner_center. The horizontal and vertical two-dimensional offsets relative to the image center are calculated based on inner_center to obtain the XY axis centering deviation.

3. The corneal topography alignment method based on Placido ring image and laser ranging fusion according to claim 2, characterized in that, After contour filtering, the local coordinates are restored to the original image's global coordinates to obtain the global inner circle center coordinates (inner_center). This process includes the following steps: The obtained Placido inner circle segmentation results are filtered according to the area ratio of connected regions to obtain the effective inner circle connected regions. Find the centroid of the effective inner circle connected domain as the local inner circle center point, and calculate the distance from the local inner circle center to the edge of the effective inner circle connected domain to obtain the inner circle radius; The coordinates of the inner circle center corresponding to the local ROI image are mapped back to the global coordinate system of the original image to obtain the global inner circle center coordinates inner_center.

4. The corneal topography alignment method based on Placido ring image and laser ranging fusion according to claim 2, characterized in that, The central branch process is also configured with a fault tolerance and degradation mechanism: During coarse pupil localization, if no effective pupil connected region is detected, the geometric center of the entire image is used directly to replace pupilCenter as the local ROI cropping reference; during fine Placido inner circle localization, if no effective Placido inner circle contour is segmented, the current frame image is discarded.

5. The corneal topography alignment method based on Placido ring image and laser ranging fusion according to claim 1, characterized in that, The focusing branch process includes the following steps: The red channel is extracted by channel separation of the original image. Based on the physical constraint that the laser spot only moves horizontally and its vertical position is fixed, a narrow horizontal strip of the region of interest is cropped to obtain a locally processed image of the laser spot. The local image of the light spot is binarized using Gaussian weighted adaptive thresholding, and an erosion operation is performed using a rectangular erosion kernel to obtain a binary image of the light spot. All connected contours are extracted from the binary image of the light spot. After filtering by multiple constraints, a set of candidate contours is obtained. Then, the optimal center coordinates of the light spot are obtained by selecting the candidate contours by applying multiple priorities. The horizontal offset relative to the center of the original image is calculated based on the optimal spot center coordinates, and this horizontal offset is used as the Z-axis focusing deviation.

6. The corneal topography alignment method based on Placido ring image and laser ranging fusion according to claim 5, characterized in that, The process of obtaining a candidate contour set through multi-constraint contour filtering, and then obtaining the optimal spot center coordinates by applying multi-priority spot selection to the candidate contour set, includes the following steps: The extracted connected contours are subjected to multiple constraints, including contour area, number of contour points, mean gray value of region, and aspect ratio of large contour, to remove stray light spots and invalid reflections, thus obtaining a set of candidate contours. A two-level competition strategy is used to screen the candidate contour set for a unique main spot; wherein, the first priority is to select the spot with a larger contour area and a position further to the right; when there is no spot that meets the first priority, the spot with a larger contour area and a position closer to the center of the image is selected, and the optimal spot center coordinates are output.

7. The corneal topography alignment method based on Placido ring image and laser ranging fusion according to claim 1, characterized in that, The DCSA-Unet model uses MobileNetV2 inverse residual depthwise separable convolution to build the encoder and integrates the SPPM spatial pyramid pooling module.

8. A corneal topography alignment device based on Placido ring image and laser ranging fusion, characterized in that, The device includes: The image acquisition module is used to acquire RGB raw images containing Placido ring reflection, human eye pupil, and red laser reflection spot using a single main camera without auxiliary sensors. The image processing module is used to synchronously and in parallel execute a centering branch process and a focusing branch process on the same frame of RGB raw image. The centering branch process uses two DCSA-Unet models with the same architecture to sequentially complete coarse pupil localization, fine Placido inner circle localization, and solve the XY axis centering deviation. The focusing branch process sequentially executes red light channel extraction, narrow ROI cropping, adaptive threshold segmentation, morphological denoising, multi-constraint contour filtering, multi-priority spot selection, and then solves the Z-axis focusing deviation. The verification execution module is used to synchronously verify the horizontal centering deviation threshold, the vertical centering deviation threshold, and the focus deviation threshold. When all three deviations are less than the corresponding preset thresholds, corneal image capture is triggered. If any deviation exceeds the threshold, the position of the main camera is adjusted and the original RGB image is re-acquired.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of a corneal topography alignment method based on Placido ring image and laser ranging fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a corneal topography alignment method based on Placido ring image and laser ranging fusion as described in any one of claims 1 to 7.

Images