A deep learning-based corneal center positioning system for cataract surgery

Abstract

The present invention relates to the field of computer-assisted surgery systems, specifically a method and system for locating the corneal center during cataract surgery based on deep learning. The method comprises: acquiring cataract surgery video data in real time using a video camera; converting the video data into continuous image frames and preprocessing them into an image sequence; constructing a corneal Gaussian heat map generation network based on a deep learning model; inputting the preprocessed image sequence into the corneal Gaussian heat map generation network to obtain a corneal heat map and a center position offset map; and decoding the corneal Gaussian heat map to obtain its peak position and the coordinates of the corneal center. The present invention can accurately and quickly locate the corneal center during cataract surgery.

Description

Technical Field

[0001] The present invention relates to the field of computer-assisted surgery systems, and in particular to a corneal center positioning system in cataract surgery based on deep learning. Background Art

[0002] Cataracts are degenerative changes in the lens' transparency or color, leading to decreased visual quality. The primary cause is changes in lens tissue due to aging or injury. From an evidence-based medicine perspective, there are no effective medications for cataracts, and surgery is the primary treatment. Currently, the main surgical approach is refractive cataract surgery, which combines phacoemulsification with toric intraocular lens (TOL) implantation.

[0003] Precise placement of the intraocular lens (IOL) and preventing its displacement and rotation are key challenges in refractive cataract surgery, significantly impacting postoperative visual recovery for cataract patients. Cataract surgery requires precise alignment of the toric IOL's astigmatic axis with the corneal meridian. Studies have shown that a 1° error can reduce the amount of astigmatism corrected by 3.3%. Accurate positioning of the corneal center is a crucial prerequisite for achieving accurate alignment of the toric IOL's astigmatic axis. Computer-assisted surgical navigation systems, utilizing image processing methods, can help surgeons accurately assess corneal displacement and rotation during surgery. Real-time, accurate positioning of the corneal center is an indispensable and critical component of surgical navigation systems, as its accuracy directly impacts the alignment of the toric IOL's astigmatic axis with the corneal meridian. Furthermore, existing corneal center positioning methods in computer-assisted surgical navigation systems mostly rely on traditional image processing techniques, requiring complex, manually designed feature extractors to obtain the limbal boundary contour and determine its center position. These methods are slow and have poor generalization, leaving significant room for improvement.

[0004] Therefore, with the help of deep learning methods, a method and system for positioning the corneal center during cataract surgery is realized by learning massive medical data, which has very important clinical application value. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: how to accurately locate the corneal center during cataract surgery, meet the actual application requirements in terms of real-time performance, and provide corneal center positioning services for subsequent tasks of the computer-assisted surgical navigation system.

[0006] The technical solution adopted by the present invention is: a corneal center positioning system in cataract surgery based on deep learning, comprising

[0007] a data acquisition module for acquiring cataract surgery video data;

[0008] a preprocessing module for preprocessing the cataract surgery video data obtained by the data acquisition module to form a preprocessed image sequence;

[0009] The pre-processed image sequence is input into the corneal Gaussian heat map generation network module based on the deep learning model to obtain the corneal heat map and center position offset map;

[0010] A decoding module that decodes the corneal thermal map and the center position offset map to obtain the peak position thereof, and outputs the coordinates of the corneal center position;

[0011] A data storage module, used for storing the coordinates of the corneal center position for external program call;

[0012] Data display module, used to display the coordinates of the corneal center position.

[0013] The data acquisition module is a video camera. During cataract surgery, the video camera is used to capture the surgical area of the patient's eye and generate video data. The preprocessing module first samples the video data into a continuous original image sequence at a rate of 30 frames per second, and then performs pixel completion on each original image. That is, it finds the longest value a and the widest value b in the original image sequence, makes c equal to the maximum value of a and b, and establishes a c*c image template filled with 0 pixel values. Then, each original image is placed in the image template and placed in the middle position, ensuring that each original image is not outside the image template. The layers are then merged to form a c*c image. The image is then adjusted to a 256×256 resolution image, which is the preprocessed image. All preprocessed images constitute a preprocessed image sequence.

[0014] The corneal Gaussian heat map generation network module constructed based on the deep learning model refers to first generating corneal Gaussian heat map data with real labels as a training set and a test set, then constructing a deep learning model, inputting the training set into the constructed deep learning model for training, and then inputting the test set into the trained model to detect the accuracy of the corneal center positioning result. The training and testing are repeated until the accuracy of the corneal center positioning result reaches a set value, that is, the training is completed, and the obtained deep learning model is the corneal Gaussian heat map generation network module constructed based on the deep learning model. Corneal Gaussian heat map data with real labels, that is, corneal Gaussian heat map data obtained by using an adaptive Gaussian ellipse heat map method, with real corneal Gaussian heat map data being adaptively generated and adjusted according to different input images, and its preparation method includes the following steps: Step 11, preprocessing the past cataract surgery video data obtained by the data acquisition module to form a preprocessed image sequence, and having a professional ophthalmologist annotate each image to obtain the coordinates of the corneal center position; Step 12, selecting an image, and selecting a circular sub-region around the corneal center position, where the center of the sub-region is the corneal center point and the radius is the corneal radius;

[0015] Step 13: Based on the gradient direction and amplitude of each pixel in the sub-region, a gradient direction histogram is drawn in the sub-region to obtain the main gradient direction in the sub-region, that is, the direction with the largest amplitude change;

[0016] Step 14: The main direction of the gradient is used as the major axis direction of the Gaussian ellipse, and the direction perpendicular to the gradient direction is used as the minor axis direction of the Gaussian ellipse;

[0017] Step 15: Calculate the pixel difference information in the major axis direction and the minor axis direction of the Gaussian ellipse respectively;

[0018] Step 16: Calculate the standard deviation of the Gaussian ellipse in the major axis direction and the minor axis direction based on the pixel difference information in the major axis direction and the minor axis direction;

[0019] Step 17: Construct an adaptive Gaussian ellipse heat map of the image based on the standard deviation of the Gaussian ellipse in the major axis direction, the standard deviation of the Gaussian ellipse in the minor axis direction, and the main gradient direction;

[0020] Step 18, repeat steps 12 to 18 until all images in the preprocessed image sequence are processed, and all images are divided into training sets and test sets in a ratio of 4:1. The deep learning model is constructed using an improved high-resolution key point detection network HRNet model, which can mine the deep feature information in the high-resolution feature map and generate a heat map and a center position offset map with corneal center position information. The improved high-resolution key point detection network HRNet model includes a downsampling layer, a backbone network, a deconvolution layer, and two convolution branches; the downsampling layer uses a convolution operation to continuously downsample the input image to obtain a low-resolution feature map; the backbone network is a standard HRNet model, which extracts and fuses features from the low-resolution feature map to obtain a deep feature map; the deconvolution layer uses two transposed convolution operations to upsample the scale of the deep feature map to a scale consistent with the input image; the two convolution branches are two independent convolution operations, and the number of channels is reduced while keeping the feature map size unchanged during the convolution process; the two convolution branches are used to respectively realize the prediction of the corneal heat map and the prediction of the center position offset map. The method of training the improved HRNet model includes a training method and a loss function. The training method is optimized using the Adam optimizer, and the loss function is a weighted hybrid loss function including global MSE loss, local MSE loss, center coordinate loss, and coordinate offset loss.

[0021] The decoding process of the decoding module is to determine the peak position coordinates in the corneal thermal image, obtain the coordinate offsets of the peak position coordinates in the x-axis and y-axis directions in the center position offset map, and calculate the position coordinates of the corneal center in the original image, i.e., the corneal center position coordinates, based on the peak position coordinates, the coordinate offsets of the peak position coordinates in the x-axis and y-axis directions in the center position offset map, and the zoom factor of the original high-resolution image. The data acquisition module is a video recording device.

[0022] The preprocessing module is a GPU graphics processor and a computer. The corneal Gaussian thermal map generation network module and the decoding module are CPU central processing units and computer programs. The data storage module is a hard disk. The data display module is a display. The present invention also includes a high-speed communication line.

[0023] The beneficial technical effect of the present invention is that it uses a deep learning model to extract corneal center feature information and determine its position coordinates end-to-end, eliminating the reliance on traditional methods of extracting and fitting the corneal contour to determine the center position, resulting in higher computational efficiency and accuracy. Furthermore, the adaptive Gaussian ellipse heat map employed can adaptively adjust the true heat map labels based on different training images, allowing the deep learning model to fully exploit and learn the feature information related to the center, thereby further improving the accuracy of corneal center positioning during cataract surgery.

[0024] The present invention uses video data from cataract surgery to quickly and accurately locate the corneal center, reducing the impact of corneal center deviation on the patient's postoperative residual astigmatism, further improving the quality of surgery, and providing stable center position coordinates for alignment of the corneal meridian during surgery. It is an indispensable part of the cataract surgery auxiliary navigation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a diagram of the improved HRNet network structure according to an embodiment of the present invention;

[0026] Figure 2 It is a schematic diagram of a detailed module diagram of the present invention. DETAILED DESCRIPTION

[0027] The following is a specific and detailed description of the technical solution of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention, so that the technical features of the present invention can be more easily understood by those skilled in the art. It should be noted that the specific embodiments listed here are only exemplary descriptions of the present invention and are not intended to limit the scope of protection of the present invention.

[0028] A deep learning-based corneal center positioning system for cataract surgery, comprising:

[0029] a data acquisition module for acquiring cataract surgery video data;

[0030] a preprocessing module for preprocessing the cataract surgery video data obtained by the data acquisition module to form a preprocessed image sequence;

[0031] The pre-processed image sequence is input into the corneal Gaussian heat map generation network module based on the deep learning model to obtain the corneal heat map and center position offset map;

[0032] A decoding module that decodes the corneal thermal map and the center position offset map to obtain the peak position thereof, and outputs the coordinates of the corneal center position;

[0033] A data storage module, used for storing the coordinates of the corneal center position for external program call;

[0034] Data display module, used to display the coordinates of the corneal center position.

[0035] The data acquisition module is a video camera. During cataract surgery, the video camera is used to capture the surgical area of the patient's eye and generate video data. The preprocessing module first samples the video data into a continuous original image sequence at a rate of 30 frames per second, and then performs pixel completion on each original image. That is, it finds the longest value a and the widest value b in the original image sequence, makes c equal to the maximum value of a and b, and establishes a c*c image template filled with 0 pixel values. Then, each original image is placed in the image template and placed in the middle position, ensuring that each original image is not outside the image template. The layers are then merged to form a c*c image. The image is then adjusted to a 256×256 resolution image, which is the preprocessed image. All preprocessed images constitute a preprocessed image sequence.

[0036] The corneal Gaussian heat map generation network module constructed based on the deep learning model means that first, corneal Gaussian heat map data with real labels are generated as training sets and test sets, and then a deep learning model is constructed, the training set is input into the constructed deep learning model for training, and then the test set is input into the trained model to detect the accuracy of the corneal center positioning result, and the training and testing are repeated until the accuracy of the corneal center positioning result reaches the set value, that is, the training is completed, and the obtained deep learning model is the corneal Gaussian heat map generation network module constructed based on the deep learning model. The corneal Gaussian heat map data with real labels is the corneal Gaussian heat map data obtained by the adaptive Gaussian ellipse heat map method, and the real corneal Gaussian heat map data is adaptively generated and adjusted according to different input images. The preparation method includes the following steps.

[0037] Step 11: Preprocess the past cataract surgery video data obtained by the data acquisition module to form a preprocessed image sequence. Professional ophthalmologists mark each image and obtain the coordinates of the corneal center position in the image (x c ,y c ); In one embodiment, the left vertex of the image is used as the coordinate origin, the distance from the left vertex of the image to the right vertex of the image is the x-axis, and the distance from the left vertex of the image to the left bottom point of the image is the y-axis.

[0038] Step 12. Select a picture and place it at the center of the cornea (x c ,y c ) select a circular sub-region around the cornea, the center of the sub-region is the corneal center point, and the radius is the corneal radius R;

[0039] Step 13: Calculate the gradient direction and magnitude of each pixel in the circular sub-region:

[0040]

[0041] Here, I(x,y) represents the pixel value at the (x,y) coordinate in the image, m(x,y) represents the gradient magnitude at the corresponding pixel location (at the (x,y) coordinate), and θ(x,y) represents the gradient direction at the corresponding pixel location. A gradient direction histogram is then plotted within the subregion based on the gradient direction and magnitude to obtain the main gradient direction θ within the subregion. x represents the abscissa value, y represents the ordinate value, and x±1 represents the abscissa value plus or minus 1 based on x, while y±1 represents the ordinate value plus or minus 1 based on y.

[0042] Step 14: Use the main gradient direction θ as the major axis direction θ of the Gaussian ellipse a , the direction perpendicular to the gradient direction is taken as the short axis direction θ of the Gaussian ellipse b ;

[0043] Step 15: Calculate the pixel difference in the major axis and minor axis directions of the Gaussian ellipse according to formula (2). and

[0044] Gaussian ellipse with arbitrary θ x Axis pixel difference

[0045]

[0046] Where R represents the radius of the selected sub-region, Indicates that in θ x The pixel value at a distance i from the center position in the axial direction, f0 represents the center point pixel value.

[0047] Step 16: According to formula (3), the pixel difference between the major axis and the minor axis of the Gaussian ellipse is used and Calculate the standard deviation σ of the Gaussian function in the direction of the major axis of the ellipse a and the standard deviation σ in the minor axis direction b .

[0048]

[0049] in, σ0 represents the initial standard deviation of the two-dimensional Gaussian distribution, and its value is one-third of the cornea radius in the input image.

[0050] Step 17: According to the standard deviation σ of the long axis direction of the Gaussian ellipse of the image a , the standard deviation σ of the Gaussian ellipse in the direction of the minor axis of the image b , the main gradient direction θ, and construct an adaptive Gaussian ellipse heat map according to formula (4):

[0051]

[0052] Where G(x,y)∈[0,1] represents the Gaussian value at the (x,y) position in the adaptive Gaussian ellipse heat map.

[0053] Step 18: Repeat steps 12 to 18 until all images in the preprocessed image sequence are processed, and divide all images into a training set and a test set in a ratio of 4:1.

[0054] The deep learning model is constructed using an improved high-resolution key point detection network HRNet model, which can mine the deep feature information in the high-resolution feature map and generate a heat map and a center position offset map with corneal center position information. The improved high-resolution key point detection network HRNet model includes a downsampling layer, a backbone network, a deconvolution layer and two convolution branches; the downsampling layer uses a convolution operation to continuously downsample the input image to obtain a small-resolution feature map; the backbone network is a standard HRNet model, which extracts and fuses features from the small-resolution feature map to obtain a depth feature map; the deconvolution layer uses two transposed convolution operations to upsample the depth feature map to a scale consistent with the input image; the two convolution branches are two independent groups of convolution operations, and the number of channels is reduced while the size of the feature map remains unchanged during the convolution process; the two convolution branches are respectively used to realize the prediction of the corneal heat map and the prediction of the center position offset map. The method of training the improved HRNet model includes a training method and a loss function. The training method is optimized using the Adam optimizer, and the loss function is a weighted hybrid loss function including global MSE loss, local MSE loss, center coordinate loss, and coordinate offset loss. The decoding process of the decoding module is to determine the peak position coordinates in the corneal heat map, obtain the coordinate offset of the peak position coordinates in the x-axis and y-axis directions in the center position offset map, and calculate the position coordinates of the corneal center in the original image, i.e., the corneal center position coordinates, according to the peak position coordinates, the coordinate offset of the peak position coordinates in the x-axis and y-axis directions in the center position offset map, and the zoom factor of the original high-resolution image. The data acquisition module is a video recording device.

[0055] The preprocessing module is a GPU graphics processor and a computer. The corneal Gaussian thermal map generation network module and the decoding module are CPU central processing units and computer programs. The data storage module is a hard disk. The data display module is a display. The present invention also includes a high-speed communication line.

[0056] In some embodiments, converting the corneal center coordinate label into a true adaptive Gaussian heat map comprises the following steps:

[0057] In some embodiments, the true coordinate offset of the corneal center is:

[0058]

[0059] Among them (O x ,O y ) represents the real coordinate offset of the corneal center in the x and y directions, (I x ,I y ) represents the coordinate value of the corneal center in the original image, and λ represents the downsampling ratio, that is, the ratio of the resolution of the original image to that of the input image.

[0060] In some embodiments, the corneal Gaussian heat map generation network model based on deep learning adopts an improved HRNet network. Figure 1 The improved HRNet network structure diagram is shown, including a downsampling layer, a backbone network, a deconvolution layer and two convolution branches; the downsampling layer uses a convolution operation to continuously downsample the input image to obtain a feature map of small resolution; the backbone network is a standard HRNet model, which extracts and fuses features of the feature map of small resolution to obtain a depth feature map; the deconvolution layer uses two transposed convolution operations to upsample the scale of the depth feature map to a scale consistent with the input image; the two convolution branches are two independent sets of convolution operations, and the number of channels is reduced while keeping the size of the feature map unchanged during the convolution process; the two convolution branches are used to respectively realize the prediction of the corneal heat map and the prediction of the center position offset map.

[0061] In some embodiments, the hybrid loss function is a weighted combination of four loss functions, namely:

[0062] Global MSE loss function:

[0063]

[0064] Where W and H are the width and height of the heat map image, and P(i, j) and T(i, j)∈[0,1] are the values of the (i, j) position in the predicted heat map P and the true heat map T, respectively.

[0065] Local MSE loss function:

[0066]

[0067] Where D is the set of all pixels in the Gaussian ellipse distribution area of the real heat map T, and the symbol table is ⊙

[0068] Indicates element-wise multiplication.

[0069] Center coordinate loss function:

[0070]

[0071] In the formula, (Cx ,C y )and Represent the true center coordinate position and the predicted center coordinate position respectively.

[0072] Coordinate offset loss function:

[0073]

[0074] Where, (O x ,O y )and They represent the actual corneal center position offset and the predicted corneal center position offset, respectively.

[0075] Furthermore, the corneal Gaussian heat map generation network generates the predicted corneal heat map sequence and the center position offset map sequence in chronological order. In step S4, the position coordinates of the corneal heat map sequence are decoded in sequence: by performing a global search on the predicted corneal heat map, the peak position coordinates in the corneal heat map are obtained, which are recorded as

[0076] According to the peak position coordinates Get the predicted offset of the coordinates in the x and y directions at the corresponding position in the center position offset map According to the peak position coordinates offset And the downsampling ratio λ is used to calculate the position coordinates of the corneal center in the original image The position coordinates of the corneal center in the original image The calculation formula is:

[0077]

[0078] In some embodiments, if the original image is filled, the predicted corneal center coordinate position The padding value in the corresponding direction should be subtracted.

[0079] As a second embodiment of the present invention, a deep learning-based corneal center positioning system for cataract surgery is provided, including a server terminal device, wherein the server terminal device includes: a GPU graphics processor, a CPU central processing unit, a high-speed communication line, a data storage device and a computer program, and the computer program is executed to start the corneal center positioning system. The corneal center positioning system uses a deep neural network to perform real-time corneal center positioning tasks on videos during cataract surgery, and saves the corneal center position in the data storage device. Figure 2 The system detailed module diagram is shown. The system includes the following modules:

[0080] A data acquisition module is used to acquire video stream data of the surgical area of the patient's eye during cataract surgery;

[0081] A data preprocessing module, configured to convert the video stream data into continuous frames and adjust the frames into a sequence of images of a specific resolution;

[0082] A corneal Gaussian heat map generation module is configured to construct a corneal Gaussian heat map generation network based on a deep learning model, input the image sequence into the corneal Gaussian heat map generation network, and obtain a predicted corneal heat map sequence and a center position offset map sequence;

[0083] A position coordinate decoding module, used for decoding the corneal thermal image sequence to obtain the position coordinates of the corneal center in the original image;

[0084] The data storage module is used to store the cornea center position coordinate information in a data storage device.

[0085] Preferably, the data acquisition module 501 is used to acquire real-time video stream data of the patient's eye area during cataract surgery. The data acquisition module collects video output signals from professional surgical video recording equipment with low latency and high quality through high-speed wired communication.

[0086] Preferably, the data preprocessing module 502 is configured to process the video output signal by converting the surgical video data into continuous frames, sampling the frames at a rate of 30 frames per second into a continuous raw image sequence, and padding and resizing the raw image sequence to a 256×256 resolution as an input image sequence for the network model.

[0087] In some implementations, the image is filled by evenly filling the outer areas on both sides of the short side of the original image with 0 pixel values, so that the height and width of the image are equal to maintain the consistency of the image structure.

[0088] Preferably, the corneal heat map generation module 503 calculates the input image sequence in real time through a corneal Gaussian heat map generation network model based on deep learning deployed on a high-performance GPU server, and outputs a predicted corneal heat map sequence and a center position offset map sequence in real time.

[0089] In some embodiments, the corneal heat map generation network model is an improved HRNet network model. After the network model is constructed, the training set is input into the network for training, and then the test set is input into the trained model to verify the accuracy of the positioning result. If the positioning accuracy requirement is met, the network model is deployed on the server terminal. If the positioning accuracy requirement is not met, the network will continue to be trained.

[0090] In some embodiments, the training set is constructed by converting the corneal center coordinate label into a true adaptive Gaussian heat map form as the true training set label data;

[0091] Preferably, the coordinate position decoding module 504 is used to perform position coordinate decoding on the predicted corneal heat map sequence, and obtain the coordinate information of the corneal center position in the original image in combination with the center position offset map sequence.

[0092] In some embodiments, if the original image is filled, the predicted corneal center coordinate position The padding value in the corresponding direction should be subtracted.

[0093] Preferably, the data storage module 505 is used to store the real-time corneal center position coordinate information in a data storage device for use by external programs.

[0094] In some embodiments, modules 501 to 505 are executed in real-time sequential order, that is, the image sequence passes through modules 501, 502, 503, 504 and 505 in the form of data stream until the operation is completed (the last image in the image sequence is processed) and the system stops.

[0095] From the above, the present application discloses a method and system for locating the corneal center during cataract surgery based on deep learning. First, video data of the patient's eye area during cataract surgery is acquired in real time and preprocessed to obtain an input image sequence. The input image sequence is sequentially passed through a corneal Gaussian heat map generation network based on a deep learning model in the form of a data stream to obtain a predicted corneal heat map sequence and a center position offset map sequence. The corneal heat map sequence and the center position offset map sequence are decoded by a position coordinate decoding module to obtain the position coordinates of the corneal center in the real-time original image and store them. This method is conducive to achieving fast and accurate corneal center positioning during surgery and can provide a precise corneal center position for alignment of the Toric IOL astigmatism axis with the corneal meridian.

[0096] The above embodiments are merely exemplary embodiments of the present invention. Within the scope of understanding of those skilled in the art, various changes and modifications within the spirit and principles of the present invention are within the protection scope of the present invention.

Claims

1. A deep learning-based corneal center positioning system for cataract surgery, characterized by: including a data acquisition module for acquiring cataract surgery video data; a preprocessing module for preprocessing the cataract surgery video data obtained by the data acquisition module to form a preprocessed image sequence; A corneal Gaussian heat map generation network module is constructed based on a deep learning model, and the preprocessed image sequence is input into the corneal Gaussian heat map generation network module to obtain a corneal heat map and a center position offset map; the corneal Gaussian heat map generation network module constructed based on a deep learning model refers to first generating corneal Gaussian heat map data with real labels as a training set and a test set, and then constructing a deep learning model, inputting the training set into the constructed deep learning model for training, and then inputting the test set into the trained model to detect the accuracy of the corneal center positioning result, repeating the training and testing until the accuracy of the corneal center positioning result reaches the set value, that is, the training is completed, and the obtained deep learning model is the corneal Gaussian heat map generation network module constructed based on the deep learning model; the corneal Gaussian heat map data with real labels is the corneal Gaussian heat map data obtained by adopting the adaptive Gaussian ellipse heat map method, and the real corneal Gaussian heat map data is adaptively generated and adjusted according to different input images, which specifically includes the following steps. Step 11: Preprocess the past cataract surgery video data obtained by the data acquisition module to form a preprocessed image sequence. A professional ophthalmologist annotates each image to obtain the coordinates of the corneal center. Step 12: Select an image and select a circular sub-region around the corneal center, with the center of the sub-region being the corneal center point and the radius being the corneal radius. Step 13: Based on the gradient direction and amplitude of each pixel in the sub-region, a gradient direction histogram is drawn in the sub-region to obtain the main gradient direction in the sub-region, that is, the direction with the largest amplitude change; Step 14: The main direction of the gradient is used as the major axis direction of the Gaussian ellipse, and the direction perpendicular to the gradient direction is used as the minor axis direction of the Gaussian ellipse; Step 15: Calculate the pixel difference information in the major axis direction and the minor axis direction of the Gaussian ellipse respectively; Step 16: Calculate the standard deviation of the Gaussian ellipse in the major axis direction and the minor axis direction based on the pixel difference information in the major axis direction and the minor axis direction; Step 17: Construct an adaptive Gaussian ellipse heat map of the image based on the standard deviation of the Gaussian ellipse in the major axis direction, the standard deviation of the Gaussian ellipse in the minor axis direction, and the main gradient direction; Step 18: Repeat steps 12 to 18 until all images in the preprocessed image sequence are processed, and divide all images into a training set and a test set in a ratio of 4:1; A decoding module that decodes the corneal thermal map and the center position offset map to obtain the peak position thereof, and outputs the coordinates of the corneal center position; A data storage module, used for storing the coordinates of the corneal center position for external program call; Data display module, used to display the coordinates of the corneal center position.

2. The deep learning-based corneal center positioning system for cataract surgery according to claim 1, characterized in that: The data acquisition module is a video camera. During cataract surgery, the video camera is used to capture the surgical area of the patient's eye to generate video data.

3. The deep learning-based corneal center positioning system for cataract surgery according to claim 1, characterized in that: The preprocessing module first samples the video data into a continuous original image sequence at a rate of 30 frames per second, and then performs pixel completion on each original image, that is, finding the longest value a and the widest value b in the original image sequence, making c equal to the maximum value of a and b, and establishing a c*c image template filled with 0 pixel values. Then, each original image is placed in the image template and in the middle position to ensure that each original image is not outside the image template. Then, the layers are merged to form a c*c image, and then the image is adjusted to a 256×256 resolution image, which is the preprocessed image. All preprocessed images constitute a preprocessed image sequence.

4. The deep learning-based corneal center positioning system for cataract surgery according to claim 1, characterized in that: The improved high-resolution key point detection network HRNet model is used to construct a deep learning model, which can mine the deep feature information in the high-resolution feature map and generate a heat map and a center position offset map with corneal center position information. The improved high-resolution key point detection network HRNet model includes a downsampling layer, a backbone network, a deconvolution layer and two convolution branches; the downsampling layer uses a convolution operation to continuously downsample the input image to obtain a small-resolution feature map; the backbone network is a standard HRNet model, which extracts and fuses the small-resolution feature map to obtain a deep feature map; the deconvolution layer is Two transposed convolution operations are used to upsample the scale of the depth feature map to a scale consistent with the input image; the two convolution branches are two independent convolution operations, and the number of channels is reduced while keeping the size of the feature map unchanged during the convolution process; the two convolution branches are respectively used to realize the prediction of the corneal heat map and the center position offset map, and the method of training the improved HRNet model includes a training method and a loss function. The training method is optimized using the Adam optimizer, and the loss function is a weighted hybrid loss function including global MSE loss, local MSE loss, center coordinate loss, and coordinate offset loss.

5. The deep learning-based corneal center positioning system for cataract surgery according to claim 1, characterized in that: The decoding process of the decoding module is to determine the peak position coordinates in the corneal heat map, obtain the coordinate offset of the peak position coordinates in the axis-to-axis direction in the center position offset map, and calculate the position coordinates of the corneal center in the original image, i.e., the corneal center position coordinates, based on the peak position coordinates, the coordinate offset of the peak position coordinates in the axis-to-axis direction in the center position offset map, and the zoom factor of the original high-resolution image.

Images