Retinal image registration method and system based on grid supervision and key point learning

Abstract

This invention proposes a retinal image registration method and system based on grid supervision and keypoint learning, belonging to the field of medical image processing technology. The method includes: acquiring RGB fundus image pairs; encoding the RGB fundus image pairs through a shared feature extraction branch to obtain downsampled feature maps; processing the downsampled feature maps through a keypoint extraction branch and a descriptor decoder branch respectively to extract keypoint probability maps and feature descriptors; predicting the offset of keypoints within a local window based on paired feature descriptors using a keypoint offset prediction module; calibrating the positions of keypoints in the keypoint probability map based on the offsets; and outputting the registered image. This invention utilizes densely gridded annotation points for self-supervised training, eliminating the need for manual annotation; through lightweight network design and a keypoint offset calibration mechanism, it maintains high registration accuracy while significantly reducing the number of parameters, making it suitable for real-time clinical deployment and retinal disease analysis.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and in particular relates to a retinal image registration method and system based on grid supervision and key point learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Retinal image registration is an important research direction in the field of medical image analysis. Its core objective is to spatially align retinal images acquired at different times, from different viewpoints, or using different devices to support clinical applications such as quantitative assessment of retinal diseases, monitoring of disease progression, surgical navigation, and multimodal image fusion. Traditional retinal image registration methods are usually based on feature point matching, such as scale-invariant feature transforms or their improvements. These methods can, to some extent, handle rigid transformations such as rotation and scaling between images. However, retinal images often suffer from non-rigid deformation, local structural changes, uneven illumination, and interference from pathological areas, posing challenges to the registration accuracy and robustness of traditional methods.

[0004] In recent years, with the development of deep learning technology, learning-based retinal image registration methods have gradually become mainstream. These methods typically extract image features using convolutional neural networks and train the model with a large amount of labeled data to achieve end-to-end feature matching and transformation estimation. However, when directly applied to retinal images, their performance is often limited due to the structural specificity of retinal images and the scarcity of labeled data. Furthermore, existing deep learning-based methods usually require supervised training with a large amount of manually labeled or automatically generated pseudo-labels. This not only increases the cost and complexity of data preparation but may also introduce labeling noise, affecting the model's generalization ability.

[0005] In terms of model structure, existing retinal registration networks are often complex in design and have a large number of parameters, requiring significant computational resources and GPU memory during inference, making them difficult to deploy in real-time in resource-constrained clinical environments. Furthermore, existing methods often employ a two-stage registration strategy, first performing feature matching and then iteratively optimizing and estimating transformation parameters, resulting in low computational efficiency and failing to meet the real-time requirements of clinical practice. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a retinal image registration method and system based on grid supervision and key point learning, aiming to solve the technical problems of existing retinal image registration methods, such as reliance on pre-labeled data, large parameter scale, high computational cost, and difficulty in balancing registration accuracy and efficiency.

[0007] II. Technical Solution.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] The first aspect of this invention provides a retinal image registration method based on grid supervision and key point learning;

[0010] Retinal image registration methods based on grid supervision and keypoint learning include:

[0011] Obtain RGB fundus image pairs;

[0012] The RGB fundus image pairs are encoded by a shared feature extraction branch to obtain downsampled feature maps;

[0013] The downsampled feature map is processed through a keypoint extraction branch and a descriptor decoder branch respectively to extract a keypoint probability map and a feature descriptor.

[0014] The keypoint offset prediction module predicts the offset of keypoints within a local window based on paired feature descriptors.

[0015] Based on the offset, the key points in the key point probability map are calibrated to produce a registered image.

[0016] As a further technical solution, the RGB fundus image pairs are encoded by a shared feature extraction branch to obtain downsampled feature maps, including:

[0017] The shared feature extraction branch performs multi-stage downsampling on RGB fundus image pairs through convolution, batch normalization, and linear rectified activation functions. The convolution operation downsamples the image resolution from the original H×W scale to H / 8. The W / 8 scale is used to output a downsampled feature map.

[0018] As a further technical solution, the downsampled feature map is processed through a key point extraction branch to extract a key point probability map, including:

[0019] Perform a one-dimensional convolution operation on the downsampled feature map to convert the downsampled feature map into a first channel feature map;

[0020] By classifying key points at each location in the first channel feature map, a key point image is obtained;

[0021] The keypoint image is expanded to obtain a keypoint probability map.

[0022] As a further technical solution, the downsampled feature map is processed through a descriptor decoder branch to extract feature descriptors, including:

[0023] The downsampled feature maps are convolved sequentially to scales of 1 / 16 and 1 / 32 to expand the receptive field;

[0024] Then, by bilinear sampling, the tensors at scales of 1 / 8, 1 / 16, and 1 / 32 are upsampled to restore them to H / 8. W / 8 scale;

[0025] The tensors upsampled at the three scales are summed element-wise, and the resulting convolutions are used to output feature descriptors.

[0026] As a further technical solution, a keypoint offset prediction module predicts the offset of keypoints within a local window based on paired feature descriptors, including:

[0027] Obtain the corresponding feature vector on the 1 / 8 resolution feature map;

[0028] The position of the corresponding feature vector is determined by downsampling and rounding the original H×W resolution label;

[0029] The fused features are obtained by fusing the corresponding feature vectors;

[0030] The fused features are input into a multilayer perceptron, and a classification task is used to predict the offset between the feature point coordinates and the original labeled points when the coordinates are magnified back to the original scale within a local window.

[0031] As a further technical solution, during the training phase, dense gridded annotation points are used to extract the corresponding feature vectors from the feature descriptors, and a feature similarity matrix is ​​calculated for self-supervised training.

[0032] As a further technical solution, the method is optimized during training using a total loss function, which includes keypoint loss, descriptor loss, and keypoint offset prediction loss.

[0033] As a further technical solution, the key point loss adopts negative log-likelihood loss, which is used to supervise the training of the key point extraction branch;

[0034] The descriptor loss is calculated by maximizing the probability of positive sample pairs on the diagonal of the similarity matrix;

[0035] The keypoint offset prediction loss is calculated by minimizing the classification error between the predicted offset and the actual offset.

[0036] A retinal image registration system based on grid supervision and keypoint learning includes:

[0037] The image acquisition module is configured to acquire RGB fundus image pairs;

[0038] The shared feature extraction module is configured to: encode the RGB fundus image pairs through a shared feature extraction network to obtain downsampled feature maps;

[0039] The key point extraction and descriptor decoding module is configured to process the downsampled feature map through the key point extraction branch and the descriptor decoder branch respectively, and extract the key point probability map and feature descriptor.

[0040] The offset prediction module is configured to predict the offset of keypoints within a local window based on paired feature descriptors, using the keypoint offset prediction module.

[0041] The registration module is configured to: perform position calibration on key points in the key point probability map based on the offset, and output the registered image.

[0042] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the retinal image registration method based on mesh supervision and keypoint learning as described in the first aspect of the present invention.

[0043] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the retinal image registration method based on grid supervision and key point learning as described in the first aspect of the present invention.

[0044] The above one or more technical solutions have the following beneficial effects:

[0045] (1) This invention generates a large number of precisely aligned sample pairs using densely gridded annotation points during the training phase. Training of the descriptor decoder can be completed simply by calculating the feature similarity matrix, without requiring any manual annotation or pre-annotated data. This effectively solves the problems of strong dependence on annotation data and high annotation costs in existing deep learning methods, improving the practicality and scalability of the method. By designing a keypoint offset classification task based on descriptor pairs, the network can predict the geometric deviation of feature points during downsampling, thereby calibrating the keypoint positions during the inference phase. This mechanism effectively alleviates the keypoint localization error caused by the reduced feature map resolution and improves the spatial accuracy of registration.

[0046] (2) This invention employs a lightweight keypoint extraction branch and a simplified U-Net structure as the descriptor decoder, significantly reducing the number of model parameters while maintaining feature representation capabilities. It directly estimates the transformation relationships between images through second-order polynomial fitting, avoiding the computational overhead of iterative optimization in traditional two-stage methods, and achieving end-to-end single-stage registration. This method significantly improves computational efficiency while ensuring registration accuracy, making it more suitable for real-time or near-real-time clinical applications.

[0047] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0048] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0049] Figure 1 This is a flowchart of the method in the first embodiment.

[0050] Figure 2 This is a schematic diagram of the keypoint offset prediction module in the first embodiment.

[0051] Figure 3 This is a system structure diagram of the second embodiment. Detailed Implementation

[0052] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0053] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0054] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0055] Example 1

[0056] This embodiment discloses a retinal image registration method based on grid supervision and key point learning;

[0057] like Figure 1 As shown, the retinal image registration method based on mesh supervision and keypoint learning includes:

[0058] Step S1: Obtain RGB fundus image pairs;

[0059] Step S2: Encode the RGB fundus image pairs using a shared feature extraction network to obtain downsampled feature maps;

[0060] Step S3: Process the downsampled feature map through the key point extraction branch and the descriptor decoder branch respectively to extract the key point probability map and feature descriptor;

[0061] Step S4: The keypoint offset prediction module predicts the offset of keypoints within the local window based on paired feature descriptors.

[0062] Step S5: Based on the offset, perform position calibration on the key points in the key point probability map and output the registered image.

[0063] Specifically, it also includes the following:

[0064] Step S1: Obtain RGB fundus image pairs.

[0065] The patient's retina was photographed using a fundus camera to obtain initial RGB fundus images. These images were then clinically screened to exclude substandard samples with severe motion blur, overexposure or underexposure, artifact interference, or other defects, ensuring the validity of the training data.

[0066] The acquired and preprocessed raw RGB fundus images were directly used as the reference images in the paired images. This preserves its original shape and structural information. Simultaneously, a reference image is used. Based on this, paired deformable images are generated through random homography variations, random brightness, and random shadows. .

[0067] In the Perform the above random deformation generation At the same time, The same homography transformation is performed on the pre-sampled densely gridded annotation points (scale consistent with the image resolution, i.e., H×W annotation points, uniformly distributed across the entire image) to obtain... Zhongyu The corresponding gridded annotation points form precisely aligned annotation point pairs, providing supervision signals for subsequent self-supervised training.

[0068] Step S2: Encode the RGB fundus image pairs using a shared feature extraction network to obtain downsampled feature maps.

[0069] The shared feature extraction branch consists of four convolutional units connected in series. Each convolutional unit contains a convolutional layer, a batch normalization layer, and a Rectified Linear Activation Function (ReLU). By performing multi-stage downsampling on the RGB fundus images, the image resolution is downsampled from the original H×W scale to H / 8 through convolution operations. The W / 8 scale is used to output a downsampled feature map.

[0070] The acquired RGB fundus images are input into the first convolutional unit. This convolutional unit performs preliminary feature extraction on the input image through convolution operations. While maintaining the spatial resolution, the number of channels is increased from 1 to 4, and the output feature map has a size of H×W×4.

[0071] The H×W×4 feature map output from the first convolutional layer is input into the second convolutional unit. This layer performs the first downsampling through stride convolution, reducing the spatial resolution of the feature map to H / 2×W / 2, while increasing the number of channels to 8, resulting in a feature representation of size H / 2×W / 2×8.

[0072] Next, the H / 2×W / 2×8 feature map output from the second convolutional layer is input into the third convolutional unit. This convolutional unit further encodes and compresses the features, reduces the spatial resolution to H / 4×W / 4 through downsampling, and increases the number of channels to 24, outputting a mid-layer feature map with a size of H / 4×W / 4×24.

[0073] Finally, the output of the third convolutional unit is fed into the fourth convolutional unit. This layer reduces the feature map resolution to H / 8×W / 8 through a third downsampling operation and further increases the number of channels to 64, ultimately outputting a high-dimensional shared feature map with a size of H / 8×W / 8×64.

[0074] Step S3: Process the downsampled feature map through the key point extraction branch and the descriptor decoder branch respectively to extract the key point probability map and feature descriptor.

[0075] Step S31: Perform a 1×1 one-dimensional convolution operation on the H / 8×W / 8×64 downsampled feature map output by the shared feature extraction branch to convert the 64-channel feature map into a 65-channel feature map, i.e., the first channel feature map. Channels 0-63 correspond to the keypoint coordinates within each 8×8 pixel region on the original H×W scale (a total of 8×8=64 possible positions), and channel 64 indicates that there are no keypoints within that 8×8 pixel region.

[0076] For each location (H / 8×W / 8 locations) in the 65-channel feature map, a keypoint classification task is performed to determine whether there are specific coordinates of keypoints within the corresponding 8×8 pixel region or no keypoints. During the classification process, this branch is trained using knowledge distillation, with SuperRetina, which has excellent performance in retinal image keypoint extraction tasks, as the teacher model. The keypoint coordinates output by SuperRetina are mapped onto an H / 8×W / 8×65 tensor as labels (unmapped regions are labeled with 64), guiding the keypoint extraction branch to learn retinal structural features.

[0077] The classified H / 8×W / 8×65 keypoint image is unfolded along the channel dimension and converted into an H×W×1 keypoint probability map through a reshape operation. Each pixel value in the probability map represents the confidence that the location is a keypoint.

[0078] Step S32: Input the downsampled feature map of H / 8×W / 8×64 into the simplified U-Net structure, and first downsample it to 1 / 16 and 1 / 32 scales sequentially through two convolutional units:

[0079] 1 / 16 scale conversion: 3×3 convolution kernel, 2 stride, 64 output channels, feature map scale is H / 16×W / 16×64;

[0080] 1 / 32 scale transformation: convolution kernel 3×3, stride 2, number of output channels 64, feature map scale is H / 32×W / 32×64.

[0081] Subsequently, bilinear upsampling operations were performed on the feature tensors at scales of 1 / 8, 1 / 16, and 1 / 32, respectively, to restore them to the H / 8×W / 8 scale, ensuring that the feature map size is consistent across scales, which facilitates subsequent fusion.

[0082] The feature tensors upsampled at three scales are summed element-wise to integrate multi-scale feature information. Then, a fusion convolution is performed through a 1×1 convolution layer, keeping the number of channels at 64. The final output is a feature descriptor F with size H / 8×W / 8×64, which can accurately represent the local structural features of the retina.

[0083] Step S4: The keypoint offset prediction module predicts the offset of keypoints within the local window based on paired feature descriptors.

[0084] During the training phase, the original H×W resolution dense gridded annotation points generated in step S1 are obtained, and the coordinates of the original H×W resolution annotation points are downsampled to obtain their corresponding positions on the 1 / 8 resolution feature map.

[0085] From the baseline image Corresponding feature descriptor and deformed images Corresponding feature descriptor In the process, feature vectors at corresponding spatial locations are extracted. and The corresponding position is determined by downsampling and rounding the original H×W resolution gridded annotation points, ensuring... and They correspond to the same retinal structure location.

[0086] A lightweight fusion strategy using element-wise addition is employed to combine paired feature vectors. and Element-wise addition is performed to generate a 128-dimensional fused feature vector, and a classification task is performed to predict the offsets Δx and Δy of the feature point coordinates on F relative to the original labeled points when magnified back to the original scale. Specifically, this fused feature vector is input into a multilayer perceptron (MLP) to predict the spatial offsets of the feature points relative to the original labeled points when magnified back to the original resolution.

[0087] Because the original high-resolution annotations involve rounding operations during downsampling to the H / 8×W / 8 feature map, the introduced geometric bias is confined to an 8×8 pixel local window centered on the feature point. Therefore, this spatial offset is modeled as a finite discrete set of offset candidates, where each candidate corresponds to a specific offset combination (Δx, Δy), for a total of 8×8=64 possibilities.

[0088] The output layer of the multilayer perceptron is constructed with 64 output nodes, each corresponding to a predefined offset candidate. By classifying and predicting this output, the offset category with the highest probability is determined, and this category is mapped to the corresponding (Δx, Δy) offset, which serves as the coordinate correction result of the feature point at the original scale.

[0089] During the training phase, the multilayer perceptron is trained under the supervision of a loss function, wherein the supervision signal is generated by the true offset between the original labeled points and the magnified feature point coordinates, and converted into the corresponding discrete offset category label; during the inference phase, the offset category with the highest predicted probability is directly output to obtain the offset.

[0090] Due to the original high-resolution annotation being downsampled to H / 8 The rounding operation in the W / 8 process has its geometric deviation strictly limited to one 8. Within a local window of 8. Therefore, training for this classification task can be supervised by calculating the true offset between the original scale labels and the downsampled, rounded coordinates.

[0091] Step S5: Based on the offset, perform position calibration on the key points in the key point probability map and output the registered image.

[0092] The initial coordinates of each key point in the key point probability map are obtained, and the initial key point coordinates are calibrated by combining the Δx and Δy output by the key point offset prediction module.

[0093] Based on the calibrated keypoint correspondence, the value is directly estimated using second-order polynomial fitting. and The view transformations between these views address the inherent 3D-to-2D mapping distortion problem of the retinal surface. As shown below:

[0094]

[0095] in, The transformed coordinates are output, corresponding to the keypoint locations matched in another image; , These are the initial coordinates of the predicted key points output by the key point probability map; These are the coordinates of the key points after final calibration.

[0096] Furthermore, during the training phase, the densely gridded annotation points generated in step S1 are used to analyze the feature descriptors. and Feature vectors at corresponding spatial locations are selected, and feature similarity matrices and symmetric similarity matrices are calculated. By maximizing the similarity of the diagonal elements of the similarity matrix, the descriptor decoder branch is guided to learn discriminative local features, achieving self-supervised training without manual annotation.

[0097] Furthermore, network training employs a joint total loss function for optimization, which is defined as follows:

[0098]

[0099] in It is the total loss. It's a critical loss. It is descriptor loss. It is the keypoint offset prediction loss. , and It is a hyperparameter that balances the three losses.

[0100] For the keypoint loss function, during keypoint detection training, the keypoint coordinates (x, y) output by the SuperRetina of the distilled object are mapped to H / 8. W / 8 The keypoints are labeled on a 65-dimensional tensor. Regions without mapping have a truth value of 64. The label is denoted as... H / 8 W / 8 Each position on the 65 sheets is recorded as Therefore, only a simple negative log-likelihood loss (NLL loss) is needed to guide the training of the feature point decoder:

[0101]

[0102] in, For tensor position; For tags.

[0103] For the descriptor loss function, the corresponding sub-descriptor set is sampled from the feature map F output by the descriptor decoder branch. , Calculate the similarity matrix. To improve training efficiency and accuracy, the symmetric matching property is used to calculate... By relying on a large number of dense labels, it is only necessary to let the corresponding vector, i.e. , Diagonal, maximizing similarity:

[0104]

[0105] in, , It is a diagonal line.

[0106] For the keypoint offset prediction loss function, the keypoint offset prediction module is supervised in the following way: using the original H... The offset between the pixel coordinates in the IB image at resolution W and their corresponding coordinates after downsampling and then upsampling is used as the supervision signal. For each pair of corresponding feature vectors and In an 8 8. Predicted offset within a local window. Actual offset. The features, obtained from the supervisory signal and fused, are passed through a multilayer perceptron to predict a discrete 8-bit signal. 8 probability diagrams Each element corresponds to a candidate offset. Here, Indicates the first Regarding features, This represents the actual offset. The keypoint offset prediction loss is defined as:

[0107]

[0108] in, Loss prediction for keypoint offset; This is a probability graph.

[0109] Example 2

[0110] This embodiment discloses a retinal image registration system based on grid supervision and key point learning;

[0111] like Figure 3 As shown, the retinal image registration system based on grid supervision and keypoint learning includes:

[0112] The image acquisition module is configured to acquire RGB fundus image pairs;

[0113] The shared feature extraction module is configured to: encode the RGB fundus image pairs through a shared feature extraction network to obtain downsampled feature maps;

[0114] The key point extraction and descriptor decoding module is configured to process the downsampled feature map through the key point extraction branch and the descriptor decoder branch respectively, and extract the key point probability map and feature descriptor.

[0115] The offset prediction module is configured to predict the offset of keypoints within a local window based on paired feature descriptors, using the keypoint offset prediction module.

[0116] The registration module is configured to: perform position calibration on key points in the key point probability map based on the offset, and output the registered image.

[0117] Example 3

[0118] The purpose of this embodiment is to provide a computer-readable storage medium.

[0119] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the retinal image registration method based on grid supervision and keypoint learning as described in Example 1.

[0120] Example 4

[0121] The purpose of this embodiment is to provide an electronic device.

[0122] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the retinal image registration method based on grid supervision and key point learning as described in Embodiment 1.

[0123] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0124] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0125] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A retinal image registration method based on grid supervision and key point learning, characterized in that, include: Obtain RGB fundus image pairs; The RGB fundus image pairs are encoded by a shared feature extraction branch to obtain downsampled feature maps; The downsampled feature map is processed through a keypoint extraction branch and a descriptor decoder branch respectively to extract a keypoint probability map and a feature descriptor. During the training phase, the corresponding feature vectors are extracted from the feature descriptors using densely gridded annotation points, and the feature similarity matrix is ​​calculated for self-supervised training. The keypoint offset prediction module predicts the offset of keypoints within a local window based on paired feature descriptors, including: Obtain the corresponding feature vector on the 1 / 8 resolution feature map; The position of the corresponding feature vector is determined by downsampling and rounding the original H×W resolution label; The fused features are obtained by fusing the corresponding feature vectors; The fused features are input into a multilayer perceptron, and a classification task is used to predict the offset between the feature point coordinates and the original labeled points when the feature point coordinates are magnified back to the original scale within a local window; Based on the offset, the key points in the key point probability map are positionally calibrated, and the registered image is output.

2. The retinal image registration method based on mesh supervision and keypoint learning of claim 1, wherein, The RGB fundus image pairs are encoded using a shared feature extraction branch to obtain downsampled feature maps, including: The shared feature extraction branch performs multi-stage downsampling processing on RGB fundus image pairs through convolution, batch normalization, and linear rectified activation functions. The image resolution is downsampled from the original H×W scale to the H / 8*W / 8 scale through convolution operations, and finally outputs the downsampled feature map.

3. The retinal image registration method based on mesh supervision and keypoint learning of claim 1, wherein, The downsampled feature map is processed through a keypoint extraction branch to extract a keypoint probability map, including: Perform a one-dimensional convolution operation on the downsampled feature map to convert the downsampled feature map into a first channel feature map; By classifying key points at each location in the feature map of the first channel, a key point image is obtained; The keypoint image is expanded to obtain a keypoint probability map.

4. The retinal image registration method based on mesh supervision and keypoint learning of claim 1, wherein, The downsampled feature map is processed through the descriptor decoder branch to extract feature descriptors, including: The downsampled feature maps are convolved sequentially to scales of 1 / 16 and 1 / 32 to expand the receptive field; Then, by bilinear sampling, the tensors at scales of 1 / 8, 1 / 16, and 1 / 32 are upsampled and restored to the H / 8*W / 8 scale. The tensors upsampled at three scales are summed element-wise, and the resulting convolutions are used to output feature descriptors.

5. The retinal image registration method based on mesh supervision and keypoint learning of claim 1, wherein, It also includes optimization using a total loss function, which includes keypoint loss, descriptor loss, and keypoint offset prediction loss; The keypoint loss uses negative log-likelihood loss to supervise the training of the keypoint extraction branch; The descriptor loss is calculated by maximizing the probability of positive sample pairs on the diagonal of the similarity matrix; The keypoint offset prediction loss is calculated by minimizing the classification error between the predicted offset and the actual offset.

6. A retinal image registration system based on grid supervision and keypoint learning, characterized in that, The retinal image registration method based on mesh supervision and keypoint learning as described in any one of claims 1-5 includes: The image acquisition module is configured to acquire RGB fundus image pairs; The shared feature extraction module is configured to: encode the RGB fundus image pairs through a shared feature extraction network to obtain downsampled feature maps; The key point extraction and descriptor decoding module is configured to process the downsampled feature map through the key point extraction branch and the descriptor decoder branch respectively, and extract the key point probability map and feature descriptor. The offset prediction module is configured to predict the offset of keypoints within a local window based on paired feature descriptors, using the keypoint offset prediction module. The registration module is configured to: perform position calibration on key points in the key point probability map based on the offset, and output the registered image.

7. A computer-readable storage medium having stored thereon a program, characterized in that, When executed by the processor, the program implements the steps in the retinal image registration method based on mesh supervision and key point learning as described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the retinal image registration method based on grid supervision and key point learning as described in any one of claims 1-5.

Images