OCT image stitching method

Abstract

The present application relates to the field of image processing technology and provides an OCT image stitching method, comprising: obtaining an image pair and a pair of matching points corresponding to the image pair, wherein the image pair is adjacent OCT images; calculating a homography matrix based on the matching point pair to align the image pair, and outputting a preliminary aligned image pair; constructing a fusion weight map, wherein the fusion weight map includes weight values ​​and region types for pixel positions; and performing fusion stitching on the pixel positions based on the weight values ​​and region types to output a panoramic OCT image. The method implements partitioned fusion by fusing the weight map, thereby resolving tissue misalignment and artifact issues in OCT stitching, and thereby addressing the problem of poor stitching quality in OCT image stitching.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an OCT image stitching method. Background Art

[0002] Optical coherence tomography (OCT) is a non-invasive imaging technique used to observe the microstructure of biological tissues such as the retina and skin. Clinical diagnosis requires stitching multiple local OCT images into a panoramic view to fully visualize vascular networks, tissue structures, and their pathological features. The stitched image must maintain the continuity of microstructures such as intact tubular microvessels, maintain uniform grayscale distribution across scanned areas, and ensure seamless anatomical layer boundaries to avoid misalignment artifacts.

[0003] During the OCT image stitching process, a linear fusion strategy is used to weight and smooth overlapping areas based on pixel spatial position. This fails to consider the unique layered signal intensity distribution patterns of OCT images, leading to fusion distortion of deep tissue structures, manifested as blurred vascular edges and inter-layer misalignment artifacts. Furthermore, the stitching process lacks targeted optimization of the pathological characteristics of the target area, resulting in the loss or distortion of key diagnostic information.

[0004] In summary, the stitching effect of OCT image stitching is poor. Summary of the Invention

[0005] The present application provides an OCT image stitching method to solve the problem of poor stitching effect of OCT image stitching.

[0006] This application provides an OCT image stitching method, comprising:

[0007] Acquire an image pair and a pair of matching points corresponding to the image pair, wherein the image pair is adjacent OCT images;

[0008] Calculating a homography matrix based on the matching point pair to align the image pair, and outputting a preliminary aligned image pair;

[0009] Constructing a fusion weight map, wherein the fusion weight map includes weight values ​​and region types of pixel positions;

[0010] According to the weight value and the region type, fusion stitching is performed on the pixel positions to output a panoramic OCT image.

[0011] In some feasible embodiments, acquiring an image pair and a matching point pair corresponding to the image pair includes:

[0012] Acquire an image pair, the image pair comprising a plurality of images;

[0013] Performing target detection on the image pair to locate the target area, wherein the target detection is performed by a target detection model;

[0014] A feature matching algorithm is used to perform feature matching on the target area to output matching point pairs, where the matching point pairs are feature matching point pairs between adjacent images.

[0015] The reliability of matching points is improved through layered target positioning, mismatching caused by background interference is avoided, and the robustness of splicing is enhanced.

[0016] In some feasible embodiments, calculating a homography matrix based on the matching point pairs to align the image pairs and outputting a preliminary aligned image pair includes:

[0017] Eliminate mismatched points in the matching point pairs through outliers to output target matching point pairs;

[0018] Based on the target matching point pairs, a transformation matrix is ​​calculated using a geometric estimation algorithm;

[0019] The transformation matrix is ​​applied to align the image pair to generate a preliminary aligned image.

[0020] Outlier removal can ensure the accuracy of the geometric transformation matrix, optimize image alignment accuracy, and reduce the accumulation of alignment deviations.

[0021] In some feasible embodiments, constructing a fusion weight map includes:

[0022] Calculating the pixel-level displacement field of the preliminary aligned image by a multi-scale optical flow algorithm to determine the deformation of the local area;

[0023] Construct fusion weight graph;

[0024] Based on the deformation amount, the fusion weight map is divided to output a high confidence area, a deformation transition area and a highly deformed area, the weight value of the high confidence area is a first value, the weight value of the deformation transition area is a second value, and the weight value of the highly deformed area is a third value, the first value is greater than the second value, and the second value is greater than the third value.

[0025] Multi-scale optical flow quantifies local deformation, and the partition weight strategy adaptively fuses different deformation areas to suppress seams.

[0026] In some feasible embodiments, performing fusion stitching on pixel positions according to the weight values ​​and the region types to output a panoramic OCT image includes:

[0027] Performing maximum fusion on the high confidence area, performing weighted hybrid fusion on the deformation transition area, and performing elastic deformation compensation fusion on the pixels in the highly deformed area to output a fusion result;

[0028] In the fusion result, locating the seam area;

[0029] Acquire the similarity of the seam area, and if the similarity is less than a similarity threshold, update the fusion weight map of the seam area;

[0030] If the similarity is greater than or equal to the similarity threshold, a panoramic OCT image is output.

[0031] Regional fusion can maximize the retention of effective information and improve image continuity through the seam self-checking mechanism.

[0032] In some feasible embodiments, the target detection model includes a first level, a second level, a third level, a fourth level, a fifth level, and a sixth level;

[0033] The first layer, the second layer, the third layer, and the fourth layer have the same structure and all include a star aggregation module, and the output feature map sizes of the first layer, the second layer, the third layer, and the fourth layer are different;

[0034] The method further comprises:

[0035] Performing double upsampling on the feature map output by the sixth level to obtain a first sampling result;

[0036] Perform channel splicing on the first sampling result and the feature map output by the third level to output a first fused feature;

[0037] Perform channel concatenation on the feature map output by the second layer, the first fused feature, and the feature map output by the first cross-layer aggregation convolution module to output a second fused feature;

[0038] Inputting the second fused features into a first cross-layer aggregation convolution module to output a first feature map;

[0039] Perform feature splicing on the second fused feature and the first feature map to output a third fused feature.

[0040] By integrating multi-scale features through cross-layer aggregate convolution, the detection ability of small targets such as retinal layer structures is enhanced.

[0041] In some feasible embodiments, performing target detection on the image pair to locate the target area includes:

[0042] Inputting the third fused feature into a second cross-layer aggregation convolution module to output a second feature map;

[0043] After passing the first feature map through the second feature extraction module, the first feature map is concatenated with the second feature map to output a third feature map;

[0044] Inputting the third feature map into a third cross-layer aggregation convolution module to obtain a fourth feature map;

[0045] Inputting the fourth feature map into a third feature extraction module, and concatenating the feature map output by the second feature extraction module and the sixth layer to output a fifth feature map;

[0046] Inputting the fifth feature map into a fourth cross-layer aggregation convolution module to obtain a sixth feature map;

[0047] The second feature map is input into the first detection head, the fourth feature map is input into the second detection head, and the sixth feature map is input into the third detection head to output the target area. Multiple detection heads collaborate to optimize the target positioning accuracy and improve the basic matching quality of key areas such as blood vessel branches.

[0048] In some feasible embodiments, the star aggregation module includes a central node and at least two radial branches;

[0049] Before performing double upsampling on the feature map output by the sixth level to obtain a first sampling result, the method further includes:

[0050] Inputting the image pair after the first feature extraction module into the central node to output an intermediate feature map;

[0051] Inputting the intermediate feature map into a first radial branch to output a first radial feature map, and inputting the intermediate feature map into a second radial branch to output a second radial feature map;

[0052] The first radial feature map and the second radial feature map are merged to generate a merged result, and the merged result is input into the sixth layer.

[0053] In some feasible embodiments, performing feature matching on the target area using a feature matching algorithm to output matching point pairs includes:

[0054] performing preprocessing on the target area to output a preprocessing result;

[0055] Passing the preprocessing results through a feature extraction network to extract feature points of multiple target areas;

[0056] Processing the feature points through graph attention matching to obtain node features;

[0057] performing a bidirectional nearest neighbor search based on the node features to generate candidate matching pairs;

[0058] performing geometric verification on the candidate matching pairs to output a confidence score for the candidate matching pairs;

[0059] If the confidence score is greater than the score threshold, the candidate matching pair corresponding to the confidence score greater than the score threshold is determined as a matching point pair.

[0060] Graph attention enhances feature relevance, and bidirectional search combined with geometric verification ensures the spatial consistency of matching point pairs.

[0061] In some feasible embodiments, the feature extraction network includes multiple first layers; the first layers include a downsampling operation and multiple dynamic convolution blocks;

[0062] The preprocessing result is passed through a feature extraction network to extract feature points of multiple target areas, including:

[0063] Inputting the preprocessing results into a plurality of the first levels in sequence, passing through the dynamic convolution blocks in the first levels, and performing a downsampling operation to output level results;

[0064] Extract target feature points from the hierarchical results.

[0065] The dynamic convolution block adaptively adapts to changes in tissue morphology and extracts feature points in layers to improve the generalization ability of the matching algorithm.

[0066] As can be seen from the above technical solution, the present application provides an OCT image stitching method, comprising: obtaining an image pair and a pair of matching points corresponding to the image pair, wherein the image pair is adjacent OCT images; calculating a homography matrix based on the matching point pairs to align the image pair, and outputting a preliminary aligned image pair; constructing a fusion weight map, wherein the fusion weight map includes weight values ​​and region types for pixel positions; and performing fusion stitching on the pixel positions based on the weight values ​​and region types to output a panoramic OCT image. The method achieves partitioned fusion by fusing the weight map, thereby resolving the problems of tissue misalignment and artifacts in OCT stitching, and thereby solving the problem of poor stitching quality in OCT image stitching. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0068] Figure 1 A schematic diagram of the process of the OCT image stitching method provided in an embodiment of the present application;

[0069] Figure 2 A schematic diagram of the nth frame image in an image pair provided in an embodiment of the present application;

[0070] Figure 3A schematic diagram of the n+1th frame image in an image pair provided in an embodiment of the present application;

[0071] Figure 4 A schematic diagram of target detection and positioning of a target area provided in an embodiment of the present application;

[0072] Figure 5 A schematic diagram of the structure of the target detection model provided in the embodiment of the present application;

[0073] Figure 6 A schematic diagram of the structure of a star-shaped aggregation module provided in an embodiment of the present application;

[0074] Figure 7 A schematic diagram of a panoramic OCT image provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] The following embodiments are described in detail, with examples shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following embodiments are not intended to represent all embodiments consistent with this application.

[0076] The signal intensity of tissues at different depths in OCT images decays exponentially. When linear fusion strategies assign weights based on spatial distance, weak signals in deep layers are overwhelmed by strong signals in the weighted average. This forces smoothing of inter-layer intensity transitions caused by the attenuation curve. Traditional methods assume that the overlapping areas of adjacent images are topologically continuous, but OCT images include vascular networks and pathological areas. The smoothing operation of linear fusion can blur microvascular bifurcations, key stitching landmarks, and weaken the abrupt transition between edematous and non-lesional areas, leading to erroneous information in the OCT images and poor stitching results.

[0077] To solve the above problems, the present invention provides an OCT image stitching method, such as Figure 1 As shown, the following steps are included:

[0078] S100: Obtain image pairs and matching point pairs corresponding to the image pairs.

[0079] An image pair is an adjacent OCT image. Specifically, an image pair is two spatially adjacent OCT tomographic images, which can be obtained by continuously scanning an OCT device along a tissue surface. For example, a sequence of continuously scanned retinal tomographic images can be obtained by an OCT device.

[0080] Matching point pairs are a set of feature point coordinates with the same anatomical structure extracted from adjacent OCT images and are used to establish a spatial mapping relationship between images.

[0081] In some embodiments, the Scale-Invariant Feature Transform (SIFT) algorithm is used to detect key points in the image, such as intersections of retinal layer boundaries and vascular bifurcations, and descriptor matching is used to generate a set of matching point pairs. The matching point pairs must meet a bidirectional consistency check. That is, when point A in the first image matches point B in the second image, the reverse matching result of point B must also be point A to eliminate artifacts.

[0082] S200: Calculating a homography matrix based on the matching point pairs to align the image pairs, and outputting a preliminary aligned image pair.

[0083] The homography matrix is ​​a 3×3 matrix that describes the perspective transformation relationship between two images. The matrix parameters are optimized using the Random Sample Consensus (RANSAC) algorithm by solving the coordinate correspondence relationship between matching point pairs.

[0084] For example, matching point pairs are substituted into the homography to construct an overdetermined system of equations. An initial matrix is ​​then calculated using iterative random sampling. Outliers with projection errors greater than a threshold are removed, and the solution with the most inliers is used as the optimized matrix. After this transformation, the spatial coordinate systems of the two images are aligned, and a preliminary aligned image pair corresponding to the corresponding pixel positions is output.

[0085] S3000: Construct a fusion weight graph.

[0086] The fusion weight map is a two-dimensional matrix of the same size as the image, containing a weight value for each pixel and an identifier for the region type. The weight value is calculated based on the distance from the pixel to the boundary of the overlapping region, with the closer the pixel is to the boundary, the smaller the weight.

[0087] The regional types are divided based on the hierarchical structure of OCT images. For example, they are divided into surface high-signal areas, such as the nerve fiber layer, middle homogeneous areas, such as the outer nuclear layer, and deep low signal-to-noise ratio areas, such as the choroid layer. A weight attenuation curve is set for each type of area. For example, an exponential attenuation curve is used for the deep low signal-to-noise ratio area, and a linear attenuation curve is used for the surface high-signal area to adapt to the signal attenuation characteristics of different tissues.

[0088] S400: performing fusion stitching on pixel positions according to weight values ​​and region types to output a panoramic OCT image.

[0089] In the overlapping area, a weighted average is performed on the same pixel position of the two images, and the output pixel value is the product of the pixel value of image A and the weight A, and the sum of the product of the pixel value of image B and the weight B, where the sum of weight A and weight B is 1.

[0090] For superficial high-signal areas, the weight curve is smoothed to ensure texture continuity. For deeper regions, when vascular structures are detected, the weight of non-vascular tissue is dynamically lowered based on morphological segmentation results to reduce blood flow signal interference. Non-overlapping areas retain the original pixel values, generating seamless panoramic OCT images.

[0091] The method provided in this embodiment applies weighted attenuation to deep, low-SNR regions, preventing linear fusion from averaging vascular signals across low-SNR tissue and enhancing the sharpness of choroidal neovascularization edges. During the homography matrix calculation phase, feature point screening eliminates anomalous matching points perpendicular to the layering direction, ensuring that the image alignment process conforms to the retinal layered structure. This weighted distribution ensures high accuracy of panoramic images in both deep and shallow tissue layers.

[0092] Before obtaining matching point pairs, target detection is performed on the image pairs through the target detection model, and then feature matching is performed through the feature matching algorithm to output matching point pairs.

[0093] The object detection algorithm, based on a fixed-size, preset anchor box mechanism, struggles to dynamically adapt to the multi-scale characteristics of tiny lesions in OCT images, leading to significant issues with missed and false detections in low-contrast, high-noise areas. Furthermore, the detection network is not optimized for the layered structure of OCT images, limiting the accuracy of target location and affecting stitching accuracy.

[0094] In some embodiments, matching point pairs are obtained by the following steps:

[0095] S210: Acquire an image pair.

[0096] In retinal imaging scenarios, image pairs are acquired continuously in the horizontal or vertical direction by a fundus scanner, and their overlapping areas contain common anatomical structures. For example, in retinal scanning, the overlapping parts of adjacent images must include the same vascular branch or the same optic nerve fiber layer cross-section structure. Figure 2 、 Figure 3 As shown, the nth frame and the (n+1)th frame image form an image pair.

[0097] There are some overlapping areas in adjacent images, and the images may be offset due to hand shaking or eye movement. In the process of continuous shooting, for example, ten images are taken, and these ten images need to be superimposed and stitched, but they cannot be directly superimposed and stitched due to displacement. Therefore, feature stitching needs to be performed.

[0098] The image pairs include multiple, that is, all consecutive overlapping image combinations in the processed image sequence, for example, frame 1-2, frame 2-3 to frame N-1-N, to achieve panoramic stitching of the entire scanning path.

[0099] like Figure 4 As shown, however, because the original image is large and the superposition time is long, it is necessary to perform target detection first, locate the OCT foreground area, and stitch the areas with signals to shorten the stitching time.

[0100] S220: Perform target detection on the image pair to locate the target area.

[0101] Target detection can be performed using a target detection model, which outputs the boundary location of the target region and a structural type label. The target region is defined by the target detection model, where the target region is a local image range. It is understood that different images have different target regions, but the target region includes areas with high feature matching value. High feature matching value can be determined by proportion and / or intersection, density, and lesion areas. For example, anatomically significant locations such as the intersection of optic nerve fiber bundles, the foveal vascular density area, and the choroidal lesion shadow area can be spliced. Alternatively, in the OCT foreground area, areas with signal can be spliced.

[0102] Exemplarily, the target area is the minimum bounding rectangle that includes key anatomical features, and its size is automatically adjusted according to the detected tissue structure characteristics. For example, the macular area occupies approximately 10%-15% of the total image area, and the optic disc peripheral area occupies 25%-30%.

[0103] In one implementation, the object detection model is a convolutional neural network-based region recognition module that locates key anatomical regions in an image. The object detection model is trained using a pre-trained dataset of retinal OCT images, taking a single OCT image as input and outputting bounding box coordinates.

[0104] For example, in the case of diabetic macular edema, the model can accurately locate the cystoid edema area, the fovea, and the coordinates of microaneurysms.

[0105] In another implementation, the target detection model is the YOLO v4-tiny lightweight framework optimized for OCT medical images. The input layer is optimized, a multi-scale reflection intensity normalization module is added, and a depthwise separable convolution is used instead of a standard convolution to reduce the number of parameters. A cross-layer retinal feature fusion module is added. For example, skip connections are added to the output ends of the 3rd, 5th, and 7th layers of CSPDarknet53 to fuse retinal layer features of different scales.

[0106] It is also possible to use the Transformer-Encoder hybrid architecture, use ConvNeXt-v2 in the feature extraction layer to extract multi-scale local features, add a retinal topology-aware Transformer module in the relationship modeling layer, enhance anatomical rationality through preset inter-layer connection constraints, output heat maps and boundary vectors, and solve the problem of blurred outlines of tiny lesions.

[0107] For example, the two images in the image pair, defined as image A and image B, are fed into an object detection model. The model uses a U-Net architecture, whose convolutional layers optimize the reflectance intensity response parameters for OCT images.

[0108] The model outputs structural and pathological features. Structural features include the internal limiting membrane reflective band and the edge of Bruch's membrane, while pathological features include subretinal fluid accumulation and neovascular plexuses. For example, if a 200μm diameter cystoid edema region is detected in the fovea in image B, the model will output the bounding box coordinates of this region [(x1, y1), (x2, y2)].

[0109] Perform matching detection on the same anatomical location in image A to ensure biological correspondence of the target region. For example, the cystic edema region in image B must be detected as a continuous structure at the corresponding location in image A.

[0110] S230: Perform feature matching on the target area using a feature matching algorithm to output matching point pairs.

[0111] The feature matching algorithm is used for the local area defined by target detection, i.e., the target region, rather than the entire image, using layered matching. In one implementation, rotationally invariant ORB (Oriented FAST and Rotated BRIEF) feature points are extracted from the target region. The matching direction is constrained based on the characteristics of the OCT slice, and point pairs are selected using the Hamming distance to achieve feature correspondence within the pathological region.

[0112] In other embodiments, a feature extraction network can be used to adopt a twin dual-path ResNet-18 architecture, share weights, improve the third layer of convolution, and perform dilated convolution to enhance the capture of OCT layered textures, output a 128-dimensional feature vector, and then output matching point pairs.

[0113] After generating the matching point pairs, a homography matrix is ​​calculated based on the matching point pairs, the image pairs are aligned, and a preliminary aligned image pair is output. In some embodiments, incorrect matching points in the matching point pairs are removed by using outliers to output a target matching point pair; a transformation matrix is ​​calculated based on the target matching point pair using a geometric estimation algorithm; and the transformation matrix is ​​applied to align the image pairs to generate a preliminary aligned image.

[0114] After generating matching point pairs, some may not conform to the displacement patterns of the actual anatomical structure. For example, in OCT image processing, vitreous reflection artifacts or repetitive texture areas in subcutaneous fat tissue may occur. Therefore, we first remove mismatched points using discrete points. Outlier point removal uses statistical analysis to identify and remove mismatched point pairs. This involves performing a bidirectional consistency check and displacement vector clustering analysis on the matching point pairs, discarding outliers that deviate from the main displacement model, and outputting the target matching point pairs. For example, when splicing retinal vessels, matching points perpendicular to the layering direction are removed.

[0115] It is understood that the target matching point pair is a set of spatially corresponding points after outlier screening. The target matching point pair satisfies a preset condition, which may be that the point pairs satisfy anatomical structure consistency rules and / or the overall distribution conforms to basic geometric constraints.

[0116] A geometric estimation algorithm is then executed based on the target matching points. This algorithm is used to establish a spatial mapping model of adjacent images and solve for the transformation matrix parameters based on the coordinates of the matching points. In this embodiment, a global affine transformation matrix is ​​estimated using the RANSAC estimation algorithm to preliminarily align the general structure of the image pairs. The algorithm then iteratively randomly samples point pairs to calculate candidate transformation matrices and selects the matrix that satisfies the most inliers, meaning that the projection error is less than a preset threshold, as the optimal solution.

[0117] Specifically, the transformation matrix is ​​a 3×3 homography. The matrix elements are divided into three parts: a 2×2 submatrix in the upper left corner represents rotation and scaling; a 2×1 vector in the upper right corner represents translation; and the bottom row controls the projection transformation. For example, in retinal OCT scenarios, the matrix form is simplified to an affine transformation to accommodate the approximately planar structure of the eye.

[0118] After obtaining the global affine transformation matrix, the image pairs are aligned. These pre-processed image pairs may contain local non-rigid displacements, requiring a transformation matrix to achieve global coordinate system 1. After mapping through the transformation matrix, a set of images is obtained, known as the preliminary aligned images. The alignment process uses bilinear interpolation to calculate pixel values ​​and maintain tissue texture continuity.

[0119] For example, in the OCT image analysis of diabetic macular edema, the initial adjacent image pairs contained 38 matching points. Direction-constrained filtering removed eight abnormally displaced points perpendicular to the retinal layers. Distance constraints further eliminated three matching points that exceeded the choroidal displacement limit. The remaining 27 matching points were then subjected to a RANSAC optimization process, with a strict 1.5-pixel error threshold applied in the nerve fiber layer and a relaxed 2.5-pixel error threshold in the choroidal layer. After five iterations, convergence was achieved, resulting in an optimal homography matrix containing 21 inliers.

[0120] During the matrix solution phase, cystoid edema lesions in the foveal region were identified and excluded from the parameter calculation. After applying the transformation, the two images were aligned with an accuracy of 2 microns at the optic nerve fiber layer, and the boundaries of the edema areas were stitched together without misalignment.

[0121] After the rough structure of the image pair is preliminarily aligned, a dense optical flow algorithm is used to calculate the pixel-level displacement field to quantify the local deformation intensity. In some embodiments, the pixel-level displacement field of the preliminarily aligned images is calculated using a multi-scale optical flow algorithm to determine the deformation amount of the local area.

[0122] The multi-scale optical flow algorithm, based on the assumption of constant brightness and spatial continuity constraints, calculates pixel motion vectors by establishing a multi-scale pyramid structure. Specifically, optical flow is calculated starting from the coarsest scale, and the result is used as the initial value for the next level, which is gradually optimized. The final output is a displacement vector field, which represents the degree of local deformation between the two images after alignment.

[0123] In OCT stitching, optical flow analysis targets preliminarily aligned image pairs. The algorithm outputs a displacement vector field, where the vector direction indicates tissue movement and the amplitude represents the degree of deformation. Algorithms implemented include the Horn-Schunck method, which assumes constant brightness, and the Lucas-Kanade method, which relies on gradient calculation.

[0124] The pixel-level displacement field is a two-dimensional vector field, with each pixel position corresponding to a displacement vector (dx, dy). The modulus of the displacement vector represents the deformation at that location, and the direction indicates the direction of tissue movement. In OCT images, the magnitude of the deformation indicates the state of tissue compression or the degree of scanning error. For example, the deformation of the fovea is smaller than that of the surrounding area. In retinal scanning scenarios, the displacement field in the temporal retinal region exhibits a diverging vector pattern, while the nasal region exhibits a converging pattern, reflecting the curved surface characteristics of the eyeball.

[0125] The fusion weight map is then constructed. This is a two-dimensional matrix of the same size as the original image, with each element storing a weight value and a region type identifier. The weight value ranges from [0, 1] and is used for subsequent image fusion. The region type identifier is a discrete classification value that indicates the fusion strategy partition to which the current pixel belongs.

[0126] The construction of the weight map is based on deformation analysis, and differentiated fusion weights are assigned to different deformation areas. That is, based on the deformation value, the fusion weight map is divided to output high confidence areas, deformation transition areas, and highly deformed areas.

[0127] Among them, the high confidence zone is the area where the deformation variable is less than the set threshold, the tissue alignment accuracy is high, and the maximum weight value, i.e., the first value, is assigned; the deformation transition zone is the area where the deformation variable is in the middle range, there is slight deformation, and a medium weight value, i.e., the second value, is assigned; the high deformation zone is the area where the deformation variable exceeds the reasonable range, there is significant misalignment or artifacts, and the minimum weight value, i.e., the third value, is assigned. The first value is greater than the second value, and the second value is greater than the third value.

[0128] Exemplarily, a high confidence zone refers to an area where the deformation amount is less than a threshold. In retinal imaging, it includes flat areas such as the fovea centralis and the middle of the optic nerve fiber layer. The tissue structures in such areas are precisely aligned and the anatomical features are clear. The deformation transition zone is a band-shaped area with a medium deformation amount. For example, the transition zone from the edge of the retina to the equator of the eyeball, and the boundary area where the blood vessels cross the layered structure. This area has a mild deformation gradient and requires a smooth fusion transition mechanism. The high deformation zone is an area where the deformation amount exceeds the upper limit value. For example, the edge of the equator of the eyeball, the nasotemporal bend of the optic disc, and other locations with sudden changes in curvature. This area causes the anatomical structure to be squeezed and deformed due to the projection of the curved surface of the eyeball, and requires special fusion compensation processing.

[0129] In some embodiments, maximum fusion is performed on high confidence areas, and maximum fusion selects the best pixel intensity values: the pixel values ​​of the two images are compared at each corresponding position, and the pixel with higher signal intensity is selected as the output result. It is suitable for high signal-to-noise ratio areas and can preserve the integrity of tissue structure.

[0130] Weighted hybrid fusion is performed on the deformation transition area. The weighted hybrid fusion linearly combines the pixels of the two images based on the preset weight coefficient. The smooth transition strategy can alleviate the edge discontinuity caused by slight deformation.

[0131] Elastic deformation compensation fusion is performed on pixels in highly deformed areas. Elastic deformation compensation fusion establishes a local elastic deformation model: tissue stress distribution is calculated through finite element analysis, and pixel arrangement is reconstructed to compensate for the projection distortion of deep tissue.

[0132] After the three areas are fused, the intermediate image after the pixel synthesis operation is output, that is, the fusion result. The image size of the fusion result covers the maximum external range of the original image pair. There are three-channel data in the overlapping area, namely the synthesized pixel value, source label, and fusion confidence. The original image data is retained in the non-overlapping area.

[0133] It is understood that the stitching result is the result of stitching multiple images. The joint between any two images includes a stitching seam area. For example, in a retinal panorama, this area is distributed in a strip along the scanning direction. Exemplarily, the angle between the extended image boundary line and the actual fusion boundary is detected. For example, an angle greater than 30° is determined as a seam turning point.

[0134] Get the similarity of the seam area. If the similarity is less than the similarity threshold, update and optimize the original weight distribution of the seam area. For example, expand the transition zone bandwidth by 5-10 pixels and increase the weight by 0.2 at the structural breakpoint.

[0135] A 5-pixel-wide linear gradient transition is used at the boundary between the overlapping and non-overlapping areas to avoid obvious seams. The transition weight decreases linearly from 100% in the overlapping area to 0% in the non-overlapping area.

[0136] For the updated fusion weight map, the updated fusion weight map is divided again based on the deformation variable to output the updated high confidence area, deformation transition area and high deformation area, and maximum fusion, weighted hybrid fusion and elastic deformation compensation fusion are performed until the similarity of the seam area in the output fusion result is greater than or equal to the similarity threshold.

[0137] The output panoramic OCT image is the processed wide-field retinal structural imaging result. For example, by stitching multiple local OCT scan images with a scanning range of 6mm×6mm, a composite image that exceeds the physical limit of a single scan is generated, for example, 12mm×9mm, covering the retinal functional area from the fovea to the edge of the optic disc.

[0138] During the target detection process, a general target detection algorithm can be used to locate the target area in the image. The algorithm relies on a preset fixed-size anchor box to generate candidate areas and screens the targets for cropping based on the confidence threshold. However, this type of method has limited detection capabilities for small target areas, and is prone to missed detections and misjudgments, especially in low signal-to-noise ratio scenarios. In the feature matching stage, a dense feature extraction algorithm is used to detect and describe feature points on the cropped image blocks, and matching pairs are screened by calculating feature similarity, and outliers are eliminated in combination with a geometric consistency algorithm. This type of method is prone to mismatching in areas with repeated textures or blurred boundaries, such as the junction of retinal layered structures, and the computational complexity is high, making it difficult to meet real-time processing requirements.

[0139] In addition, the detection network was not optimized for the layered structural characteristics of OCT images, resulting in limited positioning accuracy of the target area and affecting the accuracy of stitching.

[0140] For the target detection model, in this embodiment, the StarNet-yolo architecture is used to integrate deformable convolution and anatomical attention mechanisms. The network body is stacked by multi-level feature extraction modules. Each level of module adapts to tissue deformation through a deformable convolution layer and can capture complex anatomical structures such as retinal curvature and skin folds.

[0141] Specifically, in order to enhance the features of medical images, an anatomical attention mechanism is set up, and a spatial weight map is generated using prior knowledge such as vascular density and layered texture, and high-information areas are highlighted through adaptive weighting.

[0142] This solution uses StarNet as the backbone network of YOLOv11. By introducing the StarBlock module to replace the CBS and C3k2 units in the original structure, the backbone network depth is compressed from 11 to 7 layers. This significantly reduces computational complexity while maintaining feature extraction capabilities, targeting the specific characteristics of object position detection in OCT images.

[0143] In some embodiments, as Figure 5 As shown, the target detection model includes a first level, a second level, a third level, a fourth level, a fifth level, and a sixth level; wherein the first level, the second level, the third level, and the fourth level have the same structure and all include a star-shaped aggregation module StarBlock.

[0144] The first, second, third, and fourth layers are constructed using CBS and StarBlock. CBS consists of convolutional layers, batch normalization, and SiLU activation. A first feature extraction module is included before the first layer. The output OCT image is sized at 640×640×3, and downsampling is performed using the convolution kernel in the first feature extraction module to capture the overall layered structure of the retina.

[0145] like Figure 6 As shown in the figure, StarBlock captures features of different scales through multiple parallel branches. Each StarBlock contains three branches with convolution kernels of different sizes (1x1, 3x3, 5x5), which can capture local details and global context information at the same time.

[0146] In some embodiments, the image pair after passing through the first feature extraction module is input to the central node to output an intermediate feature map;

[0147] Inputting the intermediate feature map into a first radial branch to output a first radial feature map, and inputting the intermediate feature map into a second radial branch to output a second radial feature map;

[0148] The first radial feature map and the second radial feature map are merged to generate a merged result.

[0149] Specifically, each StarBlock consists of a convolutional group with a star topology, consisting of a central node (a 3×3 depthwise separable convolution) and two radial branches: a 1×1 convolution and a 3×3 or 5×5 dilated convolution. This module enhances the ability to extract layered texture features through parallel computation across different receptive fields. After passing through the central node, the input features are split into two branches: one undergoing a 1×1 convolution and the other a 3×3 or 5×5 dilated convolution. The convolution results are then merged, followed by feature compression (depthwise convolution), and finally residual fusion.

[0150] Among them, the large-size convolution kernel focuses on the wide-field tissue layer structure and extracts the overall layered direction of the retina. The second radial branch is configured with a small-size convolution kernel to capture the capillary microstructure and generate microvascular wall coordinate information. The branch output features are superimposed in the channel dimension in the merging layer to achieve enhancement of macroscopic structure and microscopic features.

[0151] Compared with the original CBS and C3k2 modules, StarBlock achieves similar feature expression capabilities while reducing the number of parameters through multi-scale receptive field fusion and parameter sharing.

[0152] At the first level, StarBlock receives the feature map processed by CBS. The central node can be a 3×3 depthwise separable convolution (DSC) to extract basic spatial features and reduce computational complexity. Radial branch 1 uses 1×1 convolution to adjust the number of channels, such as dimensionality increase or decrease, to control the number of feature channels and adapt to subsequent operations. Radial branches use 5×5 or 3×3 dilated convolutions. The 5×5 dilated convolution captures wide-field layered textures, such as retinal layer boundaries, while the 3×3 dilated convolution focuses on local microstructures, such as the capillary grid. The features of the two branches are then fused through channel concatenation to compress the information. Finally, the residual is fused (adding it to the input features) to obtain the output feature map. This output is then added to the 320×320×8 input map to produce a 160×160×16 feature map.

[0153] The execution process of the second, third, and fourth levels is similar to that of the first level and will not be repeated here. However, the difference is that different levels process different feature maps in a progressively larger scale. For example, the first level has a resolution of 160×160, perceives a 2mm field of view, and is suitable for analyzing the layered structure of the entire macular area. The second level has a resolution of 80×80, focusing on a 1mm area to capture capillary bifurcations. The third level has a resolution of 40×40, observing 500μm details and identifying microaneurysm morphology. The fourth level has a resolution of 20×20, resolving 250μm microstructures and locating individual edema cell clusters.

[0154] In this implementation, YOLOv11's original 11-layer architecture is restructured into a 7-layer pyramid. Each level of the feature pyramid consists of two paths: upsampling and downsampling. The downsampling path compresses the feature map size using convolutions with a stride of 2; the upsampling path restores resolution through transposed convolutions. The outputs of these two paths are fused using a gating mechanism. A sigmoid function is used to generate a spatial weight map, dynamically assigning the importance of features at each location and enhancing edge feature responses at the junction of the inner and outer retinal layers.

[0155] The input size of the input layer is 640×640×3. The first, second, third, and fourth layers perform feature abstraction. Each layer contains a star aggregation module and outputs feature maps of decreasing resolution. The output feature maps have different sizes. The feature map output by the first layer is 160×160×16, and the first order retains the fine structure of the capillary plexus. The feature map output by the second layer is 80×80×32, focusing on the mid-scale features of the vascular arch area. The feature map output by the third layer is 40×40×64, and the third layer captures the pathological features of the optic disc area. The feature map output by the fourth layer is 20×20×128, encoding the topological relationship of the entire retina.

[0156] The combined result of the residual fusion performed in the fourth layer is input to the fifth layer. The fifth layer is the SPPF (Spatial Pyramid Pooling–Fast) layer. The fifth layer simultaneously feeds the feature map input from the fourth layer into three pooling layers for multi-scale feature extraction, thereby enhancing the model's ability to detect objects of different sizes.

[0157] The SPPF module consists of an initial convolutional layer for feature compression, followed by multiple serial maximum pooling operations using a 5x5 pooling kernel with a stride of 1 and padding of half the pooling kernel size, and finally a convolutional layer that fuses features of different scales. The output feature map size is 20×20×1024.

[0158] The output of the fifth level is input to the sixth level (C2PSA), and the output feature map size is 20×20×1024. For the sixth level, in some embodiments, the feature map output by the sixth level is upsampled by a factor of two to obtain a first sampling result;

[0159] Perform channel splicing on the first sampling result and the feature map output by the third level to output a first fused feature;

[0160] Perform channel concatenation on the feature map output by the second layer, the first fused feature, and the feature map output by the first cross-layer aggregation convolution module to output a second fused feature;

[0161] Inputting the second fused features into a first cross-layer aggregation convolution module to output a first feature map;

[0162] Perform feature splicing on the second fused feature and the first feature map to output a third fused feature.

[0163] For the first fusion feature, the middle-layer tissue structure and deep-layer pathological features are integrated, and the number of channels is expanded to 1088 to enhance the lesion representation ability. For the second fusion feature, the shallow capillary coordinates are fused with the deep semantic features to eliminate the spatial information loss caused by four-fold downsampling, generating a 544-channel comprehensive feature.

[0164] In some embodiments, the target area is outputted by the following steps:

[0165] Inputting the third fused feature into a second cross-layer aggregation convolution module to output a second feature map;

[0166] After passing the first feature map through the second feature extraction module, the first feature map is concatenated with the second feature map to output a third feature map;

[0167] Inputting the third feature map into a third cross-layer aggregation convolution module to obtain a fourth feature map;

[0168] Inputting the fourth feature map into a third feature extraction module, and concatenating the feature map output by the second feature extraction module and the sixth layer to output a fifth feature map;

[0169] Inputting the fifth feature map into a fourth cross-layer aggregation convolution module to obtain a sixth feature map;

[0170] The second feature map is input to a first detection head, the fourth feature map is input to a second detection head, and the sixth feature map is input to a third detection head to output a target area.

[0171] For the third feature map, it is spliced ​​with the feature map of size 40×40×512 output by the first cross-layer aggregation convolution module to generate 768-channel features; for the fifth feature map, the feature map of size 20×20×1024 output by the sixth level is spliced ​​to generate 1536-channel features, and then passed through the fourth cross-layer aggregation convolution module to obtain the sixth feature map. The sixth feature map is the feature map after the channel is compressed, which can reduce the computational complexity of the detection head.

[0172] The 20×20×1024 deep features output by the sixth level are restored to 40×40 resolution after double upsampling, and then channel-joined with the 40×40×64 middle-level features output by the third level to generate 1088-dimensional fusion features. This feature flows through the first cross-layer aggregation convolution module, where the spatiotemporal dilated convolution kernel strengthens the contextual association of the capillary gap and outputs an optimized 40×40×512 feature map. This map is integrated with the original features of the second level at the splicing node and then input into the third feature extraction module for morphological refinement. The 20×20×512 output is spliced ​​with the original features of the sixth level to form a full-path integrated feature, which is finally extracted by the fourth cross-layer aggregation convolution module as a 20×20×1014 pathological response feature and delivered to the third detection head.

[0173] Illustratively, the first detection head receives an 80×80×256 feature map to parse the coordinates of the retinal layer boundaries, the second detection head receives a 40×40×512 feature map to locate the coordinates of microvascular lesions, and the third detection head receives a 20×20×1014 full-path feature map to mark the cystoid edema area. The three detection heads output spatial coordinates to form a lesion distribution map with millimeter-level accuracy in the macular area.

[0174] The target detection process provided by Yolov11 and this application was evaluated using OCT images. The evaluation results are shown in the following table:

[0175]

[0176] The target detection network provided in this embodiment has a maximum value of 0.995 in the mean average precision 50 (B) indicator, which is 0.00007 higher than 0.99493 of Yolov11. The introduction of the star aggregation module and the cross-layer feature fusion mechanism in the network architecture can effectively enhance the recognition accuracy of the retinal layer structure in OCT images, especially in the positioning of key anatomical structures such as the nerve fiber layer boundary.

[0177] When the average precision is 50-95 (B), the target detection network provided by this embodiment is improved by 0.0299 compared with Yolov11, which can improve the detection rate and completeness.

[0178] In the feature matching stage, based on the improved LightGlue algorithm, deep optimization is performed to target the low-texture and high-noise characteristics of OCT images.

[0179] Follow these steps:

[0180] Preprocessing is performed on the target area to output the preprocessing results. The preprocessing is used to suppress speckle noise and balance the contrast between tissue layers in the local OCT area defined by the target detection frame. For example, direction-adaptive Gaussian filtering is used to set the long-axis filter kernel along the retinal layer direction and the short-axis filter kernel in the vertical direction, and linear normalization processing based on the retinal pigment epithelium reflection intensity is used to eliminate blood flow artifact interference.

[0181] In some embodiments, filtering enhancement is performed on the target area to output an enhancement result, in which the contrast of the tubular structure is higher than the contrast of the tubular structure in the target area; local contrast normalization is performed on the enhancement result to output a preprocessing result.

[0182] During preprocessing, the Frangi filter enhances the contrast of microvascular structures. This filter uses multiscale Hessian matrix analysis to highlight tubular structures and suppress background noise, making subtle features such as retinal blood vessels and skin pores easier to discern. Local contrast normalization is then performed, and the image is processed in blocks using a 32×32 pixel sliding window to eliminate brightness fluctuations caused by uneven lighting or differences in scanning angles.

[0183] The preprocessing results are passed through a feature extraction network to extract feature points of multiple target areas. In one implementation, the feature extraction network adopts a residual architecture design, and the first layer presets the direction-constrained convolution kernel of the retinal blood vessel direction parameters to capture anatomical structure features. Combined with the gradient direction consistency in the spatial channel dual attention mechanism, the enhancement module strengthens the tissue boundary response, and enhances the pathological features through the channel attention weight, and finally outputs the feature descriptor.

[0184] In another implementation, the feature extraction network uses a weight-sharing, two-branch CNN architecture, leveraging a dynamic convolution kernel adjustment mechanism for feature extraction. The input image, or preprocessed image, undergoes five levels of downsampling to produce a multi-scale feature map. Each level consists of three dynamic convolution blocks, with the final layer outputting a 128-dimensional feature vector. A feature importance scoring mechanism is introduced to automatically adjust feature dimensions based on scene complexity, using 64-dimensional features for simple areas and 256-dimensional features for complex areas.

[0185] In some embodiments, the feature extraction network includes multiple first layers; the preprocessing results are sequentially input into the multiple first layers, passed through the dynamic convolution blocks in the first layers, and downsampling operations are performed to output layer results; and target feature points are extracted from the layer results.

[0186] The first level constructs multi-level feature abstraction in a serial manner. The first level receives the original input and the last level outputs high-level semantic features.

[0187] The first level includes downsampling operations and multiple dynamic convolution blocks. The dynamic convolution blocks consist of a configurable set of convolution kernels and a selector switch that automatically adapts the optimal convolution kernel based on the texture complexity of the input features. In one implementation, the dynamic convolution blocks contain three sets of preset kernels: standard convolution kernels for processing regular textures, dilated convolution kernels for capturing large-scale structures, and deep convolution kernels for optimizing processing of low-texture areas.

[0188] In another implementation, the dynamic convolution block acts as a convolution type switcher, switching to deformable convolution when retinal boundary features are detected and to reduced convolution when homogeneous vitreous regions are detected. Both implementations achieve real-time switching via a feature analysis module, which consists of a lightweight neural network that evaluates the spatial distribution of the current feature map and outputs a kernel selection signal.

[0189] Downsampling is the resolution conversion between layers, and spatial compression is achieved using strided convolution or pooling. In retinal vascular processing, strided convolution reduces the feature map size by setting a fixed step size while increasing the number of channels; pooling uses an adaptive window to preserve local extremes in microaneurysm regions. Both methods trade space for depth, ensuring the correspondence between key anatomical features at different layers.

[0190] Specifically, the preprocessing result is input into the plurality of dynamic convolution blocks and a downsampling operation is performed to output a first layer result;

[0191] Inputting the first layer result into a plurality of the dynamic convolution blocks and performing a downsampling operation to output a second layer result, where the number of channels of the second layer result is greater than the number of channels of the first layer result;

[0192] Inputting the second layer result into a plurality of the dynamic convolution blocks and performing a downsampling operation to output a third layer result, where the number of channels of the third layer result is greater than the number of channels of the second layer result;

[0193] Inputting the third layer result into a plurality of the dynamic convolution blocks and performing a downsampling operation to output a fourth layer result, where the number of channels of the fourth layer result is greater than the number of channels of the third layer result;

[0194] Inputting the fourth layer result into a plurality of the dynamic convolution blocks and performing a downsampling operation to output a fifth layer result, wherein the number of channels of the fifth layer result is greater than the number of channels of the fourth layer result;

[0195] Extract target feature points from the fifth layer result.

[0196] The first, second, third, fourth, and fifth layer results are the outputs of each first level, comprising a dual representation of spatial and channel feature maps. The spatial feature map records the relative position of each retinal layer, while the channel feature map encodes the topological properties of vascular branches. As the levels progress, spatial resolution decreases while channel dimensions increase. Higher-level results focus on macrostructural associations, while lower-level results preserve microscopic details.

[0197] Exemplarily, the preprocessed OCT image is fed into the first layer as a single channel. This layer employs two dynamic convolution blocks. The first performs a standard 3×3 convolution to capture the underlying edge texture. The second switches its operating mode based on the output signal from the feature analysis module. This means that when macular vessels are identified, dilated convolution is used to expand the receptive field, while when the vitreous region is marked, deep convolution is used to reduce computational effort. After convolution, a maximum pooling downsampling with a stride of two is performed, resulting in a feature map with a size reduced to half that of the input. This feature map incorporates the striped texture characteristics of the retinal nerve fiber layer while preserving the intensity gradient distribution of the vessel walls.

[0198] The processed results are then transferred to the second layer, which consists of three dynamic convolution blocks. At this stage, the feature analysis module detects heterogeneous features in the lesion area, triggering a switch in convolution type: deformable convolution is used to adapt to irregular boundaries in edema areas, while anti-noise convolution kernels are configured for exudate areas. After the three layers of convolution, strided convolution downsampling is performed, further halving the spatial resolution of the output feature map while doubling the number of channels. At this point, the feature map can distinguish the boundary contours between the inner plexiform layer and the outer nuclear layer.

[0199] After receiving the compressed feature map at the third level, the dynamic convolution block uses a special topological convolution kernel for the capillary network, enhancing bifurcation point features through an eight-neighborhood connection pattern. Downsampling uses mean pooling to preserve light intensity distribution information. During the fourth level of processing, the feature analysis module identifies drusen structures and activates the multi-scale fusion convolution block to simultaneously extract the calcification and edge features at the center of the lesion. The final level cancels downsampling and outputs a high-dimensional feature vector set containing spatial position encoding. Non-maximum suppression is performed on the feature map to locate key points.

[0200] Taking an OCT image of diabetic macular edema as an example, when the preprocessed image is input to the first layer, the dynamic convolution block detects weak vascular signals in the fovea and automatically switches to a high-sensitivity small-kernel convolution to enhance the response. After five layers of processing, the fifth layer outputs a feature map with key points located, including microaneurysm centers, venous intersections, and retinal layer boundaries. Each feature point carries a hierarchical encoding of its location within the vascular tree.

[0201] Based on the feature points output by the feature extraction network, graph attention matching is used to process these feature points to obtain node features. The extracted feature points are input into the graph attention matching network to construct a graph structure. The nodes store the feature point coordinates and descriptors. The edge weights are calculated using a four-head attention mechanism that integrates cosine similarity and relative position encoding.

[0202] Then, a bidirectional nearest neighbor search is performed based on the node features to generate candidate matching pairs, and geometric verification is performed on the candidate matching pairs to output confidence scores of the candidate matching pairs; the candidate pairs are geometrically verified, for example, 500 rounds of RANSAC iterations are performed to calculate the homography matrix hypothesis, and a confidence score in the range of 0-1 is calculated based on the proportion of inliers. If the confidence score is greater than a scoring threshold, in this embodiment, the scoring threshold is 0.8, the candidate matching pairs corresponding to the confidence score greater than the scoring threshold are determined as matching point pairs, that is, a list of feature point pairs corresponding to the coordinates.

[0203] For example, a fully connected graph with 82 nodes was constructed from the feature point set, with retinal layer coordinates encoded during initialization. In the first iteration, four attention heads performed separate calculations: head 1 focused on spatial distances, head 2 enforced vascular topology constraints, head 3 associated intra-layer structures, and head 4 suppressed cross-layer matching. After updating through a three-layer MLP (Multi-Layer Perceptron) network, low-weight edges were pruned. The similarity matrix output from the third iteration is shown. The bidirectional search phase identified 73 candidate pairs, 12 of which were labeled suspicious due to cross-layer errors. Geometric verification was performed using RANSAC calculations. When sampling four venous bifurcations, the reprojection error of the fitted homography matrix was 0.3 pixels; when sampling cross-layer points, the error reached 5.2 pixels. After 500 iterations, 68 inliers were identified with a confidence score of 0.89. All matched points accurately corresponded to vascular bifurcations, with microaneurysm morphological errors less than 1 μm, meeting clinical measurement accuracy standards.

[0204] After obtaining the matching point pair, execute steps S200-S400 to obtain the following Figure 7 The method detects the fundus OCT image, intercepts the detected OCT position, extracts feature points from multiple consecutive OCT images, and then stitches them together to improve stitching accuracy and speed.

[0205] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without expending creative work shall fall within the scope of protection of this application.

Claims

1. An OCT image stitching method, characterized in that: include: Acquire an image pair and a pair of matching points corresponding to the image pair, wherein the image pair is adjacent OCT images; Calculating a homography matrix based on the matching point pair to align the image pair, and outputting a preliminary aligned image pair; Calculating the pixel-level displacement field of the preliminary aligned image by a multi-scale optical flow algorithm to determine the deformation of the local area; Constructing a fusion weight map, wherein the fusion weight map includes weight values ​​and region types of pixel positions; Based on the deformation amount, dividing the fusion weight map to output a high confidence area, a deformation transition area, and a highly deformed area; According to the weight value and the region type, fusion stitching is performed on the pixel positions to output a panoramic OCT image.

2. The OCT image stitching method according to claim 1, characterized in that: The acquiring of an image pair and a matching point pair corresponding to the image pair comprises: Acquire an image pair, the image pair comprising a plurality of images; Performing target detection on the image pair to locate the target area, wherein the target detection is performed by a target detection model; A feature matching algorithm is used to perform feature matching on the target area to output matching point pairs, where the matching point pairs are feature matching point pairs between adjacent images.

3. The OCT image stitching method according to claim 1, characterized in that: The calculating a homography matrix based on the matching point pair to align the image pair and outputting a preliminary aligned image pair includes: Eliminate mismatched points in the matching point pairs through outliers to output target matching point pairs; Based on the target matching point pairs, a transformation matrix is ​​calculated using a geometric estimation algorithm; The transformation matrix is ​​applied to align the image pair to generate a preliminary aligned image.

4. The OCT image stitching method according to claim 1, wherein: The weight value of the high confidence zone is a first value, the weight value of the deformation transition zone is a second value, and the weight value of the high deformation zone is a third value. The first value is greater than the second value, and the second value is greater than the third value.

5. The OCT image stitching method according to claim 1, characterized in that: The performing fusion stitching on the pixel positions according to the weight values ​​and the region types to output a panoramic OCT image includes: Performing maximum fusion on the high confidence area, performing weighted hybrid fusion on the deformation transition area, and performing elastic deformation compensation fusion on the pixels in the highly deformed area to output a fusion result; In the fusion result, locating the seam area; Acquire the similarity of the seam area, and if the similarity is less than a similarity threshold, update the fusion weight map of the seam area; If the similarity is greater than or equal to the similarity threshold, a panoramic OCT image is output.

6. The OCT image stitching method according to claim 2, characterized in that: The target detection model includes a first level, a second level, a third level, a fourth level, a fifth level, and a sixth level; The first layer, the second layer, the third layer, and the fourth layer have the same structure and all include a star aggregation module, and the output feature map sizes of the first layer, the second layer, the third layer, and the fourth layer are different; The method further comprises: Performing double upsampling on the feature map output by the sixth level to obtain a first sampling result; Perform channel splicing on the first sampling result and the feature map output by the third level to output a first fused feature; Perform channel concatenation on the feature map output by the second layer, the first fused feature, and the feature map output by the first cross-layer aggregation convolution module to output a second fused feature; Inputting the second fused features into a first cross-layer aggregation convolution module to output a first feature map; Perform feature splicing on the second fused feature and the first feature map to output a third fused feature.

7. The OCT image stitching method according to claim 6, characterized in that: The performing target detection on the image pair to locate the target area includes: Inputting the third fused feature into a second cross-layer aggregation convolution module to output a second feature map; After passing the first feature map through the second feature extraction module, the first feature map is concatenated with the second feature map to output a third feature map; Inputting the third feature map into a third cross-layer aggregation convolution module to obtain a fourth feature map; Inputting the fourth feature map into a third feature extraction module, and concatenating the feature map output by the second feature extraction module and the sixth layer to output a fifth feature map; Inputting the fifth feature map into a fourth cross-layer aggregation convolution module to obtain a sixth feature map; The second feature map is input to a first detection head, the fourth feature map is input to a second detection head, and the sixth feature map is input to a third detection head to output a target area.

8. The OCT image stitching method according to claim 6, characterized in that: The star-shaped aggregation module includes a central node and at least two radial branches; Before performing double upsampling on the feature map output by the sixth level to obtain a first sampling result, the method further includes: Inputting the image pair after the first feature extraction module into the central node to output an intermediate feature map; Inputting the intermediate feature map into a first radial branch to output a first radial feature map, and inputting the intermediate feature map into a second radial branch to output a second radial feature map; The first radial feature map and the second radial feature map are merged to generate a merged result, and the merged result is input into the sixth layer.

9. The OCT image stitching method according to claim 2, characterized in that: The performing feature matching on the target area using a feature matching algorithm to output matching point pairs includes: performing preprocessing on the target area to output a preprocessing result; Passing the preprocessing results through a feature extraction network to extract feature points of multiple target areas; Processing the feature points through graph attention matching to obtain node features; performing a bidirectional nearest neighbor search based on the node features to generate candidate matching pairs; performing geometric verification on the candidate matching pairs to output a confidence score for the candidate matching pairs; If the confidence score is greater than the score threshold, the candidate matching pair corresponding to the confidence score greater than the score threshold is determined as a matching point pair.

10. The OCT image stitching method according to claim 9, characterized in that: The feature extraction network includes a plurality of first layers; the first layers include a downsampling operation and a plurality of dynamic convolution blocks; The preprocessing result is passed through a feature extraction network to extract feature points of multiple target areas, including: Inputting the preprocessing results into a plurality of the first levels in sequence, passing through the dynamic convolution blocks in the first levels, and performing a downsampling operation to output level results; Extract target feature points from the hierarchical results.

Images