OCT image splicing method

Abstract

The invention relates to the technical field of image processing, and provides an OCT (Optical Coherence Tomography) image splicing method, which comprises the following steps: acquiring image pairs and matching point pairs corresponding to the image pairs, the image pairs being adjacent OCT images; calculating a homography matrix based on the matching point pairs so as to align the image pairs, and outputting preliminary aligned image pairs; constructing a fusion weight map, wherein the fusion weight map comprises a weight value of a pixel position and a region type; and according to the weight value and the region type, carrying out fusion splicing on the pixel positions so as to output a panoramic OCT image. According to the method, partition fusion is achieved by fusing the weight map, the problems of tissue dislocation and artifacts in OCT splicing are solved, and then the problem that the splicing effect of OCT image splicing is poor is solved.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to an OCT image stitching method. Background Art

[0002] Optical Coherence Tomography (OCT), as a non-invasive imaging technology, is applied to the microscopic structure observation of biological tissues such as the retina and skin. In clinical diagnosis, multiple local OCT images need to be stitched into a panoramic view to completely present the vascular network, tissue structure, and their pathological features. The stitching result needs to maintain the continuity of fine structures such as intact tubular microvessels, the gray-scale distribution of different scanning regions needs to be uniform and unified, and the anatomical layer boundaries need to be seamlessly connected to avoid misalignment artifacts.

[0003] During the OCT image stitching process, a linear fusion strategy is adopted to perform weighted smoothing on the overlapping region according to the pixel spatial position, without considering the unique layered signal intensity distribution law of OCT images, resulting in fusion distortion of deep tissue structures, manifested as problems such as blurred blood vessel edges and interlayer misalignment artifacts. In addition, the stitching process lacks targeted optimization of the pathological features of the target region, resulting in the loss or distortion of key diagnostic information.

[0004] In summary, the stitching effect of OCT image stitching is poor. Summary of the Invention

[0005] This application provides an OCT image stitching method to solve the problem of poor stitching effect of OCT image stitching.

[0006] This application provides an OCT image stitching method, including: Obtain an image pair and a pair of matching points corresponding to the image pair, where the image pair is an adjacent OCT image; Calculate a homography matrix based on the pair of matching points to align the image pair and output a preliminarily aligned image pair; Construct a fusion weight map, where the fusion weight map includes weight values and region types of pixel positions; Perform fusion stitching on pixel positions according to the weight values and region types to output a panoramic OCT image.

[0007] In some feasible embodiments, the obtaining an image pair and a pair of matching points corresponding to the image pair includes: Obtain an image pair, where the image pair includes multiple images; Perform object detection on the image pair to locate the target region, where the object detection is performed through an object detection model; Perform feature matching on the target region using a feature matching algorithm to output a pair of matching points, where the pair of matching points is a pair of feature matching points between adjacent images.

[0008] Enhance the reliability of matching points through hierarchical target localization, avoid false matching caused by background interference, and enhance the robustness of stitching.

[0009] In some feasible embodiments, calculating the homography matrix based on the matching point pairs to align the image pairs and outputting the preliminarily aligned image pairs includes: Eliminating the false matching points in the matching point pairs through outlier rejection to output the target matching point pairs; Calculating the transformation matrix based on the target matching point pairs through a geometric estimation algorithm; Applying the transformation matrix to align the image pairs to generate the preliminarily aligned images.

[0010] Outlier rejection can ensure the accuracy of the geometric transformation matrix, optimize the image alignment accuracy, and reduce the accumulation of alignment deviations.

[0011] In some feasible embodiments, constructing the fusion weight map includes: Calculating the pixel-level displacement field of the preliminarily aligned images through a multi-scale optical flow algorithm to determine the deformation amount of the local area; Constructing the fusion weight map; Dividing the fusion weight map based on the deformation amount to output a high-confidence area, a deformation transition area, and a highly deformed area. The weight value of the high-confidence area is the first value, the weight value of the deformation transition area is the second value, and the weight value of the highly deformed area is the third value. The first value is greater than the second value, and the second value is greater than the third value.

[0012] The multi-scale optical flow quantifies the local deformation, and the partition weight strategy adaptively fuses different deformation regions to suppress the seams.

[0013] In some feasible embodiments, performing fusion stitching on the pixel positions according to the weight values and region types to output the panoramic OCT image includes: Performing maximum value fusion on the high-confidence area, weighted hybrid fusion on the deformation transition area, and elastic deformation compensation fusion on the pixels in the highly deformed area to output the fusion result; Locating the stitching seam region in the fusion result; Obtaining the similarity of the stitching seam region. If the similarity is less than the similarity threshold, updating the fusion weight map of the stitching seam region; If the similarity is greater than or equal to the similarity threshold, outputting the panoramic OCT image.

[0014] Region-based fusion can maximize the retention of effective information and improve the image coherence through the stitching seam self-check mechanism.

[0015] In some feasible embodiments, the object detection model includes a first layer, a second layer, a third layer, a fourth layer, a fifth layer, and a sixth layer; The first layer, the second layer, the third layer, and the fourth layer have the same structure, each including a star-shaped aggregation module, and the output feature map sizes of the first layer, the second layer, the third layer, and the fourth layer are different; The method further includes: Performing two-fold upsampling on the feature map output by the sixth layer to obtain a first sampling result; Performing channel splicing on the first sampling result and the feature map output by the third layer to output a first fusion feature; Performing channel splicing on the feature map output by the second layer, the first fusion feature, and the feature map output by the first cross-layer aggregation convolution module to output a second fusion feature; Inputting the second fusion feature into the first cross-layer aggregation convolution module to output a first feature map; Performing feature splicing on the second fusion feature and the first feature map to output a third fusion feature.

[0016] Integrating multi-scale features through cross-layer aggregation convolution to enhance the detection ability of small targets such as the retinal laminar structure.

[0017] In some feasible embodiments, the object detection of the image pair to locate the target area includes: Inputting the third fusion feature into the second cross-layer aggregation convolution module to output a second feature map; After passing the first feature map through the second feature extraction module, splicing it with the second feature map to output a third feature map; Inputting the third feature map into the third cross-layer aggregation convolution module to obtain a fourth feature map; Inputting the fourth feature map into the third feature extraction module, and splicing the second feature extraction module with the feature map output by the sixth layer to output a fifth feature map; Inputting the fifth feature map into the fourth cross-layer aggregation convolution module to obtain a sixth feature map; Inputting the second feature map into the first detection head, inputting the fourth feature map into the second detection head, and inputting the sixth feature map into the third detection head to output the target area. The multi-detection heads cooperate to optimize the target positioning accuracy and improve the matching basic quality of key areas such as blood vessel branches.

[0018] In some feasible embodiments, the star-shaped aggregation module includes a central node and at least two radial branches; Before performing the two - fold upsampling on the feature map output from the sixth layer to obtain the first sampling result, the following steps are further included: Input the image pair that has passed through the first feature extraction module into the central node to output an intermediate feature map; Input the intermediate feature map into the first radial branch to output a first radial feature map, and input the intermediate feature map into the second radial branch to output a second radial feature map; Merge the first radial feature map and the second radial feature map to generate a merged result, and input the merged result into the sixth layer.

[0019] In some feasible embodiments, performing feature matching on the target region using a feature matching algorithm to output a pair of matching points includes: Perform pre - processing on the target region to output a pre - processing result; Pass the pre - processing result through a feature extraction network to extract feature points of multiple target regions; Perform processing on the feature points through graph attention matching to obtain node features; Perform bidirectional nearest neighbor search based on the node features to generate candidate matching pairs; Perform geometric verification on the candidate matching pairs to output the confidence scores of the candidate matching pairs; If the confidence score is greater than the score threshold, determine the candidate matching pairs corresponding to the confidence scores greater than the score threshold as the matching point pairs.

[0020] Graph attention enhances feature relevance, and bidirectional search combined with geometric verification ensures the spatial consistency of the matching point pairs.

[0021] In some feasible embodiments, the feature extraction network includes multiple first levels; the first level includes a downsampling operation and multiple dynamic convolution blocks; The step of passing the pre - processing result through the feature extraction network to extract feature points of multiple target regions includes: Input the pre - processing result into multiple first levels in sequence, pass through the dynamic convolution blocks in the first level, and perform the downsampling operation to output a hierarchical result; Extract the target feature points from the hierarchical result.

[0022] The dynamic convolution blocks adaptively organize morphological changes, and hierarchical extraction of feature points improves the generalization ability of the matching algorithm.

[0023] As can be seen from the above technical solutions, the present application provides an OCT image stitching method, including: obtaining an image pair and a corresponding pair of matching points for the image pair, where the image pair is adjacent OCT images; calculating a homography matrix based on the pair of matching points to align the image pair and output a preliminary aligned image pair; constructing a fusion weight map, where the fusion weight map includes weight values of pixel positions and region types; performing fusion stitching on pixel positions according to the weight values and region types to output a panoramic OCT image. The method realizes partition fusion through the fusion weight map, solves the problems of tissue misalignment and artifacts in OCT stitching, and further solves the problem of poor stitching effect in OCT image stitching. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] To more clearly illustrate the technical solutions of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0025] Figure 1 It is a schematic flowchart of the OCT image stitching method provided by the embodiment of the present application; Figure 2 It is a schematic diagram of the nth frame image in the image pair provided by the embodiment of the present application; Figure 3 It is a schematic diagram of the (n + 1)th frame image in the image pair provided by the embodiment of the present application; Figure 4 It is a schematic diagram of the target detection and positioning target area provided by the embodiment of the present application; Figure 5 It is a schematic diagram of the structure of the target detection model provided by the embodiment of the present application; Figure 6 It is a schematic diagram of the structure of the star-shaped aggregation module provided by the embodiment of the present application; Figure 7 It is a schematic diagram of the panoramic OCT image provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The embodiments will be described in detail below, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following examples do not represent all embodiments consistent with the present application.

[0027] The signal intensity of tissues at different depths in OCT images shows exponential decay. In the linear fusion strategy, when assigning weights according to spatial distance, weak signals in deep layers are overwhelmed by strong signals in the weighted average, and the interlayer intensity transition caused by the decay curve is forced to be smoothed. Traditional methods assume that the tissue topology in the overlapping area of adjacent images is continuous, but OCT images include areas such as vascular networks and pathology. The smoothing operation of linear fusion will cause blurring of microvascular bifurcation points, that is, key stitching landmark points, and weaken the steep transition between the edema area and the non-lesion area, resulting in errors in the information in OCT images and thus poor stitching effects of OCT image stitching.

[0028] To solve the above problems, an embodiment of the present application provides an OCT image stitching method, as Figure 1 shown, including the following steps: S100: Obtain an image pair and a pair of matching points corresponding to the image pair.

[0029] The image pair is adjacent OCT images. Specifically, the image pair is two OCT tomographic scan images adjacent in spatial position, which can be obtained by continuously moving and scanning along the tissue surface by an OCT device. For example, a sequence of retinal tomographic images obtained by continuous scanning with an OCT device.

[0030] The pair of matching points is a set of feature point coordinates with the same anatomical structure extracted from adjacent OCT images, which is used to establish the spatial mapping relationship between images.

[0031] In some embodiments, the Scale-Invariant Feature Transform (SIFT) algorithm is used to detect key points in the image. For example, the intersection points of retinal stratification boundaries and vascular bifurcation points, and a set of matching point pairs is generated through descriptor matching. Among them, the pair of matching points satisfies the bidirectional consistency check, that is, when point A in the first image matches point B in the second image, the reverse matching result of point B must also be point A to exclude artifact interference.

[0032] S200: Calculate the homography matrix based on the pair of matching points to align the image pair and output the preliminarily aligned image pair.

[0033] The homography matrix is a 3×3 matrix describing the perspective transformation relationship between two images. By solving the coordinate correspondence relationship of the pair of matching points, the Random Sample Consensus (RANSAC) algorithm is used to optimize the matrix parameters.

[0034] Exemplarily, substitute the pair of matching points into the homography equation to construct an overdetermined system of equations, iteratively randomly sample to calculate the initial matrix, eliminate the outliers with projection errors greater than the threshold, and use the solution with the most inliers as the optimized matrix. After this transformation, the spatial coordinate systems of the two images are aligned, and the preliminarily aligned image pair corresponding to the pixel positions is output.

[0035] S3000: Construct a fusion weight map.

[0036] The fusion weight map is a two-dimensional matrix with the same size as the image, containing the weight value at each pixel position and the identification of the region type. The weight value is calculated by the distance from the pixel to the boundary of the overlapping region, and the closer to the boundary, the smaller the weight.

[0037] The region type is divided based on the OCT image hierarchical structure. Exemplarily, it is divided into a superficial high-signal region, such as the nerve fiber layer, a middle homogeneous region, such as the outer nuclear layer, and a deep low signal-to-noise ratio region, such as the choroid layer. A weight attenuation curve is set for each type of region. For example, an exponential attenuation curve is adopted for the deep low signal-to-noise ratio region, and a linear attenuation curve is adopted for the superficial high-signal region to adapt to the signal attenuation characteristics of different tissues.

[0038] S400: Perform fusion stitching on the pixel positions according to the weight value and the region type to output a panoramic OCT image.

[0039] In the overlapping region, perform weighted averaging on the same pixel position of the two images, and the output pixel value is the sum of the product of the pixel value of image A and weight A and the product of the pixel value of image B and weight B, where the sum of weight A and weight B is 1.

[0040] For the superficial high-signal region, the weight curve is smooth to ensure texture continuity; for the deep region, when a vascular structure is detected, the weight of non-vascular tissues is dynamically reduced based on the morphological segmentation result to reduce the interference of blood flow signals. The non-overlapping region directly retains the original pixel value to generate a seamlessly connected panoramic OCT image.

[0041] The method provided in this embodiment adopts weight attenuation for the deep low signal-to-noise ratio region, avoids the homogenization of vascular signals caused by linear fusion, and improves the sharpness of the choroidal neovascularization edge. In the homography matrix calculation stage, through feature point screening, abnormal matching points perpendicular to the stratification direction are excluded, so that the image alignment process conforms to the retinal layered structure. Through weight assignment, the panoramic image maintains high accuracy in both deep and shallow tissues.

[0042] Before obtaining the matching point pairs, perform object detection on the image pair through an object detection model, and then perform feature matching through a feature matching algorithm to output the matching point pairs.

[0043] In the object detection algorithm, based on the preset Anchor mechanism (Anchor Box Mechanism) with a fixed size, it is difficult to dynamically adapt to the multi-scale characteristics of the tiny lesion regions in OCT images, resulting in prominent problems of missed detection and false detection in low-contrast and high-noise regions. In addition, the detection network is not optimized for the layered structure characteristics of OCT images, resulting in limited positioning accuracy of the target region and affecting the accuracy of stitching.

[0044] In some embodiments, matching point pairs are obtained through the following steps: S210: Obtain an image pair.

[0045] In a retinal imaging scenario, an image pair is continuously acquired by a fundus scanner along the horizontal or vertical direction, and its overlapping area contains common anatomical structures. For example, in retinal scanning, the overlapping part of adjacent images needs to include the same vascular branch or the cross-section structure of the same optic nerve fiber layer. Among them, as Figure 2 , Figure 3 shown, the nth frame and the (n + 1)th frame of images form an image pair.

[0046] There is a partial overlapping area between adjacent images, and, due to possible hand shaking or eye movement, the images may be shifted. During continuous shooting, for example, when taking ten pictures, these ten pictures need to be superimposed and stitched, but due to the displacement, they cannot be directly superimposed and stitched. Therefore, feature matching needs to be performed.

[0047] There are multiple image pairs, that is, all consecutive overlapping image combinations in the acquired image sequence are processed. For example, frames 1 - 2, frames 2 - 3 until frames N - 1 - N, to achieve panoramic stitching of the entire scanning path.

[0048] As Figure 4 shown, however, because the original image is large in size and the superimposing time is long, target detection needs to be performed first to locate the OCT foreground area, and stitching is performed for the areas with signals, which can shorten the stitching time.

[0049] S220: Perform target detection on the image pair to locate the target area.

[0050] Target detection can be performed through a target detection model, and the target detection model outputs the boundary position of the target area and the structure type label. The target area is defined by the target detection model. Among them, the target area is a local image range. It can be understood that different image target areas are different, but the target area includes parts with high feature matching value. The high feature matching value can be determined by proportion and / or intersection, density, lesion area. For example, anatomical significant positions such as the intersection area of optic nerve fiber bundles, the vascular dense area of the fovea centralis, and the choroidal lesion shadow area. Or, the OCT foreground area, and stitching is performed for the areas with signals.

[0051] Exemplarily, the target area is the minimum circumscribed rectangle range containing key anatomical features, and the size is automatically adjusted according to the detected organizational structure characteristics. For example, the macular area accounts for about 10% - 15% of the total image area, and the area around the optic disc accounts for 25% - 30%.

[0052] In one implementation, the target detection model is a region recognition module based on a convolutional neural network, which is used to locate key anatomical structure regions in an image. The target detection model is trained through a pre-trained retinal OCT image dataset, with a single-frame OCT image as the input and the coordinates of bounding boxes as the output.

[0053] For example, in cases of diabetic macular edema, the model can accurately locate the coordinates of cystoid edema regions, the fovea centralis, and microaneurysms.

[0054] In another implementation, the target detection model is a lightweight YOLO v4-tiny framework optimized for OCT medical images. Among them, the input layer is optimized by adding a multi-scale reflection intensity normalization module, replacing standard convolutions with depthwise separable convolutions to reduce the number of parameters, and adding a cross-layer retinal feature fusion module. For example, skip connections are added at the output ends of the 3rd, 5th, and 7th layers of CSPDarknet53 to fuse retinal hierarchical features at different scales.

[0055] It is also possible to use a Transformer-Encoder hybrid architecture to extract multi-scale local features using ConvNeXt-v2 in the feature extraction layer. In the relationship modeling layer, a retinal topology-aware Transformer module is added to enhance anatomical rationality through preset inter-layer connection constraints, and a heat map and boundary vectors are output to solve the problem of blurred contours of tiny lesions.

[0056] Exemplarily, the two images in the image pair are defined as image A and image B, and are respectively input into the target detection model. The model adopts a U-Net architecture, and its convolutional layer optimizes the reflection intensity response parameters for OCT images.

[0057] The model outputs a structural feature area and a pathological feature area. Among them, the structural feature area includes, for example, the inner limiting membrane reflection band and the Bruch membrane edge, and the pathological feature area includes, for example, the subretinal fluid area and the neovascular plexus. Exemplarily, a cystoid edema area with a diameter of 200 μm is detected at the fovea centralis in image B, and the model will output the bounding box coordinates [(x1, y1), (x2, y2)] of this area.

[0058] Perform matching detection on the same anatomical position in image A to ensure the biological correspondence of the target area. For example, the cystoid edema area in image B needs to detect a continuous structure at the corresponding position in image A.

[0059] S230: Use a feature matching algorithm to perform feature matching on the target area to output a pair of matching points.

[0060] The feature matching algorithm is used for the local area delimited by the target detection, that is, the target area, rather than the entire image, and hierarchical matching is adopted. In one implementation, ORB (Oriented FAST and Rotated BRIEF, keypoint detector and descriptor) feature points with rotation invariance are extracted within the target area, the matching direction is constrained according to the OCT tomography characteristics, and point pairs are screened through the Hamming distance to achieve feature correspondence of the pathological area.

[0061] In other implementations, a feature extraction network can also be used, adopting a Siamese dual-path ResNet-18 architecture, sharing weights, improving the third-layer convolution, and using dilated convolution to enhance the capture of OCT layered textures, outputting a 128-dimensional feature vector, and then outputting matching point pairs.

[0062] After generating the matching point pairs, a homography matrix is calculated based on the matching point pairs to align the image pairs, and a preliminary aligned image pair is output. In some embodiments, the mismatched points in the matching point pairs are removed by outliers, so as to output target matching point pairs; based on the target matching point pairs, a transformation matrix is calculated through a geometric estimation algorithm; the transformation matrix is applied to align the image pairs to generate a preliminary aligned image.

[0063] After generating the matching point pairs, some of the matching point pairs do not conform to the displacement law of the real anatomical structure. For example, in OCT image processing, vitreous reflection artifacts or texture repetition areas of subcutaneous adipose tissue. Therefore, the mismatched points are removed by discrete points first. Outlier removal identifies and removes mismatched point pairs through statistical analysis methods, that is, performs two-way consistency verification and displacement vector clustering analysis on the matching point pairs, discards the outliers that deviate from the main displacement model, and outputs the target matching point pairs. For example, when splicing retinal blood vessels, the matching points perpendicular to the stratification direction are removed.

[0064] It can be understood that the target matching point pairs are a set of spatially corresponding points after outlier screening. The target matching point pairs meet the preset conditions, and the preset conditions can be that the point pairs meet the anatomical structure consistency rule and / or the overall distribution conforms to the basic geometric constraints.

[0065] Then, a geometric estimation algorithm is executed based on the target matching points. The geometric estimation algorithm is used to establish a spatial mapping model of adjacent images and solve the transformation matrix parameters according to the matching point coordinates. In this embodiment, the global affine transformation matrix is estimated through the RANSAC estimation algorithm to preliminarily align the general structure of the image pairs. The algorithm calculates the candidate transformation matrix by iteratively randomly sampling point pairs, and selects the matrix that satisfies the most inliers, that is, the matrix with a projection error less than the preset threshold as the optimal solution.

[0066] Specifically, the transformation matrix is a 3×3 homography matrix. The matrix elements are divided into three parts. The upper-left 2×2 sub-matrix represents rotation and scaling; the upper-right 2×1 vector represents translation; the bottom row controls the projective transformation. For example, in the retinal OCT scenario, the matrix form is simplified to an affine transformation to adapt to the approximate planar structure of the eyeball.

[0067] After obtaining the global affine transformation matrix, align the image pair. The image pair can be the pre-processed and aligned image pair, but there are local non-rigid displacements, and the global coordinate system needs to be unified through the transformation matrix. After mapping through the transformation matrix, an image set is obtained, that is, the preliminarily aligned image. The alignment process uses bilinear interpolation to calculate pixel values to maintain the continuity of tissue texture.

[0068] Exemplarily, during the OCT image analysis of diabetic macular edema, the initially obtained adjacent frame image pair contains 38 pairs of matching points. When performing direction constraint filtering, 8 abnormal displacement points perpendicular to the retinal stratification are removed; the distance constraint further filters out 3 matching points that exceed the choroidal displacement limit. The remaining 27 matching points enter the RANSAC optimization process. In the nerve fiber layer region, a strict constraint is adopted with a 1.5-pixel error threshold, and in the choroid layer, it is relaxed to 2.5 pixels. After 5 iterations, it converges to obtain an optimal homography matrix containing 21 inliers.

[0069] During the matrix solution stage, cystoid edema lesions in the fovea centralis region of the macula are identified and excluded from parameter calculation. After application of the transformation, the alignment accuracy of the two images in the optic nerve fiber layer is 2 microns, and there is no misalignment at the boundary splicing of the edema region.

[0070] After roughly aligning the general structures of the preliminarily aligned image pair, use the dense optical flow algorithm to calculate the pixel-level displacement field and quantify the local deformation intensity. In some embodiments, the multi-scale optical flow algorithm is used to calculate the pixel-level displacement field of the preliminarily aligned image to determine the deformation amount in the local region.

[0071] Among them, the multi-scale optical flow algorithm is based on the assumptions of brightness constancy and spatial continuity constraints. By establishing a multi-scale pyramid structure, the pixel motion vector is calculated. Specifically, the optical flow is calculated starting from the coarsest scale, and the result is used as the initial value of the next layer for optimization level by level. Finally, a displacement vector field is output, which characterizes the local deformation degree of the two images after alignment.

[0072] In OCT stitching, the object of optical flow analysis is the preliminarily aligned image pair, and the algorithm output is a displacement vector field. The vector direction represents the tissue movement direction, and the amplitude characterizes the deformation degree. The algorithm implementation includes the Horn-Schunck method based on the assumption of brightness constancy and the Lucas-Kanade method based on gradient calculation.

[0073] The pixel-level displacement field is a two-dimensional vector field, and each pixel position corresponds to a displacement vector (dx, dy). The magnitude of the displacement vector characterizes the amount of deformation at that position, and the direction represents the direction of tissue movement. In OCT images, the magnitude of the deformation characterizes the degree of tissue compression or scanning error. For example, the deformation in the fovea centralis area is smaller than that in the peripheral area. In the retina scanning scenario, the displacement field in the temporal retina area is a divergent vector pattern, and the nasal area is a convergent pattern, reflecting the characteristics of the eyeball surface.

[0074] Then, a fusion weight map is constructed. The fusion weight map is a two-dimensional matrix with the same size as the original image, and each element stores a weight value and a region type identifier. The weight value ranges from [0, 1] and is used for subsequent image fusion; the region type identifier is a discrete classification value, characterizing the fusion strategy partition to which the current pixel belongs.

[0075] The construction of the weight map is based on deformation analysis, and different deformation regions are assigned different fusion weights. That is, based on the amount of deformation, the fusion weight map is divided to output a high-confidence region, a deformation transition region, and a highly deformed region.

[0076] Among them, the high-confidence region is the region where the amount of deformation is less than the set threshold. The tissue alignment accuracy is high, and the maximum weight value, that is, the first value, is assigned; the deformation transition region is the region where the amount of deformation is in the middle range, with slight deformation, and the medium weight value, that is, the second value, is assigned; the highly deformed region is the region where the amount of deformation exceeds the reasonable range, with significant misalignment or artifacts, and the minimum weight value, that is, the third value, is assigned. The first value is greater than the second value, and the second value is greater than the third value.

[0077] Exemplarily, the high-confidence region refers to the region where the amount of deformation is less than the threshold. In retinal imaging, it includes flat regions such as the fovea centralis and the middle of the optic nerve fiber layer. The tissue structure in such regions is accurately aligned, and the anatomical features are clear. The deformation transition region is a banded region where the amount of deformation is in the middle range. For example, the transition zone from the retinal edge to the equator of the eyeball and the demarcation area where blood vessels cross the layered structure. There is a gentle deformation gradient in this region, and a smooth fusion transition mechanism is required. The highly deformed region is the region where the amount of deformation exceeds the upper limit value. For example, the edge of the equator of the eyeball and the nasal-temporal curvature at the optic disc. The anatomical structure in this region is compressed and deformed due to the projection of the eyeball surface, and special fusion compensation processing is required.

[0078] In some embodiments, maximum value fusion is performed on the high-confidence region. Maximum value fusion performs selective optimization on the pixel intensity values: at each corresponding position, the pixel values of the two images are compared, and the pixel with a higher signal intensity is selected as the output result. It is applicable to high signal-to-noise ratio regions and can be used to preserve the integrity of the tissue structure.

[0079] Perform weighted blending fusion on the deformation transition region. The weighted blending fusion linearly combines the pixels of two images based on a preset weight coefficient. The smoothing transition strategy can reduce the edge discontinuity caused by slight deformation.

[0080] Perform elastic deformation compensation fusion on the pixels in the highly deformed region. The elastic deformation compensation fusion establishes a local elastic deformation model: calculates the tissue stress distribution through finite element analysis, and reconstructs the pixel arrangement, which can compensate for the projection distortion of deep tissues.

[0081] After the above three regions are fused, an intermediate image after the pixel synthesis operation is output, that is, the fusion result. The image size of the fusion result covers the maximum circumscribed range of the original image pair. In the overlapping region, there are three-channel data, namely the synthesized pixel value, source marker, and fusion confidence. The non-overlapping region retains the original image data.

[0082] It can be understood that the stitching result is the stitching result of multiple images. At the stitching location of any two images, there is a stitching seam region. For example, in a retinal panoramic image, this region is distributed in a band along the scanning direction. Exemplarily, by detecting the angle between the image boundary extension line and the actual fusion boundary, for example, when the angle is greater than 30°, it is determined as the turning point of the stitching seam.

[0083] Obtain the similarity of the stitching seam region. If the similarity is less than the similarity threshold, update and optimize the original weight distribution of the stitching seam region. For example, expand the bandwidth of the transition region by 5 - 10 pixels and locally increase the weight by 0.2 at the structural break point.

[0084] Adopt a 5-pixel-wide linear gradient transition at the boundary between the overlapping region and the non-overlapping region to avoid obvious seams. The transition weight linearly decreases from 100% in the overlapping region to 0% in the non-overlapping region.

[0085] For the updated fusion weight map, re-divide the updated fusion weight map based on the deformation amount to output the updated high-confidence region, deformation transition region, and highly deformed region, and perform maximum fusion, weighted blending fusion, and elastic deformation compensation fusion until the similarity of the stitching seam region in the output fusion result is greater than or equal to the similarity threshold and stop.

[0086] Output the panoramic OCT image as the processed wide-field retinal structure imaging result. For example, by stitching multiple local OCT scan images with a scanning range of 6mm × 6mm, a composite image exceeding the physical limit of a single scan is generated, such as 12mm × 9mm, covering the retinal functional area from the fovea centralis to the optic disc edge.

[0087] During the object detection process, a general object detection algorithm can be used to locate the target regions in the image. This algorithm relies on pre-set fixed-size Anchor boxes to generate candidate regions and filters the targets for cropping based on a confidence threshold. However, such methods have limited detection capabilities for tiny target regions and are prone to missed detections and misjudgments, especially in low signal-to-noise ratio scenarios. In the feature matching step, a dense feature extraction algorithm is used to detect and describe feature points in the cropped image patches, filters the matching pairs by calculating the feature similarity, and combines a geometric consistency algorithm to eliminate outliers. Such methods are prone to false matches in regions with repeated textures or blurred boundaries, such as the junction of the retinal laminar structure, and have a high computational complexity, making it difficult to meet the real-time processing requirements.

[0088] In addition, the detection network is not optimized for the laminar structure characteristics of OCT images, resulting in limited accuracy of target region localization and affecting the accuracy of stitching.

[0089] For the object detection model, in this embodiment, through the StarNet-yolo architecture, which integrates deformable convolution and anatomical attention mechanism, the main body of the network is stacked by multi-level feature extraction modules. Each level of the module adapts to tissue deformation through a deformable convolutional layer and can capture complex anatomical structures such as retinal curvature and skin folds.

[0090] Specifically, to enhance the medical image features, an anatomical attention mechanism is set up, which uses prior knowledge such as vascular density and laminar texture to generate a spatial weight map and highlights high-information regions through adaptive weighting.

[0091] The scheme uses StarNet as the backbone network of YOLOv11. By introducing the StarBlock module to replace the CBS and C3k2 units of the original structure, the depth of the backbone network is compressed from 11 layers to 7 layers. In view of the characteristics of the target position detection task in OCT images, while maintaining the feature extraction ability, the computational complexity is significantly reduced.

[0092] In some embodiments, as Figure 5 shown, the object detection model includes a first layer, a second layer, a third layer, a fourth layer, a fifth layer, and a sixth layer; among them, the first layer, the second layer, the third layer, and the fourth layer have the same structure, and each includes a star aggregation module StarBlock.

[0093] The structures of the first layer, the second layer, the third layer, and the fourth layer are CBS and StarBlock, where CBS is a convolutional layer, batch normalization, and SiLU activation. A first feature extraction module is also included before the first layer. The size of the output OCT image is 640×640×3, and downsampling is performed through the convolutional kernels in the first feature extraction module to capture the overall laminar structure of the retina.

[0094] As Figure 6 shown, StarBlock captures features of different scales through multiple parallel branches. Each StarBlock contains three branches with different-sized convolutional kernels (1x1, 3x3, 5x5), which can capture local details and global context information simultaneously.

[0095] In some embodiments, the pair of images that have passed through the first feature extraction module is input into the central node to output an intermediate feature map; The intermediate feature map is input into the first radial branch to output a first radial feature map, and the intermediate feature map is input into the second radial branch to output a second radial feature map; The first radial feature map and the second radial feature map are merged to generate a merged result.

[0096] Specifically, each StarBlock is composed of a convolutional group with a star topology structure, including 1 central node, namely a 3×3 depthwise separable convolution, and 2 radial branches, namely a 1×1 convolution and a 3×3 or 5×5 dilated convolution. The module enhances the ability to extract layered texture features through parallel computing with different receptive fields. After the input features pass through the central node, they are divided into two branches. One branch passes through a 1×1 convolution, and the other branch passes through a 3×3 or 5×5 dilated convolution. The results after convolution are merged, and then feature compression is performed, that is, depth convolution, and finally residual fusion is performed.

[0097] Among them, the large-sized convolutional kernel focuses on the wide-field tissue layer structure and extracts the overall stratification trend of the retina. The second radial branch is configured with a small-sized convolutional kernel to capture the capillary microstructure and generate the coordinate information of the microvascular wall. The features output by the branches are superimposed in the channel dimension at the merging layer to achieve the enhancement of macroscopic structure and microscopic features.

[0098] Compared with the original CBS and C3k2 modules in terms of the feature aggregation mechanism, StarBlock achieves similar feature expression ability with fewer parameters through multi-scale receptive field fusion and parameter sharing.

[0099] For the first stage, StarBlock receives the feature map processed by CBS. The central node can be a 3×3 depthwise separable convolution to extract basic spatial features and reduce the computational load. The radial branch 1 adjusts the number of channels through a 1×1 convolution, such as dimensionality increase or decrease, to control the number of feature channels and adapt to subsequent operations. The radial branch uses a 5×5 or 3×3 dilated convolution. Among them, the 5×5 dilated convolution captures wide-field laminar textures, such as the retinal laminar boundary, and the 3×3 dilated convolution focuses on local microstructures, such as the capillary network. Then, the features of the two branches are fused through channel concatenation and the information amount is compressed. Finally, residual fusion is performed, that is, added to the input features to obtain the output feature map, which is added to the input 320×320×8 to output a feature map of 160×160×16.

[0100] Among them, the execution processes of the second stage, the third stage, and the fourth stage are the same as that of the first stage and will not be elaborated here. However, the difference is that different stages process different feature maps through size progression. Exemplarily, the first stage has a resolution of 160×160, perceives a 2mm visual field range, and is suitable for analyzing the laminar structure of the entire macular area. The second stage has a resolution of 80×80, focuses on a 1mm area, and captures capillary bifurcation points. The third stage has a resolution of 40×40, observes 500μm details, and identifies the morphology of microaneurysms. The fourth stage has a resolution of 20×20, analyzes 250μm microstructures, and locates individual edema cell clusters.

[0101] The original 11-layer structure of YOLOv11 is reconstructed into a 7-layer pyramid architecture in this embodiment. Each stage of the feature pyramid architecture contains an upsampling and a downsampling dual path. The downsampling path uses a convolution with a stride of 2 to compress the size of the feature map; the upsampling path restores the resolution through a transposed convolution. The outputs of the dual paths are fused through a gating mechanism: using the Sigmoid function to generate a spatial weight map, dynamically allocating the importance of features at each position, and enhancing the edge feature response at the junction of the inner and outer layers of the retina.

[0102] The input size of the input layer is 640×640×3. The first stage, the second stage, the third stage, and the fourth stage perform feature abstraction. Each layer contains a star-shaped aggregation module and outputs feature maps with decreasing resolutions. The sizes of the output feature maps are different. The size of the feature map output by the first stage is 160×160×16. The first stage retains the fine structure of the capillary plexus. The size of the feature map output by the second stage is 80×80×32, focusing on the mesoscale features of the vascular arcade area. The size of the feature map output by the third stage is 40×40×64. The third stage captures the pathological features of the optic disc area. The size of the feature map output by the fourth stage is 20×20×128, encoding the topological relationship of the entire retina.

[0103] The combined result after fusing the execution residuals of the fourth layer is input into the fifth layer. The fifth layer is an SPPF (Spatial Pyramid Pooling–Fast) layer, which inputs the feature map from the fourth layer into three pooling layers simultaneously for multi-scale feature extraction to enhance the model's detection ability for targets of different sizes.

[0104] The SPPF module contains an initial convolutional layer for feature compression, followed by multiple serial max pooling operations using a 5x5 pooling kernel, a stride of 1, and padding equal to half of the pooling kernel size. Finally, there is a convolutional layer that fuses features of different scales, and the output feature map has a size of 20×20×1024.

[0105] The output of the fifth layer is input into the sixth layer (C2PSA), and the output feature map has a size of 20×20×1024. In some embodiments, for the sixth layer, the feature map output by the sixth layer is subjected to bilinear upsampling to obtain a first sampling result; The first sampling result and the feature map output by the third layer are concatenated in channels to output a first fused feature; The feature map output by the second layer, the first fused feature, and the feature map output by the first cross-layer aggregation convolutional module are concatenated in channels to output a second fused feature; The second fused feature is input into the first cross-layer aggregation convolutional module to output a first feature map; The second fused feature and the first feature map are concatenated in features to output a third fused feature.

[0106] For the first fused feature, the middle-level tissue structure and deep pathological features are integrated, and the number of channels is expanded to 1088 to enhance the lesion representation ability. For the second fused feature, the shallow capillary coordinates and deep semantic features are fused to eliminate the loss of spatial information caused by four downsamplings, generating a 544-channel comprehensive feature.

[0107] In some embodiments, the target region is output through the following steps: The third fused feature is input into the second cross-layer aggregation convolutional module to output a second feature map; After the first feature map passes through the second feature extraction module, it is concatenated with the second feature map to output a third feature map; The third feature map is input into the third cross-layer aggregation convolutional module to obtain a fourth feature map; The fourth feature map is input into the third feature extraction module, and the second feature extraction module is concatenated with the feature map output by the sixth layer to output a fifth feature map; Input the fifth feature map into the fourth cross-layer aggregation convolution module to obtain a sixth feature map; Input the second feature map into the first detection head, input the fourth feature map into the second detection head, and input the sixth feature map into the third detection head to output the target region.

[0108] For the third feature map, splice it with the feature map with a size of 40×40×512 output by the first cross-layer aggregation convolution module to generate a 768-channel feature; for the fifth feature map, splice the feature map with a size of 20×20×1024 output by the sixth layer to generate a 1536-channel feature, and then obtain the sixth feature map through the fourth cross-layer aggregation convolution module. The sixth feature map is the feature map after compressing the channels, which can reduce the computational load of the detection head.

[0109] The 20×20×1024 depth feature output by the sixth layer is restored to a 40×40 resolution after being upsampled by a factor of two, and then spliced with the 40×40×64 middle-level feature output by the third layer to generate a 1088-dimensional fusion feature. This feature flows through the dilated convolutional kernels in the first cross-layer aggregation convolution module to strengthen the context correlation of the capillary gaps, and outputs an optimized 40×40×512 feature map. This map is integrated with the original feature of the second layer at the splicing node, and then input into the third feature extraction module for morphological refinement. Among them, the output of 20×20×512 is spliced with the original feature of the sixth layer to form a full-path integrated feature, which is finally extracted by the fourth cross-layer aggregation convolution module as a 20×20×1014 pathological response feature and sent to the third detection head.

[0110] Exemplarily, the first detection head receives an 80×80×256 feature map to analyze the retinal stratification boundary coordinates, the second detection head receives a 40×40×512 feature map to locate the microvascular lesion coordinates, and the third detection head receives a 20×20×1014 full-path feature to mark the cystoid edema area. The spatial coordinates output by the three detection heads form a lesion distribution map with millimeter-level accuracy in the macular area.

[0111] Evaluate the object detection processes of Yolov11 and the present application through OCT images. The evaluation results are as follows in the table:

[0112] The highest value of the object detection network provided in this embodiment in terms of the mean average precision 50 (B) metric is 0.995, which is 0.00007 higher than 0.99493 of Yolov11. Introducing a star-shaped aggregation module and a cross-layer feature fusion mechanism in the network architecture can effectively enhance the recognition accuracy of the retinal laminar structure in OCT images, especially in the localization of key anatomical structures such as the boundary of the nerve fiber layer.

[0113] When the mean average precision is between 50 - 95 (B), the object detection network provided in this embodiment improves by 0.0299 compared to Yolov11, which can improve the detection rate and integrity.

[0114] In the feature matching stage, based on the improved LightGlue algorithm, in-depth optimization is carried out for the characteristics of low texture and high noise in OCT images.

[0115] It is executed through the following steps: Perform preprocessing on the target area to output the preprocessing result. The preprocessing is used to suppress speckle noise and equalize the contrast between tissue layers in the local OCT area framed by the object detection. For example, by implementing direction-adaptive Gaussian filtering to set a long-axis filter kernel along the retinal stratification direction and a short-axis filter kernel in the vertical direction, combined with linear normalization processing based on the reflection intensity of the retinal pigment epithelium layer to eliminate the interference of blood flow artifacts.

[0116] In some embodiments, perform filtering enhancement on the target area to output an enhancement result, where the contrast of the tubular structures in the enhancement result is higher than that of the tubular structures in the target area; perform local contrast normalization on the enhancement result to output the preprocessing result.

[0117] In the preprocessing stage, use the Frangi filter to enhance the contrast of microvascular structures. This filter analyzes through a multi-scale Hessian matrix, highlights tubular structures, and suppresses background noise, making it easier to identify fine features such as retinal blood vessels or skin pores. Then perform local contrast normalization, and process the image in blocks with a 32×32 pixel sliding window to eliminate brightness fluctuations caused by uneven illumination or scanning angle differences.

[0118] Pass the preprocessing result through a feature extraction network to extract feature points of multiple target areas. In one implementation, the feature extraction network is designed with a residual architecture. The first layer is preset with a direction-constrained convolutional kernel for the retinal blood vessel orientation parameters to capture anatomical structure features. Combining the gradient direction consistency in the spatial-channel dual attention mechanism, the enhancement module strengthens the tissue boundary response, and enhances pathological features through channel attention weights, and finally outputs a feature descriptor.

[0119] In another implementation, the feature extraction network adopts a dual-branch CNN structure with shared weights to achieve feature extraction through a dynamic convolutional kernel adjustment mechanism. The input image, that is, the preprocessed image, undergoes 5 levels of downsampling to obtain multi-scale feature maps. Each level contains 3 dynamic convolutional blocks, and the last layer outputs a 128-dimensional feature vector. By introducing a feature importance scoring mechanism, the feature dimension is automatically adjusted according to the scene complexity, using 64-dimensional features in simple areas and retaining 256-dimensional features in complex areas.

[0120] In some embodiments, the feature extraction network includes multiple first levels; the preprocessing result is sequentially input into the multiple first levels, and through the dynamic convolution blocks in the first levels, and a downsampling operation is performed to output a level result; the target feature points in the level result are extracted.

[0121] The first levels are constructed with multiple levels of feature abstraction in a cascaded manner. The first level receives the original input, and the last level outputs high-level semantic features.

[0122] The first levels include a downsampling operation and multiple dynamic convolution blocks. The dynamic convolution blocks are composed of configurable convolution kernel groups and selection switches. The selection switches automatically adapt the optimal convolution kernel according to the texture complexity of the input features. In one implementation, the dynamic convolution block contains three sets of preset kernels: a standard convolution kernel for processing regular textures, a dilated convolution kernel for capturing large-scale structures, and a depth convolution kernel for optimizing the processing of low-texture regions.

[0123] In another implementation, the dynamic convolution block is configured as a convolution type switcher, which switches to deformable convolution when detecting the boundary features of the retinal layer and switches to lightweight convolution when facing the uniform region of the vitreous body. Both implementations achieve real-time switching through a feature analysis module. The analysis module contains a lightweight neural network that outputs a kernel selection signal after evaluating the spatial distribution of the current feature map.

[0124] The downsampling operation is the resolution conversion link between levels, and is implemented by using strided convolution or pooling methods to achieve spatial compression. In the scenario of retinal vessel processing, strided convolution reduces the size of the feature map by setting a fixed stride while increasing the number of channels; the pooling operation uses an adaptive window to retain local extrema in the area of microaneurysms. Both methods follow the principle of trading space for depth to ensure the corresponding relationship of key anatomical structure features in different levels.

[0125] Specifically, the preprocessing result is input into the multiple dynamic convolution blocks and a downsampling operation is performed to output a first-level result; The first-level result is input into the multiple dynamic convolution blocks and a downsampling operation is performed to output a second-level result, and the number of channels of the second-level result is greater than that of the first-level result; The second-level result is input into the multiple dynamic convolution blocks and a downsampling operation is performed to output a third-level result, and the number of channels of the third-level result is greater than that of the second-level result; The third-level result is input into the multiple dynamic convolution blocks and a downsampling operation is performed to output a fourth-level result, and the number of channels of the fourth-level result is greater than that of the third-level result; Input the fourth-layer result into multiple dynamic convolution blocks and perform downsampling operations to output the fifth-layer result, where the number of channels of the fifth-layer result is greater than that of the fourth-layer result; Extract the target feature points from the fifth-layer result.

[0126] Among them, the first-layer result, the second-layer result, the third-layer result, the fourth-layer result, and the fifth-layer result are the outputs of each first-level layer, including dual representations of spatial feature maps and channel feature maps. The spatial feature map records the relative position information of each layer structure of the retina, and the channel feature map encodes the topological attributes of blood vessel branches. As the layer level progresses, the spatial resolution gradually decreases while the channel dimension gradually increases. The high-level layer results focus on macroscopic structure associations, and the low-level layer results retain microscopic detail features.

[0127] Exemplarily, the preprocessed OCT image is input into the first level in a single-channel form. Two dynamic convolution blocks are set in this level. The first convolution block performs a standard 3×3 convolution to capture basic edge textures; the second convolution block switches the operation mode according to the output signal of the feature analysis module, that is, when the blood vessels in the macular area are recognized, dilated convolution is used to expand the receptive field, and when the vitreous area is marked, depth convolution is enabled to reduce the computational amount. After convolution processing, max pooling downsampling with a stride of two is performed, and the output size is reduced to a feature map that is half of the input. This feature map carries the striped texture features of the retinal nerve fiber layer and at the same time retains the light intensity gradient distribution of the blood vessel wall.

[0128] The processing result is transmitted to the second level, which contains three dynamic convolution blocks. The feature analysis module detects the heterogeneous features of the lesion area at this stage and triggers a convolution type switch: deformable convolution is used for the edema area to adapt to irregular boundaries, and a noise-resistant convolution kernel is configured for the exudate area. After three-layer convolution processing, stride convolution downsampling is performed, and the spatial resolution of the output feature map is halved again, but the number of channels is doubled. At this time, the feature map can already distinguish the boundary contours of the inner plexiform layer and the outer nuclear layer.

[0129] After the third level receives the compressed feature map, the dynamic convolution block enables a special topological convolution kernel for the capillary network and strengthens the bifurcation point features through an eight-neighborhood connection mode. Mean pooling is used for downsampling to retain the light intensity distribution information. When processing the fourth level, the feature analysis module recognizes the drusen structure and activates the multi-scale fusion convolution block to synchronously extract the calcification point features and edge features of the lesion center. The final level cancels downsampling and outputs a high-dimensional feature vector set containing spatial position encoding, and non-maximum suppression is performed on the feature map to locate the key points.

[0130] Taking the OCT image of diabetic macular edema as an example, when the preprocessed image is input into the first layer, the dynamic convolution block detects the weak blood vessel signals in the fovea area of the macula and automatically switches to a small kernel convolution with high sensitivity to enhance the response. After five layers of processing, key points are located on the feature map output by the fifth layer, including the center points of microaneurysms, venous crossing points, and the boundary points between retinal layers. All feature points carry the layer coding information of this position in the blood vessel tree.

[0131] Based on the feature points output by the feature extraction network, the feature points are processed through graph attention matching to obtain node features. The extracted feature points are input into the graph attention matching network to construct a graph structure. The nodes store the coordinates and descriptors of the feature points, and the edge weights are calculated by a four-head attention mechanism, which fuses cosine similarity and relative position encoding.

[0132] Then, a bidirectional nearest neighbor search is performed based on the node features to generate candidate matching pairs, and geometric verification is performed on the candidate matching pairs to output the confidence scores of the candidate matching pairs; geometric verification is performed on the candidate pairs. For example, 500 rounds of RANSAC iterations are performed to calculate the homography matrix hypothesis, and the confidence score in the 0-1 interval is calculated according to the inlier ratio. If the confidence score is greater than the score threshold, in this embodiment, the score threshold is 0.8, the candidate matching pairs corresponding to the confidence scores greater than the score threshold are determined as matching point pairs, that is, the list of feature point pairs corresponding to the coordinates.

[0133] Exemplarily, the feature point set constructs a fully connected graph of 82 nodes, and the retinal layer coordinate encoding is added during initialization. In the first iteration, the four heads of attention are calculated respectively. Head 1 focuses on the spatial distance, Head 2 strengthens the vascular topology constraint, Head 3 associates the intra-layer structure, and Head 4 suppresses cross-layer matching; after being updated by 3 layers of MLP (Multi-Layer Perceptron), the low-weight edges are pruned. The similarity matrix output by the third iteration shows. In the bidirectional search stage, 73 groups of candidate pairs are found, among which 12 groups are marked as suspicious due to cross-layers. Geometric verification performs RANSAC calculation: when 4 groups of venous bifurcation points are sampled, the reprojection error of the fitted homography matrix is 0.3 pixels; when cross-layer points are sampled, the error reaches 5.2 pixels. After 500 rounds of iteration, 68 groups of inliers are finally confirmed, the confidence score is 0.89, all matching points exactly correspond to the anatomical sites of blood vessel bifurcations, and the morphological error of microaneurysms is less than 1μm, meeting the clinical measurement accuracy standard.

[0134] After obtaining the matching point pairs, the steps of S200 - S400 are executed to obtain the Figure 7 panoramic OCT image as shown. The method detects the fundus OCT image, intercepts the position where the detected OCT is located, extracts and matches the feature points of multiple consecutive OCTs before and after, and then performs image stitching to improve the stitching accuracy and speed.

[0135] For the similar parts among the embodiments provided in this application, reference can be made to each other. The specific embodiments provided above are only several examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other embodiments extended based on the solution of this application without creative efforts fall within the protection scope of this application.

Claims

1. An OCT image stitching method, characterized in that, Including: Obtain an image pair and a corresponding pair of matching points for the image pair, where the image pair is adjacent OCT images; Calculate a homography matrix based on the pair of matching points to align the image pair and output a preliminary aligned image pair; Construct a fusion weight map, where the fusion weight map includes weight values of pixel positions and region types; Perform fusion stitching on pixel positions according to the weight values and region types to output a panoramic OCT image.

2. The OCT image stitching method according to claim 1, characterized in that, The obtaining the image pair and the corresponding pair of matching points for the image pair includes: Obtain an image pair, where the image pair includes multiple images; Perform object detection on the image pair to locate the target region, and the object detection is performed through an object detection model; Perform feature matching on the target region using a feature matching algorithm to output a pair of matching points, where the pair of matching points is a pair of feature matching points between adjacent images.

3. The OCT image stitching method according to claim 1, wherein The calculating the homography matrix based on the pair of matching points to align the image pair and output a preliminary aligned image pair includes: Remove mis-matching points in the pair of matching points through outlier rejection to output a target pair of matching points; Based on the target pair of matching points, calculate a transformation matrix through a geometric estimation algorithm; Apply the transformation matrix to align the image pair and generate a preliminary aligned image.

4. The OCT image stitching method according to claim 1, wherein The constructing the fusion weight map includes: Calculate the pixel-level displacement field of the preliminary aligned image through a multi-scale optical flow algorithm to determine the deformation amount of the local region; Construct a fusion weight map; Based on the deformation amount, divide the fusion weight map to output a high-confidence region, a deformation transition region, and a highly deformed region. The weight value of the high-confidence region is a first value, the weight value of the deformation transition region is a second value, and the weight value of the highly deformed region is a third value. The first value is greater than the second value, and the second value is greater than the third value.

5. The OCT image stitching method according to claim 4, wherein The performing fusion stitching on pixel positions according to the weight values and region types to output a panoramic OCT image includes: Perform maximum fusion on the high-confidence region, perform weighted hybrid fusion on the deformation transition region, and perform elastic deformation compensation fusion on the pixels of the highly deformed region to output a fusion result; Locate the stitching seam region in the fusion result; Obtain the similarity of the stitching seam region. If the similarity is less than the similarity threshold, update the fusion weight map of the stitching seam region; If the similarity is greater than or equal to the similarity threshold, output a panoramic OCT image.

6. The OCT image stitching method according to claim 2, wherein, The object detection model includes a first layer, a second layer, a third layer, a fourth layer, a fifth layer, and a sixth layer; The first layer, the second layer, the third layer, and the fourth layer have the same structure, each including a star-shaped aggregation module, and the output feature map sizes of the first layer, the second layer, the third layer, and the fourth layer are different; The method further includes: Perform two-fold upsampling on the feature map output by the sixth layer to obtain a first sampling result; Perform channel stitching on the first sampling result and the feature map output by the third layer to output a first fusion feature; Perform channel stitching on the feature map output by the second layer, the first fusion feature, and the feature map output by the first cross-layer aggregation convolution module to output a second fusion feature; Input the second fusion feature into the first cross-layer aggregation convolution module to output a first feature map; Perform feature concatenation on the second fusion feature and the first feature map to output a third fusion feature.

7. The OCT image stitching method according to claim 6, wherein The object detection on the image pair to locate the target region includes: Input the third fusion feature into the second cross-layer aggregation convolution module to output a second feature map; After passing the first feature map through the second feature extraction module, concatenate it with the second feature map to output a third feature map; Input the third feature map into the third cross-layer aggregation convolution module to obtain a fourth feature map; Input the fourth feature map into the third feature extraction module, and concatenate the second feature extraction module with the feature map output from the sixth layer to output a fifth feature map; Input the fifth feature map into the fourth cross-layer aggregation convolution module to obtain a sixth feature map; Input the second feature map into the first detection head, input the fourth feature map into the second detection head, and input the sixth feature map into the third detection head to output the target region.

8. The OCT image stitching method according to claim 6, wherein The star-shaped aggregation module includes a central node and at least two radial branches; Before performing the two-fold upsampling on the feature map output from the sixth layer to obtain a first upsampling result, it further includes: Input the image pair that has passed through the first feature extraction module into the central node to output an intermediate feature map; Input the intermediate feature map into the first radial branch to output a first radial feature map, and input the intermediate feature map into the second radial branch to output a second radial feature map; Merge the first radial feature map and the second radial feature map to generate a merged result, and input the merged result into the sixth layer.

9. The OCT image stitching method according to claim 2, characterized in that, The feature matching of the target region using a feature matching algorithm to output a matching point pair includes: Perform preprocessing on the target region to output a preprocessing result; Pass the preprocessing result through a feature extraction network to extract feature points of multiple target regions; Process the feature points through graph attention matching to obtain node features; Perform bidirectional nearest neighbor search based on the node features to generate candidate matching pairs; Perform geometric verification on the candidate matching pairs to output the confidence score of the candidate matching pairs; If the confidence score is greater than the score threshold, determine the candidate matching pairs corresponding to the confidence score greater than the score threshold as the matching point pairs.

10. The OCT image stitching method according to claim 9, characterized in that, The feature extraction network includes multiple first levels; the first level includes a downsampling operation and multiple dynamic convolution blocks; The passing the preprocessing result through a feature extraction network to extract feature points of multiple target regions includes: Input the preprocessing result into multiple first levels in sequence, pass through the dynamic convolution blocks in the first level, and perform the downsampling operation to output a level result; Extract the target feature points from the level result.

Images