A multi-view barrel distortion fundus image synchronous splicing method, device and storage medium

Abstract

The application discloses a kind of multi-view barrel distortion fundus image synchronous splicing method, equipment and storage medium, belong to image processing technical field.The method comprises: obtaining multiple original fundus images;Each original fundus image is respectively subjected to feature extraction processing, and the feature point map and feature descriptor map corresponding to each original fundus image are generated;Based on the current reference feature point map and current reference feature descriptor map corresponding to current reference image, and the source feature point map and source feature descriptor map corresponding to multiple source images, each source image is transformed using iterative method, and multiple transformed images are obtained;Based on the fusion weight map corresponding to multiple transformed images, multiple transformed images are fused to generate panoramic fundus image.The technical scheme provided by the application can improve the accuracy and efficiency of multi-view barrel distortion ultra-wide-angle fundus image splicing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a multi-view barrel distortion fundus image synchronous splicing method, device and storage medium. BACKGROUND

[0002] Ultra-wide field fundus imaging technology is increasingly important in ophthalmic diagnosis due to its wide field of view. However, this technology has a trade-off between single imaging field of view and resolution, and there is inevitable barrel distortion at the edge of the image. In order to obtain a high-resolution, large-view and geometrically accurate panorama, splicing multiple multi-view ultra-wide field images with overlapping areas becomes a key technical requirement.

[0003] At present, the existing method has significant limitations in terms of accuracy and efficiency when processing such images: (1) the existing registration technology and the matching degree of the ultra-wide-angle distortion are insufficient, resulting in a shaky basis for splicing: the existing technology usually adopts a hybrid model combining "rigid registration and non-rigid optimization", which first estimates a global rigid or affine transformation through feature matching, and then uses optical flow and other non-rigid models for local fine-tuning. However, the barrel distortion introduced by the ultra-wide-angle lens is a large-scale, non-linear geometric deformation, which seriously violates the basic assumption of rigid transformation. Although the subsequent non-rigid optimization aims to correct the residual error, its correction ability is limited, and it is difficult to effectively optimize the incorrect matching caused by severe non-linear distortion, resulting in the initial matching error being transmitted and amplified, and unable to provide a reliable basis for high-precision splicing. (2) The existing non-rigid deformation field generation method has insufficient accuracy for non-linear distortion images: the existing method (such as dense optical flow method) is usually used to generate local deformation fields, which is acceptable for small-scale displacement, but in the image edge area where distortion is most significant, the estimation accuracy is significantly reduced due to large pixel displacement and severe texture deformation. This leads to difficulty in achieving accurate pixel-level alignment in critical areas, and the splicing result is prone to detail blur or ghosting at the edge, affecting image accuracy. (3) The existing sequence splicing architecture is inefficient and difficult to ensure global visual consistency: for multi-image (>2) splicing, the existing technology generally adopts a sequential or pair-by-pair splicing strategy, which has two inherent bottlenecks: one is feature extraction redundancy: the current method is mostly a one-to-one sequence splicing, and each time a new image is processed, feature extraction needs to be performed on the intermediate splicing result generated in the previous step, which not only brings huge computational redundancy, but also affects the stability and accuracy of subsequent feature matching due to the degradation of the intermediate image quality. The other is short-sighted fusion decision: sequential fusion is a greedy algorithm that only optimizes the fusion of two images at a time. Early decisions (such as brightness and color balance) are fixed and transmitted, resulting in the system being unable to perform global visual optimization based on all image information, and the final panorama is prone to have obvious brightness or color jumps between different areas, resulting in poor overall visual consistency.

[0004] Therefore, a more reliable solution is needed. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, device and storage medium for synchronous stitching of multi-view barrel distortion fundus images, which can improve the accuracy and efficiency of synchronous stitching of multi-view barrel distortion ultra-wide-angle fundus images.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] On one hand, the present invention provides a method for synchronous stitching of multi-view barrel distortion fundus images, the method comprising:

[0008] Acquire multiple raw fundus images;

[0009] Each original fundus image is subjected to feature extraction processing to generate a feature point map and a feature descriptor map corresponding to each original fundus image;

[0010] Based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, and the source feature point map and source feature descriptor map corresponding to multiple source images, an iterative method is used to transform each source image to obtain multiple transformed images; the current reference image is the first original fundus image among the multiple original fundus images, and the multiple source images include the original fundus images remaining after removing the first original fundus image from the multiple original fundus images;

[0011] Based on the fusion weight map corresponding to the multiple transformed images, the multiple transformed images are fused to generate a panoramic fundus image.

[0012] In some possible implementations, based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, and the source feature point maps and source feature descriptor maps corresponding to multiple source images, an iterative method is used to transform each source image to obtain multiple transformed images, including:

[0013] Based on each reference feature point in the current reference feature point map and the current reference feature descriptor map, and each source feature point in the current source feature point map corresponding to the current source image and the current source feature descriptor map, a set of corrected matching point pairs is determined;

[0014] Based on the corrected set of matching points and the current source feature descriptor, the relative displacement field is processed to determine the absolute displacement field from the current source image to the current reference image;

[0015] Based on the absolute displacement field, the current source image and its corresponding current source feature point map and current source feature descriptor map are transformed to generate a transformed image and its corresponding transformed feature point map and transformed feature descriptor map.

[0016] The transformed feature point map and transformed feature descriptor map are fused with the current reference feature point map and the current reference feature descriptor map to generate an updated current reference feature point map and an updated current reference feature descriptor map. The next source image is used as the current source image. The steps of determining the set of corrected matching point pairs based on each reference feature point and the current reference feature descriptor map in the current reference feature point map, and each source feature point and the current source feature descriptor map corresponding to the current source image, are returned until a corresponding transformed image is generated for each source image.

[0017] In some possible implementations, determining the set of corrected matching point pairs based on each reference feature point in the current reference feature point map and the current reference feature descriptor map, and each source feature point in the current source feature point map corresponding to the current source image and the current source feature descriptor map, includes:

[0018] For each reference feature point and each source feature point, feature point matching is performed to generate an initial set of matching point pairs;

[0019] The offset prediction network is used to perform offset prediction processing on the initial set of matching point pairs, the current baseline feature descriptor map, and the current source feature descriptor map to generate the target offset.

[0020] Based on the target offset, each source feature point in the initial matching point pair set is corrected to generate a corrected matching point pair set.

[0021] In some possible implementations, the step of performing offset prediction processing on the initial set of matching point pairs, the current baseline feature descriptor map, and the current source feature descriptor map using an offset prediction network to generate a target offset includes:

[0022] Based on the initial set of matching point pairs, local feature regions are extracted from the current baseline feature descriptor map and the current source feature descriptor map;

[0023] The local feature region is input into the offset prediction network to predict the offset and generate the target offset, as shown in the following formula:

[0024] ;

[0025] in, This represents the target offset corresponding to the i-th source feature point. This represents the offset in the x-direction of the target offset. This represents the offset in the y-direction of the target offset; This represents the current source feature descriptor subgraph. This represents the current baseline feature description subgraph; This represents the i-th source feature point in the initial matching point pair. This represents the i-th reference feature point in the initial matching point pair; This represents the offset prediction network. This indicates the operation of extracting local feature regions.

[0026] In some possible implementations, processing the relative displacement field based on the corrected set of matching point pairs and the current source feature descriptor map to determine the absolute displacement field from the current source image to the current reference image includes:

[0027] Based on each corrected matching point pair in the set of corrected matching point pairs, determine the relative displacement vector of each corrected matching point pair;

[0028] Based on each corrected source feature point in the corrected matching point pair set and the current source feature descriptor, as well as each interpolation point of the relative displacement field and its corresponding interpolation descriptor, the composite weight between each interpolation point and each corrected source feature point is determined.

[0029] Based on the relative displacement vector of each corrected matching point pair and the composite weight, the relative displacement vector of each point to be interpolated is determined;

[0030] The relative displacement field is transformed based on the relative displacement vector of each interpolation point to generate the absolute displacement field.

[0031] In some possible implementations, determining the composite weight between each interpolation point and each modified source feature point based on each modified source feature point in the modified matching point pair set and the current source feature descriptor, as well as each interpolation point of the relative displacement field and its corresponding interpolation descriptor, includes:

[0032] Based on the Euclidean distance between each interpolation point and each correction source feature point, the geometric spatial weights between each interpolation point and each correction source feature point are determined, as shown in the following formula:

[0033] ;

[0034] in, This represents the Euclidean distance between the q-th interpolation point and the i-th corrected source feature point. ;

[0035] in, This represents the geometric spatial weight between the q-th interpolation point and the i-th modified source feature point. Let represent the coordinates of the q-th point to be interpolated in the x-direction. This represents the coordinate of the q-th point to be interpolated in the y-direction; This represents the coordinates of the i-th corrected source feature point in the x-direction. This represents the coordinates of the i-th modified source feature point in the y-direction;

[0036] Based on the current source feature descriptor map, determine the interpolation descriptor vector corresponding to each interpolation point and the source descriptor vector corresponding to each corrected source feature point;

[0037] Based on the interpolation descriptor vector and the source descriptor vector, the feature space weights between each point to be interpolated and each modified source feature point are determined, as shown in the following formula:

[0038] ;

[0039] in, This represents the feature space weights between the q-th interpolation point and the ith-th modified source feature point; This represents the interpolation descriptor vector corresponding to the q-th interpolation point. This represents the source descriptor vector corresponding to the i-th modified source feature point; This represents a scaling factor, used to control the degree to which the feature similarity between the interpolated descriptor vector and the source descriptor vector is affected. This is used to ensure that the feature space weights are positive;

[0040] Based on the geometric space weights and the feature space weights, the composite weights between each interpolation point and each correction source feature point are determined, as shown in the following formula:

[0041] ;

[0042] in, This represents the composite weight between the q-th interpolation point and the ith-th modified source feature point.

[0043] In some possible implementations, determining the relative displacement vector of each point to be interpolated based on the relative displacement vector of each corrected matching point pair and the composite weight includes:

[0044] Based on the composite weights, inverse distance weight interpolation is performed on the relative displacement vector of each corrected matching point pair to generate the relative displacement vector of each point to be interpolated, as shown in the following equation:

[0045] ;

[0046] ;

[0047] in, Let X represent the component of the relative displacement vector of the q-th point to be interpolated in the X direction. Let Y represent the component of the relative displacement vector of the q-th point to be interpolated in the Y direction. Represents the (x,y) pixel coordinates in the relative displacement field; This represents the composite weight between the q-th interpolation point and the ith modified source feature point; Let x represent the component of the relative displacement vector of the i-th corrected matching point pair in the x-direction. represents the component of the relative displacement vector of the i-th corrected matching point pair in the y-direction; N represents the total number of corrected matching point pairs.

[0048] In some possible implementations, the step of fusing the multiple transformed images based on the fusion weight map corresponding to the multiple transformed images to generate a panoramic fundus image includes:

[0049] Based on the grayscale image of each transformed image, the fusion weight map corresponding to each transformed image is determined, as shown in the following formula:

[0050] ;

[0051] in, This represents the fusion weight map corresponding to the k-th transformed image; Represents the grayscale image of the k-th transformed image; Represents the Laplace operator;

[0052] The panoramic fundus image is generated by fusing each transformed image and the corresponding fusion weight map.

[0053] On the other hand, an electronic device is provided, the device including a processor and a memory, the memory storing at least one instruction and at least one program, the at least one instruction and the at least one program being loaded and executed by the processor to implement the synchronous stitching method of multi-view barrel distortion fundus images as described above.

[0054] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction and at least one program are stored in the computer storage medium, wherein the at least one instruction and the at least one program are loaded and executed by a processor to implement the synchronous stitching method of multi-view barrel distortion fundus images as described above.

[0055] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0056] In this invention, multiple original fundus images are acquired; feature extraction processing is performed on each original fundus image to generate a feature point map and feature descriptor map corresponding to each original fundus image. This allows for single feature pre-extraction of all original fundus images, avoiding redundant calculations of repeated feature extraction on intermediate stitching results and improving the efficiency of synchronous stitching processing.

[0057] Next, the first original fundus image and its corresponding feature point map and feature descriptor map from multiple original fundus images are used as the current reference image and its corresponding current reference feature point map and current reference feature descriptor map. The remaining original fundus images and their corresponding feature point maps and feature descriptor maps are used as the source images and their corresponding source feature point maps and source feature descriptor maps. Based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, as well as the source feature point maps and source feature descriptor maps corresponding to multiple source images, an iterative method is used to transform each source image to obtain multiple transformed images. This can overcome the matching error caused by barrel distortion, thereby improving the accuracy of image stitching. At the same time, it can achieve more accurate pixel-level alignment that maintains the continuity of semantic structure, thereby improving the stitching accuracy of severely distorted areas such as image edges. Furthermore, the pre-extracted feature point map can be reused to realize image transformation, thereby improving the efficiency of synchronous image stitching.

[0058] Next, based on the fusion weight map corresponding to multiple transformed images, multiple transformed images are fused to generate a panoramic fundus image. This can optimize the visual performance of all overlapping areas from a global perspective at once, generating a seamless panoramic fundus image with high visual consistency. This improves the accuracy and reliability of image synchronization stitching, thereby enabling high-precision, high-efficiency, and high-visual-quality multi-view barrel distortion ultra-wide-angle fundus image synchronization stitching. Attached Figure Description

[0059] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart illustrating a method for synchronously stitching multi-view barrel distortion fundus images provided in an embodiment of the present invention.

[0061] Figure 2 This is a flowchart illustrating the process of transforming each source image to obtain multiple transformed images based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, as well as the source feature point map and source feature descriptor map corresponding to multiple source images, provided by an embodiment of the present invention.

[0062] Figure 3 These are three barrel distortion images to be stitched together according to an embodiment of the present invention, wherein (a) is a barrel distortion image from the frontal view, (b) is a barrel distortion image from the left view, and (c) is a barrel distortion image from the upper left view.

[0063] Figure 4 This is a splicing effect diagram provided in an embodiment of the present invention;

[0064] Figure 5 This is a schematic diagram of the structure of a synchronous stitching device for multi-view barrel distortion fundus images provided in an embodiment of the present invention. Detailed Implementation

[0065] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0066] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0067] In this embodiment of the invention, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0068] Various exemplary embodiments, features, and aspects of the present invention will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0069] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0070] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0071] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without certain specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order to highlight the spirit of the invention.

[0072] Figure 1 This is a flowchart illustrating a method for synchronously stitching multi-view barrel distortion fundus images according to an embodiment of the present invention. This specification provides the method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or server products, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 1 As shown, the above method may include:

[0073] S101: Acquire multiple raw fundus images;

[0074] In one specific embodiment, the multiple raw fundus images can be raw ultra-wide-angle fundus images from multiple perspectives that have overlapping areas and have not been fully geometrically corrected, which are to be stitched together.

[0075] S102: Perform feature extraction processing on each original fundus image to generate a feature point map and feature descriptor map corresponding to each original fundus image;

[0076] In one specific embodiment, the feature point map is a feature point probability map, which is a single-channel probability map with the same size as the original fundus image; specifically, the feature point map can represent the probability that each pixel is a stable feature point. The feature descriptor map is a multidimensional vector with 1 / 8 the size of the original fundus image and 256 channels, where the vector at each pixel position can be the source descriptor vector of that region.

[0077] In one specific embodiment, a feature extraction model can be used to extract features from each raw fundus image, generating a feature point map and a feature descriptor map corresponding to each raw fundus image. Optionally, the feature extraction model can be set according to actual application requirements, and the feature extraction model can be the SuperPoint model.

[0078] S103: Based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, and the source feature point map and source feature descriptor map corresponding to multiple source images, an iterative method is used to transform each source image to obtain multiple transformed images;

[0079] In one specific embodiment, the current reference image can be the first original fundus image among multiple original fundus images, and the multiple source images can include the remaining original fundus images excluding the first original fundus image. Specifically, the first original fundus image and its corresponding feature point map and feature descriptor map are used as the current reference image and its corresponding current reference feature point map and current reference feature descriptor map, while the remaining original fundus images and their corresponding feature point maps and feature descriptor maps are used as the source images and their corresponding source feature point maps and source feature descriptor maps. Optionally, the current reference image can be a fixed image referenced during the image stitching process; the source images can be images that need to be aligned with the reference image through transformation.

[0080] In an optional embodiment, Figure 2This is a flowchart illustrating the process of transforming each source image to obtain multiple transformed images, based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, and the source feature point maps and source feature descriptor maps corresponding to multiple source images, using an iterative method, as provided in this embodiment of the invention. Figure 2 As shown, the above-mentioned transformation of each source image using an iterative method, based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, and the source feature point maps and source feature descriptor maps corresponding to multiple source images, can include:

[0081] S201: Based on each reference feature point in the current reference feature point map and the current reference feature descriptor map, and each source feature point in the current source feature point map corresponding to the current source image and the current source feature descriptor map, determine the set of corrected matching point pairs;

[0082] S202: Based on the corrected set of matching point pairs and the current source feature descriptor map, process the relative displacement field to determine the absolute displacement field from the current source image to the current reference image;

[0083] S203: Based on the absolute displacement field, transform the current source image and its corresponding current source feature point map and current source feature descriptor map to generate the transformed image and its corresponding transformed feature point map and transformed feature descriptor map.

[0084] S204: Fuse the transformed feature point map and the transformed feature descriptor map with the current reference feature point map and the current reference feature descriptor map to generate the updated current reference feature point map and the updated current reference feature descriptor map, and take the next source image as the current source image, and return to step S201, until a corresponding transformed image is generated for each source image.

[0085] In an optional embodiment, the above-described determination of the corrected matching point pair set based on each reference feature point in the current reference feature point map and the current reference feature descriptor map, and each source feature point in the current source feature point map corresponding to the current source image and the current source feature descriptor map, includes:

[0086] For each baseline feature point and each source feature point, feature point matching is performed to generate an initial set of matching point pairs;

[0087] The offset prediction network is used to perform offset prediction processing on the initial set of matching point pairs, the current baseline feature descriptor map and the current source feature descriptor map to generate the target offset.

[0088] Based on the target offset, each source feature point in the initial set of matching point pairs is corrected to generate a corrected set of matching point pairs.

[0089] In one specific embodiment, a set of reference feature points and a set of source feature points can be determined by extracting local maxima points from the current reference feature point map and the current source feature point map. The set of reference feature points includes multiple reference feature points, and the set of source feature points includes multiple source feature points. Reference descriptor vectors corresponding to the positions of the reference feature points can be obtained using bilinear interpolation with the current reference feature descriptor map; similarly, source descriptor vectors corresponding to the positions of the source feature points can be obtained using bilinear interpolation with the current source feature descriptor map.

[0090] In a specific embodiment, the initial matching point pair set includes multiple initial matching point pairs, which may include multiple matching feature points between the current source image and the current reference image. The above-mentioned feature point matching for each reference feature point and each source feature point to generate the initial matching point pair set may include: inputting each reference feature point and its corresponding reference descriptor vector, and each source feature point and its corresponding source descriptor vector, into the SuperGlue model. The SuperGlue model, through its built-in graph neural network and attention mechanism, outputs a score matrix, which represents the matching score between each reference feature point and each source feature point; and determining the initial matching point pair set based on the score matrix. Specifically, based on the score matrix, the highest matching score for each reference feature point and the highest matching score for each source feature point are determined, thereby determining the matching relationship between each reference feature point and each source feature point, and obtaining the initial matching point pair set. ,in, This represents the i-th source feature point corresponding to the i-th initial matching point pair. This represents the i-th baseline feature point corresponding to the i-th initial matching point pair. Optionally, the SuperGlue model can also be replaced with any other deep neural network architecture capable of performing end-to-end feature matching and intrinsically learning complex geometric transformations, such as a Transformer-based feature matching model.

[0091] In an optional embodiment, the above-described offset prediction network performs offset prediction processing on the initial set of matching point pairs, the current baseline feature descriptor map, and the current source feature descriptor map to generate a target offset, including:

[0092] Based on the initial set of matching point pairs, local feature regions are extracted from the current baseline feature descriptor map and the current source feature descriptor map;

[0093] The local feature region is input into the offset prediction network to predict the offset and generate the target offset, as shown in the following formula:

[0094] ;

[0095] in, This represents the target offset corresponding to the i-th source feature point. This represents the offset in the x-direction of the target offset. This represents the offset in the y-direction of the target offset; This represents the current source feature descriptor subgraph. This represents the current baseline feature description subgraph; This represents the i-th source feature point in the initial matching point pair. This represents the i-th reference feature point in the initial matching point pair; This represents the offset prediction network. This indicates the operation of extracting local feature regions.

[0096] In a specific embodiment, the extraction of local feature regions from the current baseline feature descriptor map and the current source feature descriptor map based on the initial set of matching point pairs may include: extracting local feature regions from the current baseline feature descriptor map and the current source feature descriptor map using bilinear interpolation, centered on the baseline feature points and the source feature points, according to each initial matching point pair. Specifically, This describes the operation of extracting a local feature region centered at specified coordinates from a feature descriptor map. The offset prediction network is used to predict the offset from the source feature point location; specifically, it regresses to determine a sub-pixel offset. The target offset can be the offset used to correct the source feature point.

[0097] In one specific embodiment, each source feature point is corrected based on the target offset to generate corrected source feature points; a set of corrected matching point pairs is generated based on each corrected source feature point and each reference feature point. Specifically, the predicted target offset is used. Each source feature point is corrected to obtain the corrected source feature points. ,in, , This represents the coordinates of the source feature point in the x-direction. This represents the coordinates of the source feature point in the y-direction; the set of corrected matching point pairs can be... .

[0098] In the above embodiments, since the reference feature descriptor image and the source feature descriptor image contain rich contextual semantic information, the offset prediction network can utilize the contextual features of the source feature image and the reference feature image to perform offset regression calculation on the sub-pixel set of the initial matching point pair. Then, the source feature points are corrected, which effectively reduces the feature point positioning error caused by lens distortion, thereby optimizing the positioning of the initial matching point pair and overcoming the matching error caused by barrel distortion. This can improve the accuracy and reliability of subsequent absolute field generation, providing a solid foundation for high-precision stitching.

[0099] In an optional embodiment, the above-described processing of the relative displacement field based on the modified set of matching point pairs and the current source feature descriptor map to determine the absolute displacement field from the current source image to the current reference image may include:

[0100] For each corrected matching point pair in the corrected matching point pair set, determine the relative displacement vector of each corrected matching point pair;

[0101] Based on each corrected source feature point and the current source feature descriptor in the corrected matching point pair set, as well as each interpolation point in the relative displacement field and its corresponding interpolation descriptor, the composite weight between each interpolation point and each corrected source feature point is determined.

[0102] Based on the relative displacement vector and composite weight of each corrected matching point pair, the relative displacement vector of each point to be interpolated is determined;

[0103] Based on the relative displacement vector of each interpolation point, the relative displacement field is transformed to generate the absolute displacement field.

[0104] In one specific embodiment, each corrected matching point pair includes corrected source feature points. and reference feature points ,in, This represents the coordinates of the corrected source feature point in the x-direction. This indicates the coordinates of the corrected source feature point in the y-direction; This represents the coordinates of the reference feature point in the x-direction. This represents the coordinates of the reference feature point in the y-direction; optionally, the relative displacement vector between the source feature point and the reference feature point is corrected as follows: , Let x represent the component of the relative displacement vector of the i-th corrected matching point pair in the x-direction. This represents the component of the relative displacement vector of the i-th corrected matching point pair in the y-direction. Optionally, each corrected matching point pair and its corresponding relative displacement vector can constitute a sparse control point set describing the deformation relationship between images. This allows the source feature points to be corrected by the target offset predicted by the offset prediction network, providing a more accurate and reliable sparse control point set for subsequent absolute displacement field generation, thus providing a solid foundation for high-precision stitching.

[0105] In an optional embodiment, the above-described determination of the composite weight between each interpolation point and each modified source feature point based on each modified source feature point in the modified matching point pair set and the current source feature descriptor, and each interpolation point to be interpolated in the relative displacement field and its corresponding interpolation descriptor, includes:

[0106] Based on the Euclidean distance between each interpolation point and each correction source feature point, the geometric spatial weights between each interpolation point and each correction source feature point are determined, as shown in the following equation:

[0107] ;

[0108] in, This represents the Euclidean distance between the q-th interpolation point and the i-th corrected source feature point. ;

[0109] in, This represents the geometric spatial weight between the q-th interpolation point and the i-th modified source feature point. Let represent the coordinates of the q-th point to be interpolated in the x-direction. This represents the coordinate of the q-th point to be interpolated in the y-direction; This represents the coordinates of the i-th corrected source feature point in the x-direction. This represents the coordinates of the i-th modified source feature point in the y-direction;

[0110] Based on the current source feature descriptor map, determine the interpolation descriptor vector corresponding to each interpolation point and the source descriptor vector corresponding to each corrected source feature point;

[0111] Based on the interpolation descriptor vector and the source descriptor vector, the feature space weights between each point to be interpolated and each corrected source feature point are determined, as shown in the following equation:

[0112] ;

[0113] in, This represents the feature space weights between the q-th interpolation point and the ith-th modified source feature point; This represents the interpolation descriptor vector corresponding to the q-th interpolation point. This represents the source descriptor vector corresponding to the i-th modified source feature point; This represents the scaling factor, which controls the degree to which the feature similarity between the interpolated descriptor vector and the source descriptor vector affects the interpolation; This is used to ensure that the feature space weights are positive;

[0114] Based on geometric space weights and feature space weights, the composite weights between each interpolation point and each correction source feature point are determined, as shown in the following equation:

[0115] ;

[0116] in, This represents the composite weight between the q-th interpolation point and the ith-th modified source feature point.

[0117] In one specific embodiment, an empty matrix with the same size as the source image can be created to store the relative displacement field. Since the relative displacement field has the same size as the source image, the coordinates of each pixel in the relative displacement field directly correspond to the coordinates of that pixel in the source image, which can be (x, y). Optionally, the position of each pixel in the relative displacement field contains a two-dimensional vector. This two-dimensional vector can represent the amount of displacement required for this pixel to move from the source image to the reference image, where, This represents the relative displacement in the X direction at the (x,y) pixel coordinates in the relative displacement field. This represents the relative displacement in the Y direction at pixel coordinates (x, y) in the relative displacement field. The interpolation point can be any pixel in the relative displacement field corresponding to the source image; specifically, the coordinates of the q-th interpolation point in the relative displacement field. The relative displacement vector corresponding to the pixel coordinates (x, y) of the point to be interpolated in the source image. .

[0118] Optionally, based on the similarity between the feature descriptor vector of the point to be interpolated and the feature descriptor vector of the modified source feature point, bilinear interpolation can be used to obtain the interpolated descriptor vector corresponding to the point to be interpolated and the source descriptor vector corresponding to the modified source feature point from the current source feature descriptor map. Specifically, the feature space weights between the point to be interpolated and the modified source feature point can be determined by calculating the chord similarity between the interpolated descriptor vector and the source descriptor vector; the composite weights can be determined by combining the geometric space weights with the feature space weights.

[0119] In an optional embodiment, determining the relative displacement vector of each point to be interpolated based on the relative displacement vector and composite weight of each corrected matching point pair may include:

[0120] Based on composite weights, inverse distance weight interpolation is performed on the relative displacement vector of each corrected matching point pair to generate the relative displacement vector of each point to be interpolated, as shown in the following equation:

[0121] ;

[0122] ;

[0123] in, Let X represent the component of the relative displacement vector of the q-th point to be interpolated in the X direction. Let Y represent the component of the relative displacement vector of the q-th point to be interpolated in the Y direction. Represents the (x,y) pixel coordinates in the relative displacement field; This represents the composite weight between the q-th interpolation point and the ith modified source feature point; Let x represent the component of the relative displacement vector of the i-th corrected matching point pair in the x-direction. represents the component of the relative displacement vector of the i-th corrected matching point pair in the y-direction; N represents the total number of corrected matching point pairs.

[0124] In one specific embodiment Specifically, it can be represented as the component of the displacement vector of the (x,y) pixel coordinates in the relative displacement field in the X direction. Specifically, it can be represented as the component of the displacement vector of the (x,y) pixel coordinates in the Y direction in the relative displacement field; or it can be represented by Kriging interpolation algorithm, radial basis function interpolation algorithm, etc., to generate a dense deformation field based on discrete matching points.

[0125] In the above embodiments, the feature-aware composite inverse distance weighted interpolation strategy can ensure that in regions with complex textures and rich structures, even if the geometric distance is slightly far, control points with highly similar features will be given higher weights, thereby guiding the deformation field to better follow the semantic boundaries and structural features of the image, and thus playing a positive role in the pixel displacement of the distorted image.

[0126] In one specific embodiment, to facilitate subsequent image transformation operations, the relative displacement field is... Convert to absolute displacement field The vector at each position in the absolute displacement field The absolute target position of the source image pixels in the reference image coordinate system is defined; optionally, the relative displacement field is transformed according to the relative displacement vector of each point to be interpolated to generate the absolute displacement field, as shown in the following formula:

[0127] ;

[0128] ;

[0129] in, This represents the absolute displacement in the X direction at the (x,y) pixel coordinates in the absolute displacement field. This represents the absolute displacement in the Y direction at pixel coordinates (x, y) in the absolute displacement field, where x represents the pixel coordinate of the source image pixel in the X direction and y represents the pixel coordinate of the source image pixel in the Y direction.

[0130] In one specific embodiment, the source image can be mapped using bilinear interpolation and other resampling algorithms based on the absolute displacement field to generate a transformed image, and the source feature point map and source feature descriptor map can be mapped to generate a transformed feature point map and a transformed feature descriptor map.

[0131] In one specific embodiment, the transformed feature point map and the transformed feature descriptor map can be fused with the current baseline feature point map and the current baseline feature descriptor map to update the current baseline feature point map and the current baseline feature descriptor map. Then, the updated current baseline feature point map and the current baseline feature descriptor map can be used to process the next source image to determine the next transformed image. This ensures that the source feature point map and the source feature descriptor map are always reused during the iteration process, without the need to re-extract features from any intermediate fusion results, thus improving the efficiency of synchronous image stitching.

[0132] In the above embodiments, by performing a single feature pre-extraction on the original fundus image, the pre-extracted feature point map and feature descriptor map can be reused for feature point matching during the sequential deformation process, avoiding redundant calculations of feature extraction on the intermediate stitching results, thereby improving the processing efficiency of multi-image stitching.

[0133] In the above embodiments, by using a composite inverse distance weighted interpolation algorithm that integrates geometric space and feature space, a dense pixel displacement field that can cover the entire image range is determined based on the obtained sparse and corrected feature point matching pairs. This not only considers the Euclidean geometric distance between feature points, but also introduces the cosine similarity of their feature descriptors as weights. This improves the guiding role of feature-similar control points on deformation in key structural regions such as vascular networks and distorted image edges, generating a dense displacement field that better maintains the continuity of semantic structure. This enhances the accuracy and robustness of displacement field determination, thereby effectively improving the stitching accuracy in severely distorted regions such as image edges.

[0134] S104: Based on the fusion weight map corresponding to multiple transformed images, perform fusion processing on multiple transformed images to generate a panoramic fundus image;

[0135] In one specific embodiment, the fusion weight map may be derived from local contrast. Optionally, the fusion weight map may reflect the texture clarity of the image in the corresponding region; for high-contrast regions containing more detailed information, the fusion weight map is assigned a higher weight.

[0136] In an optional embodiment, the above-mentioned fusion processing of multiple transformed images based on a fusion weight map corresponding to multiple transformed images to generate a panoramic fundus image may include:

[0137] Based on the grayscale image of each transformed image, the fusion weight map corresponding to each transformed image is determined, as shown in the following formula:

[0138] ;

[0139] in, This represents the fusion weight map corresponding to the k-th transformed image; Represents the grayscale image of the k-th transformed image; Represents the Laplace operator;

[0140] Each transformed image and its corresponding fusion weight map are fused to generate a panoramic fundus image.

[0141] In one specific embodiment, Laplacian-Gaussian pyramid fusion can be performed on each transformed image and its corresponding fusion weight map to generate a panoramic fundus image. Optionally, other multi-resolution fusion algorithms, such as wavelet transform fusion, can also be used for image integration.

[0142] Optional, Figure 3 These are three barrel distortion images to be stitched together, as provided in this embodiment of the invention. Figure 3 As shown, the three barrel distortion images to be stitched are the frontal view barrel distortion image, the left view barrel distortion image, and the upper left view barrel distortion image. The frontal view barrel distortion image, the left view barrel distortion image, and the upper left view barrel distortion image can be used as the original fundus images to be stitched using the above method. Figure 4 This is a splicing effect diagram provided by an embodiment of the present invention, such as... Figure 4 As shown, the spliced ​​effect image is a comparison of... Figure 3 The image shown is a stitched image of the three barrel distortion images to be stitched together using the method described above.

[0143] In the above embodiments, transformed images can be fused synchronously to improve visual effects and visual quality. Specifically, by constructing a weighted map that comprehensively considers exposure, contrast and saturation, and using the Laplacian-Gaussian pyramid fusion algorithm, the visual performance of all overlapping areas is optimized at once from a global perspective, generating a seamless panoramic fundus image with high visual consistency, thereby improving the visual quality of image stitching processing.

[0144] As can be seen from the technical solutions provided in the embodiments of this specification above, this specification obtains multiple original fundus images; performs feature extraction processing on each original fundus image separately, and generates a feature point map and feature descriptor map corresponding to each original fundus image. This allows for single feature pre-extraction of all original fundus images, avoiding redundant calculations of repeated feature extraction on intermediate stitching results and improving the efficiency of synchronous stitching processing.

[0145] Next, the first original fundus image and its corresponding feature point map and feature descriptor map from multiple original fundus images are used as the reference image and its corresponding reference feature point map and reference feature descriptor map. The remaining original fundus images and their corresponding feature point maps and feature descriptor maps are used as the source images and their corresponding source feature point maps and source feature descriptor maps. Based on the reference feature point maps and reference feature descriptor maps corresponding to the reference images, and the source feature point maps and source feature descriptor maps corresponding to the multiple source images, an iterative method is used to determine multiple transformed images. Each iteration includes: determining a set of corrected matching point pairs based on each reference feature point and reference feature descriptor map in the reference feature point map, and each source feature point and source feature descriptor map in the current source image. This can overcome the matching error caused by barrel distortion, optimize the localization of matching points, and thus improve the accuracy and reliability of displacement field determination. This process improves the accuracy of image stitching. Next, based on the corrected matching point pair set and the source feature descriptor map, the relative displacement field is processed to determine the absolute displacement field from the current source image to the reference image, improving the accuracy of displacement field determination. Then, based on the absolute displacement field, each source image and its corresponding source feature point map and source feature descriptor map are transformed, generating multiple transformed images and their corresponding transformed feature point maps and transformed feature descriptor maps. This achieves more accurate pixel-level alignment while maintaining semantic structure continuity, thus improving the stitching accuracy in severely distorted areas such as image edges. Finally, the transformed feature point maps and transformed feature descriptor maps, as well as the reference feature point maps and reference feature descriptor maps, are fused to generate updated reference feature point maps and updated reference feature descriptor maps. The next source image is then used as the current source image, allowing the reuse of the pre-extracted feature point maps for matching, improving the efficiency of synchronous image stitching.

[0146] Next, based on the fusion weight map corresponding to multiple transformed images, multiple transformed images are fused to generate a panoramic fundus image. This can optimize the visual performance of all overlapping areas from a global perspective at once, generating a seamless panoramic fundus image with high visual consistency. This improves the accuracy and reliability of image synchronization stitching, thereby enabling high-precision, high-efficiency, and high-visual-quality multi-view barrel distortion ultra-wide-angle fundus image synchronization stitching.

[0147] This invention also provides a device for synchronously stitching multi-view barrel distortion fundus images, and correspondingly, Figure 5 This is a schematic diagram of the structure of the synchronous stitching device for multi-view barrel distortion fundus images provided in an embodiment of the present invention; as shown below. Figure 5 As shown, the above-mentioned device includes:

[0148] Image acquisition module 510 is used to acquire multiple raw fundus images;

[0149] The feature extraction module 520 is used to perform feature extraction processing on each original fundus image to generate a feature point map and a feature descriptor map corresponding to each original fundus image;

[0150] The transformed image determination module 530 is used to transform each source image using an iterative method based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, and the source feature point map and source feature descriptor map corresponding to multiple source images, to obtain multiple transformed images; the current reference image is the first original fundus image among the multiple original fundus images, and the multiple source images include the original fundus images remaining after removing the first original fundus image from the multiple original fundus images;

[0151] The image fusion module 540 is used to perform fusion processing on the multiple transformed images based on the fusion weight map corresponding to the multiple transformed images to generate a panoramic fundus image.

[0152] This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the synchronous stitching method for multi-view barrel distortion fundus images as described in any of the method embodiments.

[0153] Embodiments of the present invention also provide a computer storage medium, which may be disposed in a server to store at least one instruction, at least one program, code set, or instruction set for implementing the method embodiments. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the synchronous stitching method for multi-view barrel distortion fundus images as described in any of the method embodiments.

[0154] Optionally, in embodiments of the present invention, the storage medium may be located at at least one of a plurality of network servers in a computer network. Optionally, in embodiments of the present invention, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0155] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more flowcharts and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0157] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more flowcharts and / or boxes Figure 1 The function specified in one or more boxes.

[0158] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more flowcharts and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0160] Finally, it should be noted that the embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for synchronously stitching multi-view barrel distortion fundus images, characterized in that, The method includes: Acquire multiple raw fundus images; Each original fundus image is subjected to feature extraction processing to generate a feature point map and a feature descriptor map corresponding to each original fundus image; Based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, and the source feature point map and source feature descriptor map corresponding to multiple source images, an iterative method is used to transform each source image to obtain multiple transformed images; the current reference image is the first original fundus image among the multiple original fundus images, and the multiple source images include the original fundus images remaining after removing the first original fundus image from the multiple original fundus images; Based on the fusion weight map corresponding to the multiple transformed images, the multiple transformed images are fused to generate a panoramic fundus image; Specifically, based on the current reference feature point map and current reference feature descriptor map corresponding to the current reference image, and the source feature point maps and source feature descriptor maps corresponding to multiple source images, an iterative method is used to transform each source image to obtain multiple transformed images, including: Based on each reference feature point in the current reference feature point map and the current reference feature descriptor map, and each source feature point in the current source feature point map corresponding to the current source image and the current source feature descriptor map, a set of corrected matching point pairs is determined; Based on the corrected set of matching points and the current source feature descriptor, the relative displacement field is processed to determine the absolute displacement field from the current source image to the current reference image; Based on the absolute displacement field, the current source image and its corresponding current source feature point map and current source feature descriptor map are transformed to generate a transformed image and its corresponding transformed feature point map and transformed feature descriptor map. The transformed feature point map and transformed feature descriptor map are fused with the current reference feature point map and the current reference feature descriptor map to generate an updated current reference feature point map and an updated current reference feature descriptor map. The next source image is used as the current source image. The steps of determining the set of corrected matching point pairs based on each reference feature point and the current reference feature descriptor map in the current reference feature point map, and each source feature point and the current source feature descriptor map corresponding to the current source image are returned until a corresponding transformed image is generated for each source image. The step of processing the relative displacement field based on the corrected matching point pair set and the current source feature descriptor map to determine the absolute displacement field from the current source image to the current reference image includes: Based on each corrected matching point pair in the set of corrected matching point pairs, determine the relative displacement vector of each corrected matching point pair; Based on each corrected source feature point in the corrected matching point pair set and the current source feature descriptor, as well as each interpolation point of the relative displacement field and its corresponding interpolation descriptor, the composite weight between each interpolation point and each corrected source feature point is determined. Based on the relative displacement vector of each corrected matching point pair and the composite weight, the relative displacement vector of each point to be interpolated is determined; The relative displacement field is transformed based on the relative displacement vector of each interpolation point to generate the absolute displacement field; The step of determining the composite weight between each interpolation point and each modified source feature point based on each modified source feature point in the modified matching point pair set and the current source feature descriptor, as well as each interpolation point of the relative displacement field and its corresponding interpolation descriptor, includes: Based on the Euclidean distance between each interpolation point and each correction source feature point, the geometric spatial weights between each interpolation point and each correction source feature point are determined, as shown in the following formula: ； in, This represents the Euclidean distance between the q-th interpolation point and the i-th corrected source feature point. ; in, This represents the geometric spatial weight between the q-th interpolation point and the i-th modified source feature point. Let represent the coordinates of the q-th point to be interpolated in the x-direction. This represents the coordinate of the q-th point to be interpolated in the y-direction; This represents the coordinates of the i-th corrected source feature point in the x-direction. This represents the coordinates of the i-th modified source feature point in the y-direction; Based on the current source feature descriptor map, determine the interpolation descriptor vector corresponding to each interpolation point and the source descriptor vector corresponding to each corrected source feature point; Based on the interpolation descriptor vector and the source descriptor vector, the feature space weights between each point to be interpolated and each modified source feature point are determined, as shown in the following formula: ； in, This represents the feature space weights between the q-th interpolation point and the ith-th modified source feature point; This represents the interpolation descriptor vector corresponding to the q-th interpolation point. This represents the source descriptor vector corresponding to the i-th modified source feature point; This represents a scaling factor, used to control the degree to which the feature similarity between the interpolated descriptor vector and the source descriptor vector is affected. This is used to ensure that the feature space weights are positive; Based on the geometric space weights and the feature space weights, the composite weights between each interpolation point and each correction source feature point are determined, as shown in the following formula: ； in, This represents the composite weight between the q-th interpolation point and the ith modified source feature point; The step of determining the relative displacement vector of each point to be interpolated based on the relative displacement vector of each corrected matching point pair and the composite weight includes: Based on the composite weights, inverse distance weight interpolation is performed on the relative displacement vector of each corrected matching point pair to generate the relative displacement vector of each point to be interpolated, as shown in the following equation: ； ； in, Let X represent the component of the relative displacement vector of the q-th point to be interpolated in the X direction. Let Y represent the component of the relative displacement vector of the q-th point to be interpolated in the Y direction. Represents the (x,y) pixel coordinates in the relative displacement field; This represents the composite weight between the q-th interpolation point and the ith modified source feature point; Let x represent the component of the relative displacement vector of the i-th corrected matching point pair in the x-direction. represents the component of the relative displacement vector of the i-th corrected matching point pair in the y-direction; N represents the total number of corrected matching point pairs.

2. The method for synchronously stitching multi-view barrel distortion fundus images according to claim 1, characterized in that, The step of determining the set of corrected matching point pairs based on each reference feature point in the current reference feature point map and the current reference feature descriptor map, and each source feature point in the current source feature point map corresponding to the current source image and the current source feature descriptor map, includes: For each reference feature point and each source feature point, feature point matching is performed to generate an initial set of matching point pairs; The offset prediction network is used to perform offset prediction processing on the initial set of matching point pairs, the current baseline feature descriptor map, and the current source feature descriptor map to generate the target offset. Based on the target offset, each source feature point in the initial matching point pair set is corrected to generate a corrected matching point pair set.

3. The synchronous stitching method for multi-view barrel distortion fundus images according to claim 2, characterized in that, The step of performing offset prediction processing on the initial set of matching point pairs, the current baseline feature descriptor map, and the current source feature descriptor map through an offset prediction network to generate a target offset includes: Based on the initial set of matching point pairs, local feature regions are extracted from the current baseline feature descriptor map and the current source feature descriptor map; The local feature region is input into the offset prediction network to predict the offset and generate the target offset, as shown in the following formula: ； in, This represents the target offset corresponding to the i-th source feature point. This represents the offset in the x-direction of the target offset. This represents the offset in the y-direction of the target offset; This represents the current source feature descriptor subgraph. This represents the current baseline feature description subgraph; This represents the i-th source feature point in the initial matching point pair. This represents the i-th reference feature point in the initial matching point pair; This represents the offset prediction network. This indicates the operation of extracting local feature regions.

4. The method for synchronous stitching of multi-view barrel distortion fundus images according to claim 1, characterized in that, The step of fusing the multiple transformed images based on the fusion weight map corresponding to the multiple transformed images to generate a panoramic fundus image includes: Based on the grayscale image of each transformed image, the fusion weight map corresponding to each transformed image is determined, as shown in the following formula: ； in, This represents the fusion weight map corresponding to the k-th transformed image; Represents the grayscale image of the k-th transformed image; Represents the Laplace operator; The panoramic fundus image is generated by fusing each transformed image and the corresponding fusion weight map.

5. An electronic device comprising a processor and a memory, the memory storing at least one instruction and at least one program, the at least one instruction and the at least one program being loaded and executed by the processor to implement the synchronous stitching method for multi-view barrel distortion fundus images as described in any one of claims 1 to 4.

6. A computer storage medium storing at least one instruction and at least one program, wherein the at least one instruction and the at least one program are loaded and executed by a processor to implement the synchronous stitching method for multi-view barrel distortion fundus images as described in any one of claims 1 to 4.

Images