Method and apparatus for matching medical images

Abstract

The application relates to the technical field of image matching methods, and provides a medical image matching method and device, and solves the problem of insufficient matching accuracy of lesion regions in the prior art. The method comprises the following steps: aligning two fundus images at different time points based on key points of a retinal blood vessel network; performing multi-scale decomposition on the aligned images to obtain multi-scale frequency subbands; forming a propagation path by linking local extreme points of signal strength in the subbands across scales, and respectively determining candidate lesion seed points in the two images; taking the seed points as starting points, performing region growing according to the signal strength gradient direction of the frequency subbands, and defining a lesion contour; mapping the lesion contour to a retinal blood vessel topological space, analyzing and comparing the structure and spatial relationship consistency of topological subgraphs corresponding to the lesion contours in the two images, and establishing a matching link between the lesion regions. The application can realize accurate matching and correlation of lesion regions in fundus images collected at different time points.

Description

Technical Field

[0001] This application relates to the field of image matching methods, and more particularly to a method and apparatus for matching medical images. Background Technology

[0002] Medical image matching methods, especially the comparison of sequential fundus images acquired at different time points, have important application prospects in clinical scenarios such as disease progression monitoring and treatment effect evaluation.

[0003] In existing technologies, matching is directly based on feature points of the lesion itself extracted from the image. Contour or texture feature points of the lesion area are detected on two fundus images at different time points. By calculating the similarity between these feature points, the correspondence between the lesion areas in the two images is established.

[0004] However, due to the potential for subtle errors in the extraction and matching of vascular bifurcation points, and because the registration process based solely on blood vessels primarily focuses on aligning global anatomical structures without fully considering the stability of local image characteristics of the lesion region, direct identification and matching of lesion regions in the aligned image are easily affected by residual registration errors and variations in the lesion's morphology and brightness. This makes it difficult to establish accurate and reliable matching relationships. Therefore, existing technologies suffer from insufficient accuracy in lesion region matching. Summary of the Invention

[0005] The purpose of this application is to provide a method and apparatus for matching medical images, so as to solve the problem of insufficient accuracy in matching lesion areas in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a method for matching medical images, comprising:

[0007] First and second fundus images were acquired at different time points. The first and second fundus images are digital fundus images containing the lesion area.

[0008] Based on the key points of the retinal vascular network in the first fundus image and the second fundus image, a spatial transformation function is calculated and applied to the first fundus image and the second fundus image to achieve alignment of the key points of the retinal vascular network.

[0009] In the first and second fundus images after applying the spatial transformation function, a multi-scale image decomposition operation is performed to obtain image frequency sub-bands at different scales;

[0010] For multiple image frequency sub-bands at different scales, a propagation path is formed by linking the local extreme points of signal intensity in the image frequency sub-bands across scales, and candidate lesion seed points are determined in the first fundus image and the second fundus image respectively;

[0011] Starting from the candidate lesion seed point, growth is performed based on the signal intensity gradient direction of the image frequency sub-band, and the outline of the lesion region is defined in the first fundus image and the second fundus image, respectively.

[0012] The contour of the lesion region is mapped to the retinal vascular topology space. By analyzing the relationship between the topological subgraph corresponding to the contour and the vascular segment, and comparing the consistency of the structure and spatial relationship of the topological subgraphs between the first fundus image and the second fundus image, a matching link for the lesion region is established.

[0013] Optionally, based on key points of the retinal vascular network in the first fundus image and the second fundus image, a spatial transformation function is calculated, and the spatial transformation function is applied to the first fundus image and the second fundus image to achieve alignment of key points of the retinal vascular network, including:

[0014] Based on the bifurcation points of retinal vessels in the first fundus image and the second fundus image, the topological connection relationship of the bifurcation points of the retinal vessels is determined;

[0015] Stable reference bifurcation points are selected from the retinal vessel bifurcation points based on the topological connectivity.

[0016] The spatial transformation function is calculated based on the stable reference bifurcation point, and the spatial transformation function is used to register the first fundus image and the second fundus image to achieve alignment of key points of the retinal vascular network.

[0017] Optionally, in the first fundus image and the second fundus image after applying the spatial transformation function, a multi-scale image decomposition operation is performed to obtain image frequency sub-bands at different scales, including:

[0018] For the first fundus image and the second fundus image after applying the spatial transformation function, downsampling operation is performed according to multiple preset scale levels to obtain multi-level images with decreasing resolution;

[0019] Based on the multi-level images, the signal intensity difference between corresponding pixels in adjacent level images is calculated to obtain the difference image corresponding to each level.

[0020] Based on the intensity variation regions in each difference image, the associated image patches are located from the original images of the corresponding levels;

[0021] By analyzing the variation pattern of pixel signal intensity within the image block, the feature components characterizing the main intensity variation pattern are determined.

[0022] The feature components corresponding to adjacent image blocks in the same level are combined to form image frequency sub-bands corresponding to each level;

[0023] By summing up the image frequency subbands corresponding to all levels, we can obtain image frequency subbands at different scales.

[0024] Optionally, for multiple image frequency sub-bands at different scales, a propagation path is formed by linking local extrema of signal intensity in the image frequency sub-bands across scales, and candidate lesion seed points are determined in the first fundus image and the second fundus image, respectively, including:

[0025] In multiple image frequency sub-bands at different scales, local extreme points of signal intensity are detected. These local extreme points include first-type points where the signal intensity value is a local maximum and second-type points where the signal intensity value is a local minimum.

[0026] Starting from the local extremum point detected in the coarsest-scale image frequency sub-band, the corresponding position of the local extremum point in the adjacent fine-scale image frequency sub-band is traced to obtain the cross-scale propagation path.

[0027] Based on the consistency of signal intensity values ​​at all local extreme points along the propagation path, target endpoints with continuous and stable propagation paths across multiple scales are selected.

[0028] The coordinates of the target endpoint are mapped to the corresponding pixel positions in the first fundus image and the second fundus image to obtain candidate lesion seed points.

[0029] Optionally, starting from the candidate lesion seed point, growth is performed based on the signal intensity gradient direction of the image frequency sub-band to define the contour of the lesion region in the first fundus image and the second fundus image, respectively, including:

[0030] Starting from the candidate lesion seed point, based on the gradient direction of the pixel signal intensity around the candidate lesion seed point in the finest scale image frequency sub-band, the process extends to adjacent pixels with the same gradient direction.

[0031] When the expansion of the candidate lesion seed point encounters a pixel where the gradient direction changes abruptly or the signal strength is lower than a preset threshold, the expansion stops and the pixel is marked as a boundary point.

[0032] Repeat the expansion operation until all possible growth directions originating from the candidate lesion seed point have been traversed, and connect all marked boundary points to form a closed contour.

[0033] The closed contour is mapped back to the original coordinates of the first and second fundus images to serve as the contour of the lesion region.

[0034] Optionally, the contour of the lesion region is mapped to the retinal vascular topological space. By analyzing the relationship between the topological subgraph corresponding to the contour and the vascular segment, and comparing the consistency of the structure and spatial relationship of the topological subgraphs between the first fundus image and the second fundus image, a matching link for the lesion region is established, including:

[0035] The outline of the lesion area is superimposed onto the corresponding retinal vascular topology space to obtain associated topology data containing the positional relationship between the outline and the vascular segment;

[0036] Based on the associated topology data, blood vessel segments located inside the contour are extracted, and a topology subgraph is constructed according to the interconnection relationship between the blood vessel segments.

[0037] Based on the contours and associated topological data in the first fundus image and the second fundus image, a first topological sub-graph and a second topological sub-graph are extracted respectively;

[0038] Calculate the structural similarity between the first topological subgraph and the second topological subgraph, and compare the relative positions of the first topological subgraph and the second topological subgraph in their respective retinal vascular networks;

[0039] Based on the structural similarity and the consistency of the relative positions, the lesion region in the first fundus image is paired with the lesion region in the second fundus image to establish a matching link for the lesion region.

[0040] Optionally, based on the associated topology data, blood vessel segments located within the contour are extracted, and a topological subgraph is constructed according to the interconnections between the blood vessel segments, including:

[0041] From the associated topology data, blood vessel segments that are completely located inside the contour are selected as internal blood vessel segments;

[0042] The connection point is determined based on the spatial clustering relationship of the endpoint coordinates of the internal vascular segments;

[0043] Based on the connection relationship between the internal blood vessel segment and the connection point, an initial graph structure is constructed, wherein the initial graph structure has the connection point as the vertex and the internal blood vessel segment as the edge.

[0044] Based on the initial graph structure, the attribute information of all internal blood vessel segments belonging to the same connection point is fused to form a topological subgraph.

[0045] Secondly, this application provides a medical image matching device, comprising:

[0046] The acquisition module is used to acquire first and second fundus images at different time points. The first and second fundus images are digital fundus images containing lesion areas.

[0047] The calculation module is used to calculate a spatial transformation function based on the key points of the retinal vascular network in the first fundus image and the second fundus image, and apply the spatial transformation function to the first fundus image and the second fundus image to achieve alignment of the key points of the retinal vascular network.

[0048] The decomposition module is used to perform multi-scale image decomposition operations on the first fundus image and the second fundus image after applying the spatial transformation function to obtain image frequency sub-bands at different scales;

[0049] The determination module is used to form a propagation path by linking the local extreme points of signal intensity in the image frequency sub-bands at different scales, and to determine the candidate lesion seed points in the first fundus image and the second fundus image respectively.

[0050] The delineation module is used to grow based on the signal intensity gradient direction of the image frequency sub-band, starting from the candidate lesion seed point, and to delineate the outline of the lesion region in the first fundus image and the second fundus image respectively.

[0051] A module is established to map the contour of the lesion region to the retinal vascular topology space. By analyzing the relationship between the topological subgraph corresponding to the contour and the vascular segment, and comparing the consistency of the structure and spatial relationship of the topological subgraphs between the first fundus image and the second fundus image, a matching link for the lesion region is established.

[0052] Thirdly, this application provides an electronic device, comprising:

[0053] Memory, used to store computer programs;

[0054] A processor, configured to execute the computer program to implement the steps of the medical image matching method as described in the first aspect above.

[0055] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the medical image matching method described in the first aspect above.

[0056] The beneficial effects of this application are:

[0057] The medical image matching method provided in this application can provide basic data for subsequent temporal comparative analysis by acquiring fundus images containing lesion areas at different time points; by calculating and applying spatial transformation functions based on key points of the retinal vascular network to achieve alignment, it can eliminate the overall positional deviation between images caused by differences in shooting angle and time, and establish a unified spatial benchmark for local lesion comparison; by performing multi-scale decomposition on the aligned images to obtain image frequency sub-bands at different scales, it can simultaneously capture and separate lesion feature information in the images from multiple levels, from overall structure to local details; by linking local extreme points of signal intensity in the image frequency sub-bands across scales to form a propagation path and determine candidate lesion seed points, it can utilize the stability characteristics of lesions at multiple scales to accurately locate the starting core position of suspected lesions; by using candidate lesion seed points as the starting point and growing based on the signal intensity gradient direction of the image frequency sub-bands to define the lesion outline, it can automatically and accurately delineate the complete shape boundary of each lesion area.

[0058] Furthermore, the lesion region contours obtained from the two images are mapped onto a stable topological space composed of retinal vessels. By analyzing and comparing the consistency in morphology and relative position of the local vascular connections associated with each contour, regions belonging to the same lesion in images at different time points are identified and linked. By utilizing the stable and personalized topological structure of the retinal vascular network as a high-level, robust feature reference system, the direct matching difficulties caused by potential changes in the size, shape, or brightness of the lesions themselves can be effectively overcome. This enables the establishment of reliable and accurate correspondences between lesions across time points, improving the robustness and accuracy of the matching method. Attached Figure Description

[0059] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 A flowchart illustrating a medical image matching method provided in an embodiment of this application;

[0061] Figure 2 A schematic diagram illustrating a specific implementation of another medical image matching method provided in this application embodiment;

[0062] Figure 3 This is a schematic diagram illustrating a specific implementation of a medical image matching method provided in this application. Detailed Implementation

[0063] Existing technologies for overall image alignment based on blood vessel bifurcation points have a core limitation: they mainly focus on the consistency of global spatial location. However, in the aligned image, the lesion area itself may change due to changes in shape, brightness, or size, making reliable matching impossible based solely on the aligned pixel position. Therefore, there is a problem of insufficient accuracy in matching lesion areas across time points.

[0064] To address this issue, this application provides a matching method based on the fusion of multi-scale lesion features and vascular topological relationships. The core of this method involves independently and accurately extracting the lesion region contour from each image through multi-scale decomposition, cross-scale feature linking, and local growth techniques. Each contour is then placed within its highly individualized and stable retinal vascular network for localization and description. A unique topological "fingerprint" is formed by analyzing the connectivity and relative positional structure between the lesion contour and surrounding vessels. The similarity of this topological fingerprint is used to determine whether lesions in different images correspond to the same entity. This method effectively bypasses the limitations of directly relying on pixel comparison or simple spatial location matching. Even if the lesion's appearance or location changes slightly due to disease progression, reliable matching can be achieved through its stable topological association with a fixed vascular network, thus solving the core problem of insufficient lesion matching accuracy.

[0065] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0066] The core of this application is to provide a method for matching medical images, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0067] S101. Collect first and second fundus images at different time points. The first and second fundus images are digital fundus images containing the lesion area.

[0068] In one specific implementation, a dedicated fundus imaging device, such as a fundus camera, is used to take pictures of the same eye of the patient at two preset time points by adjusting the device parameters and the patient's position, thereby obtaining a first fundus image and a second fundus image containing the lesion area. The lesion area refers to the pathological changes that appear abnormal in shape, color or structure in these images, such as hemorrhages, exudates, microaneurysms or neovascularizations. These areas are key observation targets for assessing disease progression or treatment response.

[0069] S102. Based on the key points of the retinal vascular network in the first fundus image and the second fundus image, calculate the spatial transformation function and apply the spatial transformation function to the first fundus image and the second fundus image to achieve alignment of the key points of the retinal vascular network.

[0070] Among them, key points of the retinal vascular network refer to specific locations on the retinal vascular structure in fundus images that have significant features and are easy to repeatedly identify, such as the bifurcation or intersection of blood vessels. These key points are often used as markers for image alignment because of their structural stability and high individual variability.

[0071] In this embodiment of the application, S102 specifically includes:

[0072] S1021. Based on the bifurcation points of retinal vessels in the first fundus image and the second fundus image, determine the topological connection relationship of the bifurcation points of retinal vessels.

[0073] Among them, the retinal vascular bifurcation point is the intersection where a blood vessel in the retinal vascular network divides into two or more branches, and it is one of the key feature points of the vascular network; the topological connection relationship describes how the bifurcation points are interconnected through vascular segments to form a network structure.

[0074] In this embodiment, firstly, the first and second fundus images are processed using a blood vessel segmentation algorithm, such as threshold segmentation, to extract the retinal vascular network; secondly, computer image analysis technology is used to identify all the bifurcation points of retinal vessels from the extracted retinal vascular network; finally, path tracing technology is used to trace the vascular segments connecting these bifurcation points and establish a topological connection relationship describing how the bifurcation points are interconnected through vascular paths.

[0075] S1022. Based on the topological connectivity, select stable reference bifurcation points from the bifurcation points of retinal vessels.

[0076] A stable reference bifurcation point refers to a bifurcation point that can be clearly identified in both images, and whose topological connection relationship, such as the number of branches, the relative angle of the branches, and the thickness of the branches, remains consistent in both images.

[0077] In this embodiment of the application, by comparing the topological connectivity between the first fundus image and the second fundus image using a feature comparison method, it is found that not all bifurcation points are suitable for alignment. Based on the stability of the topological structure, bifurcation points that are consistent and clear in both images in terms of the number and direction of connected blood vessel branches are preferentially selected as reference bifurcation points. Bifurcation points that are clearly visible in one image but whose connectivity is blurred or changed in another image due to imaging quality, lesion occlusion, or minor changes in the shape of the blood vessel itself will be excluded.

[0078] S1023. Calculate the spatial transformation function based on the stable reference bifurcation point, and use the spatial transformation function to register the first fundus image and the second fundus image to achieve alignment of key points of the retinal vascular network.

[0079] In this embodiment, based on a stable reference bifurcation point, a spatial transformation function is calculated using a point matching algorithm, which determines the parameters required for affine transformation or perspective transformation. The obtained spatial transformation function is then applied to the first fundus image and the second fundus image. Pixel resampling or geometric transformation techniques in image registration are used to perform rotation, translation, or scaling operations on the image, ultimately achieving alignment of key points in the retinal vascular network.

[0080] As an example, firstly, through step 1021, using the blood vessel segmentation and feature point detection algorithm, 50 blood vessel bifurcation points are extracted from the first fundus image taken by patient A in the first year, and 48 blood vessel bifurcation points are extracted from the second fundus image taken in the second year, and their respective blood vessel connection relationship diagrams are established.

[0081] Next, through step 1022, the two sets of points and their connection relationship diagrams are matched and compared. It is found that there are 42 pairs of bifurcation points. Their connection patterns are highly consistent in the two images. For example, they are all "three-way" bifurcations, and the thickness and direction of the three branches are similar. Therefore, these 42 pairs of points are determined as stable reference bifurcation points.

[0082] Finally, in step 1023, a spatial transformation function is calculated using the coordinates of these 42 pairs of stable reference bifurcation points as the correspondence. Assuming a similarity transformation model is used, this model can be expressed in the following mathematical form:

[0083] Let the coordinates of a stable reference bifurcation point from the first fundus image be... The coordinates of the matching point in the second fundus image are Similarity transformation attempts to find a set of parameters such that the transformed points and To make it as close as possible, the transformation formula is:

[0084] ;

[0085] in, Indicates the scaling factor. Indicates the rotation angle. and Let E represent the translation in the horizontal and vertical directions. The parameters are solved using the least squares method to minimize the total error of all matching point pairs after the above transformation, i.e., minimizing the following objective function E:

[0086] ;

[0087] in, Represents the number of stable reference fork pairs, superscript Representing the For each point, find the set of parameters that minimizes E. The optimal spatial transformation function was obtained. After applying this specific transformation function to adjust the second fundus image as a whole, the image rotation caused by the slight head tilt during shooting was corrected, and the vascular networks in the two images achieved precise overlap, laying the foundation for subsequent accurate comparison of lesion changes at the same location. The above example is only one example of this application. In practical applications, the number of bifurcation points, matching algorithm, and transformation model can be adjusted according to image quality and specific needs, and this application does not limit this.

[0088] This application utilizes the topological connectivity of key points in the retinal vascular network to screen stable corresponding points and calculates a global spatial transformation function based on these points, achieving high-precision and robust registration of fundus images. This effectively eliminates geometric distortion caused by differences in imaging and lays a reliable foundation for subsequent accurate tracking and quantitative analysis of lesions.

[0089] S103. In the first and second fundus images after applying the spatial transformation function, perform multi-scale image decomposition to obtain image frequency sub-bands at different scales.

[0090] Multiscale image decomposition is an image processing method that simulates the human eye's cognitive process of observing objects from coarse to fine, decomposing a complete image into multiple expressions with different levels of detail (i.e., scales). This includes lesion regions of different sizes, such as large-area hemorrhages and small-scale microaneurysms, as well as image information of different frequencies, such as smooth backgrounds and abrupt blood vessel edges, distributed across different scales. Image frequency subbands refer to the components of image information obtained after decomposition that contain specific frequency ranges or levels of detail. For example, one subband may mainly contain large-scale, slowly changing brightness information, i.e., low frequency, while another subband may mainly contain subtle edge and texture information, i.e., high frequency.

[0091] In this embodiment of the application, S103 specifically includes:

[0092] S1031. For the first and second fundus images after applying the spatial transformation function, downsampling operations are performed according to multiple preset scale levels to obtain multi-level images with decreasing resolution.

[0093] Downsampling refers to the process of reducing the resolution of an image by decreasing the number of pixels, for example, reducing the resolution of an image by subtracting the number of pixels. The image of pixels is reduced to Pixels; multi-level images refer to a series of image copies with continuously reduced resolution obtained through multiple downsampling. They together form an image pyramid structure, with the top layer (lowest resolution) reflecting the most general information of the image and the bottom layer (original resolution) retaining the finest details.

[0094] In this embodiment, a set of scale level parameters is first set. For example, level 1 is the original image, level 2 is an image whose length and width are each reduced by half, level 3 is an image reduced by half again based on level 2, and so on. Then, an image interpolation algorithm, such as bilinear interpolation or Gaussian smoothing followed by resampling, is used to perform downsampling operations on the aligned first and second fundus images to generate their respective multi-level images. For example, the original image is... Pixels, obtained after one downsampling The image of pixels is at level 2, and then it is downsampled once more. The image of pixels is at level 3.

[0095] S1032. Based on multi-level images, calculate the signal intensity difference between corresponding pixels in adjacent level images to obtain the difference image corresponding to each level.

[0096] Among them, the difference image is an image obtained by calculating the difference in brightness values ​​of corresponding pixels in the same image at different resolution levels, reflecting the image details that are added or lost from a coarser scale to a finer scale.

[0097] In this embodiment, based on the image pyramid, for each pair of adjacent levels, such as a low-resolution image of level 2 and a high-resolution image of level 1, the lower-level image needs to be upsampled and enlarged to restore it to the same size as the higher-level image. Then, for the two images that are now the same size, the difference between their grayscale values ​​or color channel values ​​is calculated pixel by pixel, and the difference (usually the absolute value) is saved as a new image, that is, the difference image corresponding to the level, which essentially captures the detail changes that were smoothed out by the downsampling operation or ignored by the previous layer.

[0098] S1033. Based on the intensity change region in each difference image, locate the associated image patch from the original image of the corresponding level.

[0099] Among them, the intensity change region refers to the connected region in the difference image where the pixel value is significantly higher than the background (i.e., the difference is large). These regions identify the image locations where obvious detail changes occur at the current scale. The image patch refers to a rectangular pixel region extracted from the original image or the approximate image restored by appropriate interpolation at the corresponding level, centered on or corresponding to the intensity change region.

[0100] In this embodiment, firstly, a threshold segmentation technique is used to mark pixels in each difference image whose difference is greater than a certain empirical threshold, forming a binary mask; then, a connected component analysis algorithm is used to aggregate these marked pixels into several independent intensity variation regions. For each identified region, based on its bounding box, a rectangular sub-image containing the region and a certain range of pixels around it is extracted from the corresponding original layer image that has not undergone difference calculation, for example, the higher resolution image in the pair of images used to calculate the difference. This sub-image is the image patch.

[0101] S1034. By analyzing the variation law of pixel signal intensity within the image block, the feature components representing the main intensity variation pattern are determined.

[0102] Among them, the feature component refers to the mathematical expression that can summarize and represent the main gray-level or structural change trend within an image block. It can be understood as a set of basic patterns. The content of the image block can be approximated as being composed of these basic patterns combined with different weights.

[0103] In this embodiment, the grayscale values ​​of all pixels in an image patch are first arranged into a column vector; then, the vectors corresponding to all image patches at the current scale are collected to form a data matrix; next, principal component analysis is performed on the data matrix to solve for its eigenvalues ​​and eigenvectors. Among them, the top few eigenvectors with the largest eigenvalues ​​(i.e., principal components) represent the most common and dominant intensity change patterns in the image patch at this scale, and these principal components are the feature components.

[0104] S1035. Combine the feature components corresponding to adjacent image blocks in the same level to form the image frequency sub-band corresponding to each level.

[0105] In this embodiment of the application, for all image blocks at the same scale level, according to the geographical location of these image blocks in the original image, their corresponding projection coefficients or local images reconstructed from these coefficients are filled into a blank image that matches the size of the original image; during filling, the information of adjacent image blocks can be smoothly transitioned at the boundary, for example by weighted averaging; finally, such a global map is generated for each major feature component, and each such global map constitutes an image frequency sub-band at that scale level.

[0106] S1036. Summarize the image frequency subbands corresponding to all levels to obtain image frequency subbands at different scales.

[0107] In this embodiment, the image frequency sub-bands generated from each scale level are collected in scale order to form a complete set. This set contains image frequency sub-bands at different scales, which together constitute a multi-level, multi-frequency decomposition representation of the original fundus image.

[0108] As an example, firstly, in step 1031, the two aligned fundus images of patient A are processed, setting three scale levels, for example, level Original image ,level Image that is halved in both length and width ,level In order to be in The image is reduced by half. And so on, using a Gaussian smoothing kernel. The operation of convolutionally blurring an image and then performing interval sampling can be represented as follows:

[0109] ;

[0110] in, Indicates the first Layer images, This indicates a downsampling operation with a factor of 2 in both the width and height directions. * represents a convolution operation. The standard deviation is A two-dimensional Gaussian kernel function is used to smooth the image. By repeating this process, multiple layers of images with decreasing resolution are generated. .

[0111] For example, the original image for Pixels, using standard deviation After smoothing with a Gaussian kernel, downsampling with a factor of 2 is performed to obtain... Pixel image ,right Repeat this operation to obtain Pixel image .

[0112] Next, through step 1032, Upsampling Pixels obtained , and then with Pixel-by-pixel subtraction: Take its absolute value to generate a difference image representing the finest details. Similarly, Upsampling to Pixels obtained ,and Subtraction: To obtain the difference image It represents a moderate level of detail variation.

[0113] Next, in step 1033, the image patch is located, and an intensity threshold is set. For difference images Perform binarization processing, The intensity value of each pixel in the middle and In comparison, pixels with a difference intensity greater than 15 are set to white, representing areas of significant change; pixels with a difference intensity less than or equal to 15 are set to black, representing areas of insignificant change. Assuming a connected region where all white pixels have x-coordinates between 200 and 250 and y-coordinates between 300 and 350, this rectangular region... This is defined as an intensity variation region, which identifies locations in the image where significant changes in detail occur. Returning to the original image at the highest resolution level 0 generated in step 1031... Using the center of the hypothetical intensity variation region as a reference, from the original image Extract the portion centered on that region, with a size of A rectangular area of ​​pixels; this rectangular area is the image block containing the original image information. .

[0114] Then, in step 1034, the feature components are determined: assuming from the current scale... 100 images were collected from all the image patches. These image patch vectors constitute a data matrix. Principal component analysis was used to analyze... Perform analysis, and divide each The grayscale values ​​of all pixels in an image patch are arranged sequentially, transforming it into a sequence containing 4096 values. This sequence is called an image patch vector, where 4096 is a number. The total number of pixels, combined by stacking 100 such image patch vectors row-wise, forms a data matrix with 100 rows and 4096 columns. Principal component analysis is then performed on this data matrix. The core of the calculation is to find the principal directions that best represent the overall direction of change of these 100 image patches. Assuming that the first two largest eigenvectors obtained through the calculation... and These represent the two main grayscale variation patterns in the image patch at this scale, among which... This might correspond to an edge pattern that transitions from dark to light. This might correspond to a bright spot pattern at the center. Therefore, any specific image patch can be considered as a mixture of these two basic patterns in different proportions. This mixing proportion is the projection coefficient. Assuming the image patch is obtained through calculation... exist and The projection coefficients on the surface are 0.8 and -0.3, respectively. Described It is mainly composed of edge patterns, with a small amount of inverted speckle pattern components.

[0115] Then, in step 1035, these local, discrete feature information are integrated into a global map that can completely reflect the distribution of the feature across the entire image plane, i.e., an image frequency sub-band. Assuming that for all image patches extracted at level 0, a map with the exact same size as the original image at that level is created... Given a blank matrix graph, iterate through each image patch, assuming the image patch... It is extracted from a certain position in the original image, so the corresponding coordinates of its center point on the blank image can be determined. A similar operation is performed on all image patches to find the coordinates of all image patches in the first principal component. The projection coefficients on the image are mapped back to the center position of each image patch. The corresponding pixel positions in the blank matrix image are used to form a pattern composed of many discrete points on the blank matrix image. The position of each point represents the center of an image patch, and the value of the point represents the position of the image patch in the first principal component. The intensity of the pattern; however, there are still many gaps between these discrete points. In order to generate a continuous and smooth feature distribution map, an interpolation algorithm is used to estimate and fill the value of each pixel in the gap area based on the values ​​of the surrounding known points, and then perform interpolation smoothing to obtain a complete and continuous image, thus forming level 0 based on the first principal component. Image frequency subband Similarly, using the second principal component The projection coefficients on the top, and the above process is repeated to form level 0 based on the first principal component. Image frequency subband .

[0116] Finally, step 1036 is used to repeat the above process for layers 2 and 3 to obtain their respective image frequency sub-bands. For example, layer 2 has sub-bands. , Level 3 has sub-bands Wait, and then all the sub-bands After summarizing, a set of image frequency subbands covering multiple scales is obtained. The above example is only one example of this application. In practical applications, the number of scale levels, downsampling factor, difference calculation method, feature extraction method, etc., can be set according to requirements, and this application does not limit them.

[0117] This application systematically decomposes registered fundus images into multi-scale segments, separating image information into sub-bands of different frequencies and levels of detail. This allows subsequent analysis to independently and meticulously examine lesion features of different sizes and image components of different frequencies, effectively improving the comprehensiveness of lesion feature capture and the specificity of analysis, and laying the foundation for highly sensitive lesion change detection.

[0118] S104. For multiple image frequency sub-bands at different scales, a propagation path is formed by linking the local extreme points of signal intensity in the image frequency sub-bands across scales, and candidate lesion seed points are determined in the first fundus image and the second fundus image, respectively.

[0119] Among them, a local extreme point refers to the pixel location in a certain image region, such as the neighborhood of a pixel, where the signal intensity value reaches the local highest point (maximum value) or the local lowest point (minimum value); a candidate lesion seed point refers to the pixel location that is finally located on the image at the finest scale after the above cross-scale verification, serving as the starting point of the potential lesion region.

[0120] In the embodiments of this application, such as Figure 2 As shown, S104 specifically includes:

[0121] S1041. In multiple image frequency sub-bands at different scales, detect local extreme points of signal intensity. Local extreme points include first-class points where the signal intensity value is a local maximum and second-class points where the signal intensity value is a local minimum.

[0122] In this embodiment, in multiple image frequency sub-bands at different scales, a sliding window comparison algorithm is used to compare the signal strength value of each pixel in each image frequency sub-band with the values ​​of all neighboring pixels within a fixed-size window. If the value of the pixel is greater than the values ​​of all other pixels within the window, it is marked as a first-class point, i.e., a local maximum point; if it is less than the values ​​of all other pixels, it is marked as a second-class point, i.e., a local minimum point, thereby detecting all local extreme points.

[0123] S1042. Starting from the local extremum points detected in the coarsest-scale image frequency sub-band, trace the corresponding positions of the local extremum points in the adjacent fine-scale image frequency sub-bands to obtain the cross-scale propagation path.

[0124] The adjacent fine scale refers to the next scale level with higher resolution and richer details than the current scale.

[0125] In this embodiment, starting with the local extrema in the coarsest-scale image frequency sub-band, a cross-scale tracking algorithm is used to map the coordinates of each local extrema in the coarsest-scale sub-band to the approximate area of ​​the next adjacent finer-scale sub-band according to the scaling ratio of the image pyramid. Within the corresponding area of ​​the finer-scale sub-band, for example, a small search window, the algorithm searches for points with similar signal strengths (i.e., both being maximum or minimum values) and the closest positions. These points are then identified as the corresponding points. This process is repeated step by step from coarse to fine between adjacent scale sub-bands, connecting the corresponding points found at each scale to form a cross-scale propagation path from the coarsest scale to the finest scale.

[0126] S1043. Based on the consistency of signal intensity values ​​at all local extreme points along the propagation path, select target endpoints with continuous and stable propagation paths across multiple scales.

[0127] In this embodiment of the application, after obtaining all propagation paths, a path filtering algorithm is used to check the signal strength values ​​of all local extreme points contained in each propagation path, calculate the change pattern of these strength values, for example, determine whether they all maintain high or low values, whether the fluctuation is very small, filter out those propagation paths where the intensity change patterns of all points on the path are consistent and the path is uninterrupted from coarse scale to fine scale, and determine the endpoint of these paths at the finest scale as a stable and continuous target endpoint.

[0128] S1044. Map the coordinates of the target endpoint to the corresponding pixel positions in the first and second fundus images to obtain candidate lesion seed points.

[0129] Mapping refers to converting the coordinates of the target endpoint on the finest-scale sub-band image back to the original input image, i.e., the coordinates in the pixel coordinate system of the first or second fundus image.

[0130] In this embodiment of the application, after obtaining the selected target endpoints, they are processed by inverse coordinate transformation. These target endpoints are the pixel coordinates located in the frequency sub-band of the finest scale image. Based on the downsampling ratio used in multi-scale image decomposition, the coordinate positions of these points are accurately transformed back to the corresponding pixel positions in the original size first fundus image and second fundus image through coordinate scaling calculation. These points located in the original image are the candidate lesion seed points.

[0131] As an example, firstly, through step 1041, in the set of image frequency subbands obtained by processing the first fundus image of patient A, for the coarsest scale... a certain sub-band ,use A local maximum point was detected in the neighborhood. Assuming The coordinates are (64, 80), and the intensity value is 120.

[0132] Next, through step 1042, to Starting from this point, cross-scale tracking will be performed. Multiplying the coordinates by a scale factor of 2 (where the scale factor can be set as needed) yields the results at adjacent finer scales. The expected position (128, 160) on the sub-band Above, centered at (128, 160), Within the search window, find the local maximum point. Assume a point is found. Its coordinates are (129, 162), and its intensity value is 118. Starting from this point, continue towards the finest scale. Tracking, will Multiplying the coordinates by 2 gives (258, 324), which is in the sub-band. Find the point in the corresponding search window The coordinates are (260, 325), and the intensity value is 115. This forms a line from... arrive Then The transmission path.

[0133] Finally, through step 1044, due to the subband Same size as the registered first fundus image Target End The coordinates (260, 325) are directly mapped to a candidate lesion seed point in the first fundus image. Performing the same process on the second fundus image will also yield a candidate seed point near its corresponding location. The above example is merely one illustration of this application. In practical applications, parameters such as neighborhood size, search window size, and consistency evaluation criteria can be set according to image resolution and features; this application does not impose any limitations on these settings.

[0134] This application achieves high-precision and robust automatic localization of candidate lesion seed points by detecting local extrema points and tracing cross-scale paths on multi-scale frequency sub-bands, and by filtering based on path continuity and signal consistency. This provides a reliable and high-quality starting input for subsequent refined analysis.

[0135] S105. Starting from the candidate lesion seed point, growth is performed based on the signal intensity gradient direction of the image frequency sub-band, and the outline of the lesion region is defined in the first fundus image and the second fundus image respectively.

[0136] Among them, the signal intensity gradient direction refers to the direction in which the signal intensity changes the fastest at each pixel in the image frequency sub-band. It points to the direction in which the signal intensity increases most significantly, and is usually perpendicular to the edge or boundary of the region in the image. At the edge of the lesion region, the gradient direction often points to the inside or outside of the lesion, reflecting the expansion trend of the region. The consistent gradient direction means that the gradient directions of adjacent pixels are very small and tend to be parallel, indicating that they may belong to the same uniformly expanding or contracting region. The abrupt change in gradient direction means that the gradient direction angle of adjacent pixels suddenly becomes very large, such as approaching perpendicularity. This usually occurs at the junction of different regions, i.e., the boundary position.

[0137] In this embodiment of the application, S105 specifically includes:

[0138] S1051. Starting from the candidate lesion seed point, expand to adjacent pixels with the same gradient direction based on the gradient direction of the pixel signal intensity around the candidate lesion seed point in the image frequency sub-band at the finest scale.

[0139] Among them, the finest-scale image frequency sub-band retains the richest image details and precise edge information; the gradient direction indicates the direction in which the signal intensity increases most significantly from the current pixel. For lesion areas, this usually points to the inside of the area, i.e., for bright lesions, or the outside of the area, i.e. for dark lesions. Extending along this direction helps to include pixels belonging to the same lesion.

[0140] In this embodiment, firstly, starting from the candidate lesion seed point, the initialization operation of the region growing algorithm is performed to place the candidate lesion seed point into a set of points to be processed. For the finest scale image frequency sub-band, the gradient operator is used to calculate the signal intensity gradient direction of the seed point and all its neighboring pixels. Starting from the seed point, the surrounding neighboring pixels are checked, and those pixels that are consistent with the gradient direction of the seed point are selected, for example, whose direction angle difference is less than a certain small angle. These pixels are added to the growing region and placed into the set of points to be processed for subsequent expansion.

[0141] S1052. When the expansion of candidate lesion seed points encounters pixels where the gradient direction changes abruptly or the signal intensity is lower than a preset threshold, the expansion is stopped and the pixels are marked as boundary points.

[0142] The preset threshold is a pre-defined signal intensity threshold used to distinguish between lesion feature areas and background or weak feature areas.

[0143] In this embodiment of the application, when a point is taken from the set of points to be processed for expansion, a boundary judgment condition is used for inspection. For the currently inspected pixel, the gradient direction of its neighboring pixels is recalculated. If it is found that the gradient direction of a certain neighboring pixel has changed abruptly compared with the gradient direction of the current point, for example, the direction angle difference exceeds a set threshold, or the signal strength value of the neighboring pixel is lower than a preset strength threshold, then the expansion in the direction of the neighboring pixel is stopped, and the pixel that causes the expansion to stop is marked as a boundary point.

[0144] S1053. Repeat the expansion operation until all possible growth directions originating from the candidate lesion seed point have been traversed, and connect all marked boundary points to form a closed contour.

[0145] A closed profile is a closed curve formed by connecting boundary points, which completely encloses a region.

[0146] In this embodiment, firstly, the expansion and boundary determination operations are repeated through iterative loops. New points are continuously extracted from the set of points to be processed, and their adjacent pixels are repeatedly checked. The consistency of gradient direction and signal strength are judged. Points that meet the conditions are added to the growth region, and points that cause the process to stop are marked as boundary points. This process continues until the set of points to be processed is empty, which means that all possible directions that meet the growth conditions have been explored from the initial seed point. Finally, a contour connection algorithm is used, for example, to connect the boundary points according to their spatial order, to connect all the marked boundary points in sequence, forming a closed contour that surrounds the growth region.

[0147] S1054. Map the closed contour back to the original coordinates of the first and second fundus images as the contour of the lesion region.

[0148] In this embodiment, after obtaining the closed contour formed in the finest-scale image frequency sub-band, it is processed by coordinate inverse mapping. According to the coordinate transformation relationship from the original image to the finest-scale sub-band during multi-scale decomposition, inverse coordinate calculation is performed to convert the coordinates of each boundary point on the closed contour back to its precise pixel position in the original size first fundus image and second fundus image. This closed shape reconstructed in the original image is finally defined as the contour of the lesion region.

[0149] As an example, firstly, in step 1051, starting from a candidate lesion seed point located at coordinates (260, 325) in the first fundus image of patient A, the corresponding finest-scale image frequency subband is... In the calculation, the gradient direction at that point points downwards and to the right; for example, the angle between the gradient and the horizontal axis is [missing information]. Examining its eight neighboring pixels, we find that the gradient direction of the pixel on the right is... The pixel below is The bottom right pixel is Their gradient directions have an angle smaller than that between them and the seed point. Since they are in the same direction, these three adjacent pixels are included in the growth region.

[0150] Next, in step 1052, using the newly included right-side pixel coordinates (261, 325) as the new leading edge point, its neighborhood is further examined, and it is found that the gradient direction of the right-side pixel coordinates (262, 325) is... , and the current cutting-edge direction The included angle is much greater than The mutation threshold is used to determine if a sudden change occurs in the gradient direction. Therefore, growth to the right stops at pixel (262, 325), and this pixel is marked as a boundary point. At the same time, when checking the pixels below, it is found that their signal strength value is 25, which is lower than the preset strength threshold, such as 30. Therefore, growth also stops at this point, and the boundary point is marked.

[0151] Next, through step 1053, the above expansion and judgment process is repeated continuously, growing from the seed point in all directions. All possible growth paths stop due to encountering gradient mutations or low intensity, forming a circle of marked boundary points. Using the boundary tracing algorithm, starting from the boundary point (262,325), all boundary points are connected in a clockwise direction, eventually forming a closed contour with the beginning and end connected.

[0152] Finally, through step 1054, due to the sub-band The coordinate system is completely consistent with the registered first fundus image coordinate system. By directly copying the coordinates of all points of the closed contour onto the first fundus image, the contour of the lesion region on that image is obtained. Performing the same growth and mapping process on the second fundus image will also yield a contour in its corresponding region. The above example is only one example of this application. In practical applications, parameters such as gradient operator selection, angle tolerance, mutation threshold, and intensity threshold can be adjusted according to specific image features, and this application does not limit them.

[0153] This application guides regional growth by starting from a reliable seed point and utilizing the gradient direction of the finest-scale feature subbands. It also combines gradient mutation and intensity threshold as intelligent criteria for growth cessation. This approach can adaptively and accurately delineate the complete outline of the lesion region, effectively distinguishing lesion tissue from surrounding healthy tissue and providing accurate spatial definition for subsequent quantitative analysis and change detection.

[0154] S106. Map the contour of the lesion area to the retinal vascular topological space. By analyzing the relationship between the topological sub-graphs corresponding to the contour and the vascular segments, compare the consistency of the structure and spatial relationship of the topological sub-graphs between the first fundus image and the second fundus image, and establish a matching link for the lesion area.

[0155] The retinal vascular topology space refers to a graph or network model abstracted from the retinal vascular network, which focuses on the connection relationships between vascular segments rather than their precise geometric shapes. In this space, the bifurcation points or endpoints of blood vessels are usually represented as points, and the vascular segments are represented as lines connecting these points. The topological subgraph refers to a local subnetwork extracted from the complete retinal vascular topology network, specifically referring to the local connection graph formed by blood vessels located inside the outline of a certain lesion area.

[0156] In this embodiment of the application, S106 specifically includes:

[0157] S1061. Overlay the contour of the lesion area onto the corresponding retinal vascular topology space to obtain associated topology data containing the positional relationship between the contour and the vascular segment.

[0158] Among them, the associated topology data refers to the data set that records which blood vessel segments are located inside the contour, which are outside, and their positional relationship with the contour boundary.

[0159] In this embodiment of the application, the lesion region contour is placed into the pre-constructed retinal vascular topology space through coordinate superposition operation. Through geometric calculation processing, the positional relationship between each vascular segment in the retinal vascular topology space and the lesion contour is determined. For example, it is calculated whether the sampling points on the vascular segment are all located inside the contour polygon, thereby obtaining a detailed record of which vascular segments are located within which lesion contour and their relative positions, i.e., associated topology data.

[0160] S1062. Based on the associated topological data, extract the blood vessel segments located inside the contour, and construct a topological subgraph according to the interconnection relationship between the blood vessel segments.

[0161] Specifically, S1062 may include:

[0162] From the associated topological data, blood vessel segments completely located inside the contour are selected as internal blood vessel segments; based on the spatial aggregation relationship of the endpoint coordinates of the internal blood vessel segments, connection points are determined; based on the connection relationship between the internal blood vessel segments and the connection points, an initial graph structure is constructed, with the connection points as vertices and the internal blood vessel segments as edges; based on the initial graph structure, the attribute information of all internal blood vessel segments belonging to the same connection point is fused to form a topological subgraph.

[0163] In this embodiment, firstly, the associated topological data is processed by a filtering algorithm to extract vascular segments whose pixels are completely located inside the lesion outline, which are then identified as internal vascular segments. The endpoint coordinates of these internal vascular segments are analyzed, and spatial clustering algorithms, such as distance-based clustering, are used to group endpoints that are spatially close into the same connection point. Using these connection points as vertices and the internal vascular segments connecting two vertices as edges, an initial graph structure is constructed. This initial graph structure is then optimized by merging or averaging the attributes of all vascular segments belonging to the same connection point, such as average width and direction, to finally form a simplified topological subgraph that can represent the vascular connectivity pattern inside the lesion.

[0164] S1063. Based on the contour and associated topological data in the first fundus image and the second fundus image, extract the first topological sub-graph and the second topological sub-graph respectively.

[0165] The first topological sub-map refers to the topological sub-map extracted from the associated topological data of the first fundus image, corresponding to a certain lesion region in the image; the second topological sub-map refers to the topological sub-map extracted from the associated topological data of the second fundus image, corresponding to a certain lesion region in the image.

[0166] In this embodiment, the contours of each lesion region and their corresponding associated topological data identified in the first fundus image and the second fundus image are processed in the same way. For each lesion in the first fundus image, its corresponding first topological sub-map is extracted; for each lesion in the second fundus image, its corresponding second topological sub-map is extracted.

[0167] S1064. Calculate the structural similarity between the first topological subgraph and the second topological subgraph, and compare the relative positions of the first topological subgraph and the second topological subgraph in their respective retinal vascular networks.

[0168] Structural similarity is a quantitative metric used to measure the similarity of two topological subgraphs in their connection patterns. For example, it can compare whether two subgraphs have the same number of vertices and edges, whether the vertex connectivity matches (e.g., how many edges a vertex connects to, and whether the edge length ratios are similar). Relative position refers to the distance and orientation of the topological subgraph relative to some stable and significant landmarks in the entire retinal vascular network, such as the optic disc center, macular center, and major vascular trunks.

[0169] In this embodiment of the application, any pair of topological subgraphs extracted from the first and second fundus images are processed by a graph similarity calculation algorithm to calculate the structural similarity between them. This usually involves comparing features such as the number of vertices, the number of edges, and the connection method of the two subgraphs. Through spatial relationship calculation, the relative positions of the pair of topological subgraphs in the entire retinal vascular network to which they belong are determined. For example, the distance and angle from the geometric center point of each subgraph to the center of the optic disc are calculated, and these two relative positions are compared.

[0170] S1065. Based on structural similarity and consistency of relative position, the lesion region in the first fundus image is paired with the lesion region in the second fundus image to establish a matching link for the lesion region.

[0171] In this embodiment of the application, firstly, the structural similarity and the relative position information obtained by comparison are combined and processed by setting matching rules or threshold judgment. For example, the rule is set as follows: when the structural similarity of two topological subgraphs is higher than a certain level and their relative positions in their respective vascular networks are spatially close enough, they are considered to have high consistency. Based on this consistency judgment, the lesion area in the first fundus image that meets the conditions is logically paired with the lesion area in the second fundus image, and a matching link is established for the two areas in the data, indicating that they are likely to be the image manifestations of the same lesion at different times.

[0172] As an example, firstly, in step 1061, a lesion outline segmented from the first fundus image of patient A taken in January 2023, assumed to be an approximately circular region, is superimposed on a pre-constructed retinal vascular topological space in the image. This outline is defined by a series of boundary point coordinates, and the vascular topological space contains all vascular segments. The positional relationship between each vascular segment and the outline is determined one by one using the ray method. For example, a vascular segment... The coordinates are (150, 300) for the starting point and (180, 330) for the ending point. Ten points are sampled evenly on this blood vessel segment. It is determined whether all these points are located inside the polygonal outline of the lesion. If the determination result is yes, then... Marked as completely located inside, after traversing all vascular segments, three vascular segments were finally identified as completely located inside the lesion outline. These segments were recorded and, together with the outline information, formed associated topological data.

[0173] Next, in step 1062, based on the aforementioned associated topological data, these three internal vascular segments were selected, and their endpoint coordinates are as follows:

[0174] (150,300) and (180,330);

[0175] (180,330) and (200,310);

[0176] (200,310) and (190,280);

[0177] Set a distance threshold Spatial clustering analysis was performed on all these endpoint coordinates, and it was found that coordinates (180, 330) and (180, 330) are the same point. The end point and The starting points naturally cluster into one category; the coordinates (200, 310) are... The end point and The starting points are clustered into one class; the coordinates (150, 300) are... The starting point and (190, 280) are The endpoints are spatially relatively independent, each forming a separate category, resulting in a total of 4 connection points: (150,300) (180,330) (200,310) (190,280).

[0178] Based on this, construct the initial graph structure: vertices are... ; the edge is { connect and , connect and , connect and The properties of each connection point are fused, for example, the average width and total length of all blood vessel segments connected to that point are calculated, ultimately forming a topological subgraph containing 4 vertices, 3 edges, and their fused properties, denoted as . .

[0179] Next, following step 1063, the exact same operation was performed on the second fundus image of patient A taken in January 2024. In the corresponding region of this image, a lesion outline was also segmented, which may vary slightly depending on changes in the patient's condition. Through overlay and analysis, three vascular segments completely located within this outline were extracted. After endpoint clustering, a topological subgraph consisting of four connection points and three edges was obtained, denoted as... .

[0180] Then, through step 1064, calculate... and Structural similarity between the two subgraphs: First, extract the feature vectors of the two subgraphs, for example: Features: 4 vertices, 3 edges, vertex degree sequence The degree is 1. The degree is 2. The degree is 2. The degree is 1, and the approximate length of the longest path in the graph is 85.

[0181] Features: 4 vertices, 3 edges, vertex degree sequence The degree is 1. The degree is 2. The degree is 2. With a degree of 1, the longest path has an approximate length of 88.

[0182] By calculating the cosine similarity between the two feature vectors, a similarity score is obtained, such as 0.98, which is very close to 1, indicating high similarity. At the same time, their relative positions are compared, and the center of the visual disc is selected as a stable global reference point with coordinates (500, 500).

[0183] calculate The vector from the center point of the optic disc, which is the average of the coordinates of all its vertices, to the center of the optic disc, is assumed to be... .

[0184] calculate The vector from the center point of the disk to the average of the coordinates of all its vertices to the center of the same disk is assumed to be... The difference in the Euclidean distance and the small difference in the orientation angle between the two vectors indicate that their relative positions are highly consistent.

[0185] Finally, through step 1065, because... and The structural similarity between the two topological sub-images is extremely high (e.g., 0.98 > the preset threshold of 0.85), and their relative positional differences are minimal (e.g., distance difference < 5 pixels, angle difference < 5 degrees). Therefore, these two sub-images are determined to correspond to the same local region of an inherent vascular structure on the retina. The corresponding lesion area in the January 2023 image is... The lesion area in the corresponding January 2024 image was identified as the follow-up manifestation of the same lesion in two separate visits. A solid matching link was established between the two, thereby achieving accurate tracking of the lesion across time points.

[0186] The above example is only one example of this application. In practical applications, parameters such as the sampling density for vascular segment inclusion judgment, the distance threshold for connection point clustering, the specific features and calculation methods of structural similarity, and the consistency tolerance of relative position can all be adjusted according to image resolution and clinical accuracy requirements. This application does not limit these parameters.

[0187] This application performs correlation analysis between lesion areas and stable retinal vascular topology networks, using the uniqueness and invariance of the vascular network as a reference framework. By comparing the structural similarity and global relative position of vascular sub-maps within lesions, it achieves accurate and robust matching of lesions across time points, effectively solving the problem of tracking difficulties caused by lesion morphological changes or interference from new lesions, and providing a reliable automated tool for clinical disease monitoring.

[0188] Figure 3 This is a schematic diagram illustrating a specific embodiment of a medical image matching device provided in this application. (Refer to...) Figure 3 The system may include:

[0189] The acquisition module 31 is used to acquire first and second fundus images at different time points. The first and second fundus images are digital fundus images containing the lesion area.

[0190] The calculation module 32 is used to calculate a spatial transformation function based on the key points of the retinal vascular network in the first fundus image and the second fundus image, and apply the spatial transformation function to the first fundus image and the second fundus image to achieve alignment of the key points of the retinal vascular network.

[0191] The decomposition module 33 is used to perform multi-scale image decomposition operations on the first fundus image and the second fundus image after applying the spatial transformation function to obtain image frequency sub-bands at different scales.

[0192] The determination module 34 is used to form a propagation path by linking the local extreme points of signal intensity in the image frequency sub-bands at different scales, and to determine the candidate lesion seed points in the first fundus image and the second fundus image respectively.

[0193] The delineation module 35 is used to grow based on the signal intensity gradient direction of the image frequency sub-band, starting from the candidate lesion seed point, and to delineate the outline of the lesion region in the first fundus image and the second fundus image respectively.

[0194] Module 36 is established to map the contour of the lesion area to the retinal vascular topological space. By analyzing the relationship between the topological subgraph corresponding to the contour and the vascular segment, and comparing the consistency of the structure and spatial relationship of the topological subgraphs between the first fundus image and the second fundus image, a matching link for the lesion area is established.

[0195] The medical image matching device of this application embodiment is used to implement the aforementioned medical image matching method. Therefore, the specific implementation of the medical image matching device can be found in the embodiment section of the medical image matching method above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0196] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the medical image matching method described above.

[0197] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described medical image matching methods.

[0198] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0199] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described embodiments of the medical image matching method.

[0200] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0201] The above provides a detailed description of a medical image matching method and apparatus provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for matching medical images, characterized in that, include: First and second fundus images were acquired at different time points. The first and second fundus images are digital fundus images containing the lesion area. Based on the key points of the retinal vascular network in the first fundus image and the second fundus image, a spatial transformation function is calculated and applied to the first fundus image and the second fundus image to achieve alignment of the key points of the retinal vascular network. In the first and second fundus images after applying the spatial transformation function, a multi-scale image decomposition operation is performed to obtain image frequency sub-bands at different scales; For multiple image frequency sub-bands at different scales, a propagation path is formed by linking the local extreme points of signal intensity in the image frequency sub-bands across scales, and candidate lesion seed points are determined in the first fundus image and the second fundus image, respectively. Starting from the candidate lesion seed point, growth is performed based on the signal intensity gradient direction of the image frequency sub-band, and the outline of the lesion region is defined in the first fundus image and the second fundus image respectively. The contour of the lesion region is mapped to the retinal vascular topological space. By analyzing the relationship between the topological subgraph corresponding to the contour and the vascular segment, and comparing the consistency of the structure and spatial relationship of the topological subgraphs between the first fundus image and the second fundus image, a matching link for the lesion region is established.

2. The medical image matching method according to claim 1, characterized in that, Based on key points of the retinal vascular network in the first and second fundus images, a spatial transformation function is calculated and applied to the first and second fundus images to align the key points of the retinal vascular network, including: Based on the bifurcation points of retinal vessels in the first fundus image and the second fundus image, the topological connection relationship of the bifurcation points of the retinal vessels is determined; Stable reference bifurcation points are selected from the retinal vessel bifurcation points based on the topological connectivity. The spatial transformation function is calculated based on the stable reference bifurcation point, and the spatial transformation function is used to register the first fundus image and the second fundus image to achieve alignment of key points of the retinal vascular network.

3. The medical image matching method according to claim 1, characterized in that, In the first and second fundus images after applying the spatial transformation function, a multi-scale image decomposition operation is performed to obtain image frequency sub-bands at different scales, including: For the first fundus image and the second fundus image after applying the spatial transformation function, downsampling operation is performed according to multiple preset scale levels to obtain multi-level images with decreasing resolution; Based on the multi-level images, the signal intensity difference between corresponding pixels in adjacent level images is calculated to obtain the difference image corresponding to each level. Based on the intensity variation regions in each difference image, the associated image patches are located from the original images of the corresponding levels; By analyzing the variation pattern of pixel signal intensity within the image block, the feature components characterizing the main intensity variation pattern are determined. The feature components corresponding to adjacent image blocks in the same level are combined to form image frequency sub-bands corresponding to each level; By summing up the image frequency subbands corresponding to all levels, we can obtain image frequency subbands at different scales.

4. The medical image matching method according to claim 1, characterized in that, For multiple image frequency sub-bands at different scales, a propagation path is formed by linking local extrema of signal intensity in the image frequency sub-bands across scales, and candidate lesion seed points are determined in the first fundus image and the second fundus image, respectively, including: In multiple image frequency sub-bands at different scales, local extreme points of signal intensity are detected. These local extreme points include first-type points where the signal intensity value is a local maximum and second-type points where the signal intensity value is a local minimum. Starting from the local extremum point detected in the coarsest-scale image frequency sub-band, the corresponding position of the local extremum point in the adjacent fine-scale image frequency sub-band is traced to obtain the cross-scale propagation path. Based on the consistency of signal intensity values ​​at all local extreme points along the propagation path, target endpoints with continuous and stable propagation paths across multiple scales are selected. The coordinates of the target endpoint are mapped to the corresponding pixel positions in the first fundus image and the second fundus image to obtain candidate lesion seed points.

5. The medical image matching method according to claim 1, characterized in that, Starting from the candidate lesion seed point, growth is performed based on the signal intensity gradient direction of the image frequency sub-band, defining the contour of the lesion region in the first fundus image and the second fundus image respectively, including: Starting from the candidate lesion seed point, based on the gradient direction of the pixel signal intensity around the candidate lesion seed point in the finest scale image frequency sub-band, the process extends to adjacent pixels with the same gradient direction. When the expansion of the candidate lesion seed point encounters a pixel where the gradient direction changes abruptly or the signal strength is lower than a preset threshold, the expansion stops and the pixel is marked as a boundary point. Repeat the expansion operation until all possible growth directions originating from the candidate lesion seed point have been traversed, and connect all marked boundary points to form a closed contour. The closed contour is mapped back to the original coordinates of the first and second fundus images to serve as the contour of the lesion region.

6. The medical image matching method according to claim 1, characterized in that, The contour of the lesion region is mapped to the retinal vascular topological space. By analyzing the relationship between the topological subgraph corresponding to the contour and the vascular segment, and comparing the consistency of the structure and spatial relationship of the topological subgraphs between the first fundus image and the second fundus image, a matching link for the lesion region is established, including: The outline of the lesion area is superimposed onto the corresponding retinal vascular topology space to obtain associated topology data containing the positional relationship between the outline and the vascular segment; Based on the associated topology data, blood vessel segments located inside the contour are extracted, and a topology subgraph is constructed according to the interconnection relationship between the blood vessel segments. Based on the contours and associated topological data in the first fundus image and the second fundus image, a first topological sub-graph and a second topological sub-graph are extracted respectively; Calculate the structural similarity between the first topological subgraph and the second topological subgraph, and compare the relative positions of the first topological subgraph and the second topological subgraph in their respective retinal vascular networks; Based on the structural similarity and the consistency of the relative positions, the lesion region in the first fundus image is paired with the lesion region in the second fundus image to establish a matching link for the lesion region.

7. The medical image matching method according to claim 6, characterized in that, Based on the associated topological data, blood vessel segments located within the contour are extracted, and a topological subgraph is constructed according to the interconnections between the blood vessel segments, including: From the associated topology data, blood vessel segments that are completely located inside the contour are selected as internal blood vessel segments; The connection point is determined based on the spatial clustering relationship of the endpoint coordinates of the internal vascular segments; Based on the connection relationship between the internal blood vessel segment and the connection point, an initial graph structure is constructed, wherein the initial graph structure has the connection point as the vertex and the internal blood vessel segment as the edge. Based on the initial graph structure, the attribute information of all internal blood vessel segments belonging to the same connection point is fused to form a topological subgraph.

8. A medical image matching device, characterized in that, include: The acquisition module is used to acquire first and second fundus images at different time points. The first and second fundus images are digital fundus images containing lesion areas. The calculation module is used to calculate a spatial transformation function based on the key points of the retinal vascular network in the first fundus image and the second fundus image, and apply the spatial transformation function to the first fundus image and the second fundus image to achieve alignment of the key points of the retinal vascular network. The decomposition module is used to perform multi-scale image decomposition operations on the first fundus image and the second fundus image after applying the spatial transformation function to obtain image frequency sub-bands at different scales; The determination module is used to form a propagation path by linking the local extreme points of signal intensity in the image frequency sub-bands at different scales, and to determine the candidate lesion seed points in the first fundus image and the second fundus image respectively. The delineation module is used to grow based on the signal intensity gradient direction of the image frequency sub-band, starting from the candidate lesion seed point, and to delineate the outline of the lesion region in the first fundus image and the second fundus image respectively. A module is established to map the contour of the lesion region to the retinal vascular topological space. By analyzing the relationship between the topological subgraph corresponding to the contour and the vascular segment, and comparing the consistency of the structure and spatial relationship of the topological subgraphs between the first fundus image and the second fundus image, a matching link for the lesion region is established.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the medical image matching method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the matching method for medical images as described in any one of claims 1 to 7.

Images