Fundus blood flow imaging method and system based on cascade registration and storage medium

Abstract

The application relates to the technical field of optical imaging, in particular to an eye fundus blood flow imaging method and system based on cascade registration and a storage medium, which comprises the following steps: acquiring an eye fundus laser image sequence arranged according to a collection time sequence; based on the eye fundus laser image sequence, a plurality of mutually overlapped local time regions are divided according to the collection time sequence, local registration is performed on the eye fundus laser images in each local time region, and based on the eye fundus laser images after the local registration, a local structure image and a local blood flow image corresponding to the local time region are obtained; global unified registration is performed based on the local structure images, spatial transformation parameters corresponding to the local structure images are determined, and the spatial transformation parameters are synchronously applied to the corresponding local blood flow images to obtain a blood flow image sequence in a unified coordinate system; and fusion processing is performed on the blood flow image sequence in the unified coordinate system to obtain an eye fundus blood flow image. The application can improve the robustness, continuity and definition of eye fundus blood flow imaging.

Description

Technical Field

[0001] This application relates to the field of optical imaging technology, and in particular to a method, system and storage medium for fundus blood flow imaging based on cascade registration. Background Technology

[0002] The microcirculation status of the fundus reflects the blood perfusion of fundus tissues such as the retina and choroid, serving as a crucial reference for observing fundus tissue condition, analyzing blood flow changes, and conducting related examinations and assessments. Traditional methods of observing fundus blood flow typically rely on contrast agents to visualize the vascular perfusion process. While these methods provide a direct view of the distribution of blood vessels in the fundus, they present drawbacks such as invasiveness, allergy risks, and inconvenience in repeating examinations. With the development of optical imaging technology, the Scanning Laser Ophthalmoscope (SLO) can continuously acquire dynamic fundus images without the need for contrast agents, providing a technological foundation for analyzing fundus blood flow status based on temporal changes in image sequences.

[0003] In existing blood flow imaging methods based on SLO fundus image sequences, a single frame is typically selected from a series of continuously acquired fundus images as a reference image. The remaining images are then registered frame by frame to the coordinate system of this reference image, creating a unified spatial coordinate system for the entire image sequence. After image registration, blood flow responses are extracted based on grayscale changes, correlation changes, or statistical fluctuations between consecutive images. For example, differential algorithms, decorrelation algorithms, statistical variance algorithms, or laser speckle contrast imaging algorithms are used to differentiate the dynamic blood flow region from the relatively static fundus tissue region in terms of image response. The obtained blood flow responses are then averaged, fused, or enhanced to generate fundus blood flow images, which are then used for quantitative analysis.

[0004] However, in actual acquisition, SLO fundus image sequences are easily affected by micro-movements, drift, saccades, and local jitter, causing inter-frame displacement and motion artifacts between consecutive images, thus disrupting the correspondence of the same fundus tissue point in the time series. For image sequences with long acquisition times, the structural and grayscale differences between distant temporal images and reference images usually increase with the extension of the acquisition time. For image sequences with complex eye movements, some image frames may simultaneously exhibit overall translation, local deformation, and instantaneous jitter, making it difficult to stably unify the entire sequence to the coordinate system of a single reference image. Furthermore, if differential algorithms, decorrelation algorithms, statistical variance algorithms, or laser speckle contrast imaging algorithms are used to extract the blood flow response, noise disturbances, brightness unevenness, static tissue residues, and registration deviations in low signal-to-noise ratio images can easily be mixed with actual blood flow changes, making the blood flow response unable to accurately reflect the fundus blood flow distribution. Therefore, existing blood flow imaging methods based on SLO fundus image sequences are prone to registration instability and blood flow extraction distortion under conditions of long sequences, complex eye movements, and low signal-to-noise ratio. Summary of the Invention

[0005] This application provides a cascaded registration-based fundus blood flow imaging method, system, and storage medium, which can improve the robustness, continuity, and clarity of fundus blood flow imaging. The technical solution provided in this application is as follows: In a first aspect, this application provides a fundus blood flow imaging method based on cascaded registration, the method comprising: Acquire a sequence of fundus laser images arranged in chronological order of acquisition; Based on the fundus laser image sequence, multiple overlapping local time regions are divided according to the acquisition time sequence. Fundus laser images in each local time region are locally registered, and local structural images and local blood flow images corresponding to the local time regions are obtained based on the locally registered fundus laser images. Global unified registration is performed based on each of the local structural images to determine the spatial transformation parameters corresponding to each of the local structural images, and each of the spatial transformation parameters is synchronously applied to the corresponding local blood flow image to obtain a blood flow image sequence in a unified coordinate system. The blood flow image sequence under the unified coordinate system is fused to obtain the fundus blood flow image.

[0006] In one specific implementation, the step of dividing the fundus laser image sequence into multiple overlapping local time regions according to the acquisition time sequence includes: According to the preset local window length and preset sliding step size, multiple consecutive frames of fundus laser images are selected from the fundus laser image sequence along the acquisition time sequence to obtain multiple local time regions; Each of the local time regions includes multiple frames of continuously acquired fundus laser images, and at least one identical fundus laser image is contained between two adjacent local time regions.

[0007] In one specific implementation, the step of locally registering the fundus laser images within each local time region and obtaining a local structural image corresponding to the local time region based on the locally registered fundus laser images includes: For each local time region, one frame of fundus laser image is selected from multiple frames of fundus laser images within the local time region as a local reference image; Register the remaining fundus laser images within the local time region, excluding the local reference image, to the coordinate system where the local reference image is located, to obtain the locally registered image sequence corresponding to the local time region; The local reference image is determined as a local structural image corresponding to the local time region.

[0008] In one specific implementation, a local blood flow image corresponding to the local time region is obtained based on the locally registered fundus laser image, including: Based on the original image difference information and filter domain difference information between locally registered fundus laser images, a basic blood flow response is constructed. The basic blood flow response is subjected to nonlinear mapping to obtain the enhanced blood flow response; Based on the image intensity information of the locally registered fundus laser image, the enhanced blood flow response is subjected to brightness normalization processing to obtain a local blood flow image corresponding to the local time region.

[0009] In one specific implementation, the step of performing global unified registration based on each of the local structural images to determine the spatial transformation parameters corresponding to each of the local structural images includes: Select one frame of local structure image from each of the aforementioned local structure images as a global reference image; For each frame of target local structure image other than the global reference image, feature point detection and feature point matching are performed on the target local structure image and the global reference image to obtain matching feature point pairs between the target local structure image and the global reference image. Based on the matching feature point pairs, the spatial transformation relationship of the target local structure image relative to the coordinate system of the global reference image is determined; Based on the spatial transformation relationship, the target local structure image is registered to the coordinate system where the global reference image is located, and the transformation parameters corresponding to the spatial transformation relationship are determined as the spatial transformation parameters corresponding to the target local structure image.

[0010] In one specific implementation, the step of synchronously applying each of the spatial transformation parameters to the corresponding local blood flow image to obtain a blood flow image sequence in a unified coordinate system includes: Based on the relationship between generating a frame of local structural image and a frame of local blood flow image corresponding to the same local time region, a one-to-one correspondence between each of the local structural images and each of the local blood flow images is determined; For each frame of the target local blood flow image, spatial transformation parameters corresponding to the local structure image that has a one-to-one correspondence with the target local blood flow image are obtained; According to the obtained spatial transformation parameters, the target local blood flow image is spatially transformed to the coordinate system of the global reference image. The local blood flow images, transformed to the coordinate system of the global reference image, are arranged according to the acquisition sequence of the corresponding local time regions to obtain a blood flow image sequence in a unified coordinate system.

[0011] In one specific implementation, the fusion processing of the blood flow image sequence in the unified coordinate system to obtain the fundus blood flow image includes: In the blood flow image sequence under the unified coordinate system, determine the pixels with the same coordinate position in each blood flow image; The pixel values ​​of multiple pixels with the same coordinate position are fused to obtain the fused pixel value at the corresponding coordinate position; The fundus blood flow image is generated based on the fused pixel values ​​at each coordinate position.

[0012] Secondly, this application provides a fundus blood flow imaging system based on cascaded registration, which adopts the following technical solution: A fundus blood flow imaging system based on cascaded registration includes: The image sequence acquisition module is used to acquire a sequence of fundus laser images arranged in the acquisition time sequence; The local registration module is used to divide the fundus laser image sequence into multiple overlapping local time regions according to the acquisition time sequence, perform local registration on the fundus laser images in each local time region, and obtain local structural images and local blood flow images corresponding to the local time regions based on the locally registered fundus laser images. The global registration module is used to perform global unified registration based on each of the local structural images, determine the spatial transformation parameters corresponding to each of the local structural images, and synchronously apply each of the spatial transformation parameters to the corresponding local blood flow images to obtain a blood flow image sequence in a unified coordinate system. The blood flow image fusion module is used to fuse blood flow image sequences under the unified coordinate system to obtain fundus blood flow images.

[0013] Thirdly, this application provides an electronic device, the device including a processor and a memory; the memory stores a program, the program being loaded and executed by the processor to implement a cascaded registration-based fundus blood flow imaging method as described in the first aspect.

[0014] Fourthly, this application provides a computer-readable storage medium storing a program that, when executed by a processor, is used to implement a cascaded registration-based fundus blood flow imaging method as described in the first aspect.

[0015] By acquiring a sequence of fundus laser images arranged chronologically, the system first divides the image into multiple overlapping local time regions according to the acquisition sequence. Local registration is then performed on the fundus laser images within each local time region to obtain local structural and blood flow images corresponding to that region. Next, global registration is performed based on these local structural images to determine the spatial transformation parameters for each image. These parameters are then synchronously applied to the corresponding local blood flow images, resulting in a blood flow image sequence in a unified coordinate system. Finally, the blood flow image sequences in the unified coordinate system are fused to obtain the fundus blood flow image. Because the fundus laser image sequence is first divided into multiple overlapping local time regions, local registration is no longer directly applied to the entire long sequence but is completed within a shorter timeframe. This reduces the temporal span and structural differences between images within each local time region. Furthermore, the overlap between adjacent local time regions ensures that the processing results from different local time regions remain continuous. Building upon this foundation, instead of directly performing global registration on the local blood flow images, spatial transformation parameters are determined using the corresponding local structural images. These parameters are then synchronously applied to the corresponding local blood flow images, ensuring that the local blood flow images are unified to the same coordinate system along with the registration results of the local structural images. Thus, local registration and global unified registration form a cascaded processing relationship. This reduces the impact of long sequences and complex eye movements on single global registrations, while also enabling the fusion of local blood flow images within a unified coordinate system. Consequently, this improves the spatial consistency and blood flow response accuracy of the final fundus blood flow image.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of the fundus blood flow imaging method based on cascaded registration in the embodiments of this application.

[0018] Figure 2 This is a schematic diagram of the overall process of the fundus blood flow imaging method based on cascaded registration in the embodiments of this application.

[0019] Figure 3 These are comparison images of blood flow corresponding to different registration processing methods in the embodiments of this application.

[0020] Figure 4 This is a comparison chart of the quantitative evaluation results of the traditional single-reference full-sequence registration scheme and the two-stage registration link of this application in the embodiments of this application.

[0021] Figure 5 This is a comparison of the blood flow image effects corresponding to different blood flow extraction algorithms in the embodiments of this application.

[0022] Figure 6 This is a structural block diagram of the fundus blood flow imaging system based on cascaded registration in the embodiments of this application.

[0023] Figure 7 This is a block diagram of an electronic device for fundus blood flow imaging based on cascaded registration, as described in this application. Detailed Implementation

[0024] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0025] Optionally, this application uses the fundus blood flow imaging method based on cascaded registration provided in various embodiments as an example for description in an electronic device. The electronic device is a terminal or server. The terminal can be a computer, tablet computer, etc. This embodiment does not limit the type of electronic device.

[0026] Reference Figure 1 This is a flowchart illustrating a cascaded registration-based fundus blood flow imaging method according to an embodiment of this application. The method includes at least the following steps: Step S101: Obtain a sequence of fundus laser images arranged according to the acquisition time sequence.

[0027] In step S101, a scanning laser ophthalmoscope (SLO) is used to continuously scan the target fundus region, acquiring fundus laser images of the target fundus region at multiple consecutive acquisition times. These fundus laser images are then arranged according to their acquisition sequence to obtain a fundus laser image sequence. This fundus laser image sequence is used to characterize the structural grayscale changes of the same target fundus region over a continuous time range.

[0028] Specifically, during the acquisition process, the SLO device scans the target fundus region frame by frame according to a preset imaging area, obtaining one frame of fundus laser image per scan. Each frame of fundus laser image corresponds to the same target fundus region, and each frame of fundus laser image has its own corresponding acquisition time. The fundus laser images are arranged in chronological order of acquisition time to obtain a fundus laser image sequence, represented as follows: ;in, This represents a sequence of laser images of the fundus; This represents the first frame of fundus laser image arranged in the order of acquisition time; This represents the second frame of fundus laser image arranged in the order of acquisition time; Indicates the number arranged in the order of collection time. Frame of fundus laser images; This indicates the total number of fundus laser image frames contained in the fundus laser image sequence, and It is an integer greater than 1.

[0029] In practice, each frame of the fundus laser image sequence includes structural grayscale information of the target fundus region. Since the frames of fundus laser images are arranged according to the acquisition sequence, the fundus laser image sequence can reflect the temporal changes of the target fundus region during continuous acquisition.

[0030] Step S102: Based on the fundus laser image sequence, divide multiple overlapping local time regions according to the acquisition time sequence, perform local registration on the fundus laser images in each local time region, and obtain local structural images and local blood flow images corresponding to the local time regions based on the locally registered fundus laser images.

[0031] In step S102, based on the fundus laser image sequence obtained in step S101, the fundus laser image sequence is first divided into multiple overlapping local time regions according to the acquisition time sequence, so that each local time region includes multiple continuously acquired fundus laser images. Then, for each local time region, the fundus laser images in the local time region are locally registered, and the local structure image and local blood flow image corresponding to the local time region are obtained based on the locally registered fundus laser images.

[0032] Specifically, during the local time region segmentation process, multiple consecutive frames of fundus laser images are selected sequentially from the fundus laser image sequence along the acquisition time sequence, according to a preset local window length and a preset sliding step size, resulting in multiple local time regions. Each local time region includes multiple consecutively acquired fundus laser images, and adjacent local time regions contain at least one identical fundus laser image. By retaining identical fundus laser images between adjacent local time regions, temporal continuity can be maintained between different local time regions, and boundary fragmentation and abrupt changes in local blood flow images caused by segmentation processing can be reduced.

[0033] Taking a fundus laser image sequence consisting of 20 frames of fundus laser images, with a preset local window length of 3 and a preset sliding step size of 1 as an example, the local time region is represented as follows: ; in, This indicates the first local time region. This indicates the second local time region. This indicates the third local time region. This indicates the 18th local time region; These represent the fundus laser images, frames 1, 2, 3 through 20, arranged chronologically according to their acquisition time. Adjacent local time regions contain overlapping images; for example, and All include and , and All include and .

[0034] For each local time region, one frame of fundus laser image is selected from multiple frames of fundus laser images within that local time region as a local reference image. The remaining fundus laser images within that local time region, excluding the local reference image, are then registered to the coordinate system of the local reference image, resulting in a locally registered fundus laser image. The local reference image is defined as the local structural image corresponding to that local time region. Local registration methods include at least one of phase-correlation registration, feature point registration, thin-plate spline non-rigid registration, B-spline free deformation registration, and optical flow-constrained registration. Since each local time region includes multiple consecutively acquired frames of fundus laser images, the structural differences between adjacent short-time images are small. Registration within the local time region can correct inter-frame displacement caused by eye micromovements, drift, or saccades within a local short time range.

[0035] In one specific embodiment, if the local time region contains an odd number of fundus laser images, then the fundus laser image located in the middle position is used as the local reference image; if the local time region contains an even number of fundus laser images, then either of the two fundus laser images located in the middle position is used as the local reference image, and the same selection rule is used in each local time region of the same fundus laser image sequence.

[0036] In the process of local blood flow image extraction, a basic blood flow response is constructed based on the original image difference information and the filtered domain difference information between two adjacent frames of fundus laser images in a locally registered image sequence corresponding to the same local time region. Specifically, for the locally registered image sequence corresponding to the same local time region, two adjacent frames of fundus laser images are determined sequentially according to the acquisition time sequence, and the gray-level difference at the same pixel position of each pair of adjacent frames of fundus laser images is calculated to obtain the original image difference information. Then, each frame of fundus laser images in the locally registered image sequence is smoothed, and the gray-level difference at the same pixel position of the smoothed adjacent frames of fundus laser images is calculated sequentially according to the acquisition time sequence to obtain the filtered domain difference information. The original image difference information is used to characterize the instantaneous dynamic changes between fundus laser images within the local time region, and the filtered domain difference information is used to characterize the stable changes within the local time region after neighborhood smoothing. The original image difference information and the filtered domain difference information are combined to obtain the basic blood flow response.

[0037] It should be noted that when there are multiple sets of adjacent fundus laser images within the same local time region, the original image difference information corresponding to the multiple sets of adjacent fundus laser images is averaged, the maximum value is retained, or the information is accumulated to obtain the original image difference information corresponding to the local time region; and the filter domain difference information corresponding to the multiple sets of adjacent fundus laser images is averaged, the maximum value is retained, or the information is accumulated to obtain the filter domain difference information corresponding to the local time region.

[0038] In one specific embodiment, to ensure that the original image difference information and the filtered domain difference information are on the same numerical scale, normalization processing is performed on both. Specifically, the difference value of each pixel position in the original image difference information is divided by the maximum difference value in the original image difference information to obtain normalized original image difference information with a value range of 0 to 1; the difference value of each pixel position in the filtered domain difference information is divided by the maximum difference value in the filtered domain difference information to obtain normalized filtered domain difference information with a value range of 0 to 1. If the maximum difference value in the original image difference information is 0, then the value of each pixel position in the normalized original image difference information is set to 0; if the maximum difference value in the filtered domain difference information is 0, then the value of each pixel position in the normalized filtered domain difference information is set to 0. After the normalization processing is completed, the normalized original image difference information and the normalized filtered domain difference information at the same pixel position are averaged and fused to obtain the basic blood flow response at that pixel position.

[0039] After obtaining the basic blood flow response, a nonlinear mapping process is performed on the basic blood flow response to obtain the enhanced blood flow response; then, combined with the image intensity information of the locally registered fundus laser image, the brightness of the enhanced blood flow response is normalized to obtain the local blood flow image corresponding to the local time region.

[0040] In one specific embodiment, after obtaining the baseline blood flow response, a nonlinear mapping process is applied to the baseline blood flow response to obtain the enhanced blood flow response. Specifically, since the baseline blood flow response has already been normalized and its value ranges from 0 to 1, a power-law nonlinear mapping is used to enhance the baseline blood flow response. In this embodiment, the nonlinear mapping exponent is determined based on the average value of the baseline blood flow response within the current local time region. The nonlinear mapping exponent is expressed as follows: ; in, Indicates the first The nonlinear mapping exponent corresponding to each local time region Indicates the first The average value of the baseline blood flow response at each pixel location within a local time region. Since the baseline blood flow response ranges from 0 to 1, therefore... The value range is from 0 to 1. The value ranges from 0.5 to 1.

[0041] The enhanced blood flow response is represented as follows: ; in, Indicates the first A local time region at pixel location Enhanced blood flow response Indicates the first A local time region at pixel location Basic blood flow response at the site, Indicates the first The nonlinear mapping exponent corresponding to each local time region This indicates the pixel position in the fundus laser image.

[0042] The aforementioned nonlinear mapping exponent is determined by the average value of the baseline blood flow response within the current local time region. When the overall baseline blood flow response is weak within the current local time region, Smaller Approaching 0.5, power-law nonlinear mappings exhibit a higher degree of enhancement in low-response regions; when the overall baseline blood flow response is strong within the current local time region, Larger Approaching 1, the power-law type nonlinear mapping tends to maintain the basic blood flow response, thereby reducing the over-enhancement of the strong response region.

[0043] After obtaining the enhanced blood flow response, the brightness of the enhanced blood flow response is normalized based on the image intensity information of the locally registered fundus laser image. Specifically, the brightness of the enhanced blood flow response is normalized. The local reference image corresponding to each local time region is used as the intensity benchmark for brightness normalization. A brightness deviation coefficient is determined based on the degree of brightness deviation of each pixel position in the local reference image relative to the average gray value. The brightness deviation coefficient is expressed as follows: ; in, Indicates the first A local time region at pixel location The brightness deviation coefficient at that location Indicates the first The local reference image corresponding to each local time region at pixel location grayscale value at that location Indicates the first The average gray value of the local reference image corresponding to each local time region. Indicates the first The maximum brightness deviation of each pixel position relative to the average gray value in the local reference image corresponding to each local time region. This represents a preset positive number used to avoid fractions with a denominator of 0. Because... Not greater than And add to the denominator Therefore, the brightness deviation coefficient The value of is greater than or equal to 0 and less than or equal to 1, and there is no case where the denominator is 0.

[0044] The enhanced blood flow response is normalized and corrected based on the brightness deviation coefficient to obtain the local blood flow image corresponding to this local time region. The pixel values ​​of the local blood flow image are represented as follows: ; in, Indicates the first Local blood flow images corresponding to a local time region at pixel location Pixel value at that location, Indicates the first A local time region at pixel location Enhanced blood flow response Indicates the first A local time region at pixel location The brightness deviation coefficient at that location.

[0045] The aforementioned brightness normalization process directly affects the enhanced blood flow response obtained from the power-law nonlinear mapping. When the gray value of a pixel is close to the average gray value of the local reference image, the brightness deviation coefficient is small, and the local blood flow image retains more of the enhancement result of the power-law nonlinear mapping on the basic blood flow response. When the gray value of a pixel deviates significantly from the average gray value of the local reference image, the brightness deviation coefficient increases, and the enhanced blood flow response is correspondingly suppressed. Thus, the power-law nonlinear mapping is used to improve the visibility of the basic blood flow response within a local time region, while the brightness normalization process is used to suppress abnormal enhanced responses caused by local brightness unevenness, enhanced background reflection, or regional gray-level differences. The two processes work sequentially during the generation of the same local blood flow image, enabling the local blood flow image to simultaneously enhance the blood flow response and suppress brightness interference.

[0046] because The value range is from 0 to 1. The value range is from 0.5 to 1, therefore The value range remains from 0 to 1; due to The value of is in the range of 0 to 1, therefore The value of is between 1 and 2, and there is no case where the denominator is 0; correspondingly, The value range remains 0 to 1, and there will be no situation where the pixel value of the local blood flow image exceeds the limit.

[0047] It should be noted that the above-described processing method based on power-law nonlinear mapping and brightness deviation coefficient is one specific implementation. In other implementations, nonlinear mapping processing can also be implemented using logarithmic function mapping, exponential function mapping, or at least a combination of two of power-law mapping, logarithmic function mapping, and exponential function mapping. As long as the dynamic range of the basic blood flow response can be adjusted, it falls under the implementation of nonlinear mapping processing in this application. Brightness normalization processing can also be implemented using existing image processing techniques such as intensity normalization, background brightness correction, local brightness compensation, or grayscale equalization. As long as the brightness of the enhanced blood flow response can be corrected based on the image intensity information of the locally registered fundus laser image, it falls under the implementation of brightness normalization processing in this application. This application does not limit the specific implementation methods of nonlinear mapping processing and brightness normalization processing.

[0048] In implementation, multiple local time regions correspond to multiple local structural images and multiple local blood flow images. The sequence of local structural images is represented as follows: ;in, Represents a sequence of images with local structures. These represent local structural images corresponding to multiple local time regions. The local blood flow image sequence is represented as follows: ;in, This represents a sequence of local blood flow images. These represent local blood flow images corresponding to multiple local time regions. Local structural images and local blood flow images with the same number correspond to the same local time region.

[0049] Step S103: Perform global unified registration based on each local structural image, determine the spatial transformation parameters corresponding to each local structural image, and apply each spatial transformation parameter synchronously to the corresponding local blood flow image to obtain a blood flow image sequence in a unified coordinate system.

[0050] In step S103, based on the local structure image sequence and local blood flow image sequence obtained in step S102, global unified registration is performed on each local structure image in the local structure image sequence to determine the spatial transformation parameters corresponding to each local structure image, and each spatial transformation parameter is synchronously applied to the corresponding local blood flow image to obtain a blood flow image sequence in a unified coordinate system.

[0051] Specifically, from local structural image sequences A local structural image is selected as the global reference image. After the global reference image is determined, for each target local structural image other than the global reference image, feature point detection and feature point matching are performed on the target local structural image and the global reference image to obtain matching feature point pairs between the target local structural image and the global reference image. The matching feature point pairs are used to characterize the correspondence between the locations of the same fundus structures in the target local structural image and the global reference image.

[0052] In one specific embodiment, according to the acquisition sequence of the local time region corresponding to each local structural image, a frame of local structural image located in the middle of the acquisition sequence is selected as the global reference image. If the local structural image sequence contains an even number of local structural images, either of the two local structural images located in the middle position is selected as the global reference image, and the same selection rule is used in the processing of the same fundus laser image sequence. Since the global reference image is located in the middle of the acquisition sequence of the local structural image sequence, the temporal distance between the global reference image and each preceding and following local structural image is relatively balanced, which helps to reduce the structural differences when registering distant local structural images with respect to the global reference image.

[0053] Based on matching feature point pairs, the spatial transformation relationship of the target local structure image relative to the coordinate system of the global reference image is determined. According to the spatial transformation relationship, the target local structure image is registered to the coordinate system of the global reference image, and the transformation parameters corresponding to the spatial transformation relationship are determined as the spatial transformation parameters corresponding to the target local structure image. In implementation, for the global unified registration of the target local structure image and the global reference image, feature points are first detected in both the target local structure image and the global reference image, and the detected feature points are matched to obtain matching feature point pairs between the target local structure image and the global reference image. Then, based on the matching feature point pairs, the initial spatial transformation relationship of the target local structure image relative to the global reference image is determined, and the target local structure image is initially registered according to the initial spatial transformation relationship. After the initial registration, a local displacement field is established based on the remaining local positional deviation between the target local structure image and the global reference image, and the target local structure image is finely registered according to the local displacement field to obtain the spatial transformation parameters corresponding to the target local structure image. Spatial transformation parameters include at least one of translation parameters, rotation parameters, scaling parameters, affine transformation parameters, perspective transformation parameters, or local displacement field parameters, which are determined based on the registration model used.

[0054] After obtaining the spatial transformation parameters corresponding to each local structural image, the one-to-one correspondence between each local structural image and each local blood flow image is determined based on the relationship between generating a frame of local structural image and a frame of local blood flow image corresponding to the same local time region. For each frame of target local blood flow image, the spatial transformation parameters corresponding to the local structural image that has a one-to-one correspondence with the target local blood flow image are obtained; according to the obtained spatial transformation parameters, the target local blood flow image is spatially transformed to the coordinate system of the global reference image.

[0055] After obtaining the local blood flow images transformed to the coordinate system of the global reference image, these local blood flow images are arranged according to the acquisition sequence of their corresponding local time regions to obtain a blood flow image sequence in a unified coordinate system. For example, when a local structure image... Corresponding local blood flow image Local structural images Corresponding local blood flow image Local structural images Corresponding local blood flow image And sequentially correspond to local structural images. Corresponding local blood flow image At that time, based on local structure images The determined spatial transformation parameters are synchronously applied to the corresponding local blood flow images. A blood flow image sequence in a unified coordinate system was obtained. .in, These represent the blood flow images obtained after transforming each local blood flow image to the coordinate system of the global reference image.

[0056] When performing spatial transformation on local blood flow images, the spatial transformation parameters are not recalculated based on the local blood flow images, but are synchronously obtained from the globally unified registration results of the corresponding local structural images. Since local blood flow images mainly reflect blood flow response, their texture is relatively sparse and easily affected by noise. Directly registering local blood flow images can easily affect registration stability. Local structural images retain the structural grayscale information of the target fundus region and have relatively rich and stable structural features. Therefore, determining the spatial transformation parameters through local structural images and synchronously applying the spatial transformation parameters to the corresponding local blood flow images can unify all local blood flow images to the same global coordinate system.

[0057] It should be noted that for the global reference image, since it is already located in the coordinate system of the global reference image, the spatial transformation parameter corresponding to the global reference image is determined as the unit transformation parameter. The unit transformation parameter is used to indicate that the position of each pixel in the image in the coordinate system of the global reference image is not changed. Accordingly, the local blood flow image corresponding to the global reference image remains unchanged when this unit transformation parameter is applied synchronously, and is used as a frame of blood flow image in the blood flow image sequence in a unified coordinate system.

[0058] Step S104: Perform fusion processing on the blood flow image sequence under a unified coordinate system to obtain fundus blood flow images.

[0059] In step S104, based on the blood flow image sequence in the unified coordinate system obtained in step S103, the blood flow images in the unified coordinate system are fused to obtain a fundus blood flow image. Since step S103 has unified each local blood flow image to the coordinate system of the global reference image, the blood flow images in the unified coordinate system have a spatial correspondence. Therefore, multiple pixel values ​​at the same coordinate position can be fused to form a fundus blood flow image that reflects the blood flow distribution in the target fundus region.

[0060] Specifically, in a blood flow image sequence under a unified coordinate system, pixels with the same coordinate position in each blood flow image are identified, and effective pixels participating in the fusion calculation are determined based on whether each pixel is located within the effective imaging area of ​​the corresponding blood flow image; the pixel values ​​of multiple effective pixels at the same coordinate position are fused to obtain the fused pixel value at the corresponding coordinate position; and fundus blood flow images are generated based on the fused pixel values ​​at each coordinate position.

[0061] In practice, when fusing blood flow image sequences in a unified coordinate system, for any given coordinate position, the effective pixel values ​​at that coordinate position in the blood flow image sequence are acquired, and the acquired effective pixel values ​​are fused to obtain the fused pixel value corresponding to that coordinate position. If a blood flow image at that coordinate position has a blank area or exceeds the effective imaging area due to spatial transformation, the pixel value of that blood flow image at that coordinate position is not included in the fusion calculation. After traversing all coordinate positions in the unified coordinate system in the above manner, the fused pixel values ​​corresponding to each coordinate position are used to form the fundus blood flow image.

[0062] In one specific implementation, the fusion calculation employs a cumulative averaging method. For a blood flow image sequence in a unified coordinate system, the effective pixel values ​​at the same coordinate positions in each blood flow image are summed, and the summation result is divided by the number of effective pixels participating in the fusion to obtain the fused pixel value corresponding to that coordinate position. By performing the above cumulative averaging process on each coordinate position, a fundus blood flow image is obtained. The fundus blood flow image is used to characterize the fusion result of blood flow response in multiple local time regions within the target fundus region.

[0063] In other embodiments, the fusion calculation further includes at least one of weighted averaging, maximum response preservation, denoising fusion, or vascular enhancement. When using weighted averaging, different weights are assigned to pixel values ​​at the same coordinate position in each blood flow image based on the image quality, local blood flow response intensity, or reliability of the corresponding local time region, and the fused pixel value is determined based on the weighting result. When using maximum response preservation, the maximum pixel value is selected from multiple pixel values ​​at the same coordinate position in each blood flow image as the fused pixel value. When using denoising fusion, abnormal pixel responses are suppressed before or during fusion. When using vascular enhancement, the linear vascular structures in the fused fundus blood flow image are enhanced.

[0064] Furthermore, preferably, after obtaining the fundus blood flow image, blood flow intensity statistics or flow velocity estimation can be performed based on the fundus blood flow image to obtain quantitative results corresponding to the fundus blood flow image. Blood flow intensity statistics are used to characterize the strength of blood flow response at different locations within the target fundus region, while flow velocity estimation is used to characterize the relative velocity of blood flow changes within the target fundus region. Through the above fusion processing, the blood flow responses corresponding to multiple local time regions are uniformly integrated into the same fundus blood flow image.

[0065] In summary, combining Figure 2 The process involves acquiring a sequence of fundus laser images arranged in chronological order of acquisition. Based on this sequence, multiple overlapping local time regions are divided according to the acquisition time sequence. Local registration is performed on the fundus laser images within each local time region, and local structural and blood flow images corresponding to the local time regions are obtained from the locally registered fundus laser images. Subsequently, global unified registration is performed based on each local structural image to determine the spatial transformation parameters corresponding to each local structural image. These spatial transformation parameters are then synchronously applied to the corresponding local blood flow images to obtain a blood flow image sequence in a unified coordinate system. Finally, the blood flow image sequence in the unified coordinate system is fused to obtain the fundus blood flow image. Thus, this application establishes a cascaded registration process that connects local registration and global unified registration, enabling local blood flow extraction, structural map-guided registration, and synchronous blood flow image transformation to be executed continuously within the same processing chain.

[0066] Based on the above scheme, the fundus laser image sequence is first divided into multiple overlapping local time regions, so that each registration and blood flow extraction is performed within a short time range, and the structural and gray-level differences between adjacent frames are relatively small. This can reduce the instability caused by long-distance images directly participating in unified registration in long sequences. Local registration is performed first in each local time region before local blood flow images are extracted, so that the blood flow response is based on the locally aligned image, which helps to reduce the pseudo-response caused by eye micro-movement, drift, and saccade. Furthermore, global unified registration does not directly rely on local blood flow images with sparse texture and obvious noise. Instead, it uses local structural images that correspond one-to-one with local blood flow images to determine spatial transformation parameters, and applies the spatial transformation parameters synchronously to the corresponding local blood flow images, so that each local blood flow image is unified to the same coordinate system while maintaining the blood flow response content. Through this cascaded registration and synchronous transformation process, the stability of local short-time registration and the continuity of global coordinate unification can work together to improve the spatial consistency and blood flow response realism of fundus blood flow images under long sequences, complex eye movements and low signal-to-noise ratio conditions.

[0067] Furthermore, by dividing the sequence of fundus laser images arranged according to acquisition time into multiple overlapping local time regions, and performing local registration before extracting local blood flow images within each local time region, blood flow extraction is based on images with small inter-frame differences and high structural similarity within a short time range. This reduces the impact of eye micromovements, drift, and saccades on blood flow response extraction, improving the accuracy of local blood flow images. Simultaneously, this application does not directly register local blood flow images with sparse texture and significant noise during the global unification stage; instead, it utilizes the local structure map corresponding to the local blood flow image. This invention involves obtaining spatial transformation parameters and applying them synchronously to the corresponding local blood flow images. This allows for the utilization of richer and more stable fundus structural features in the structural images to improve the reliability of global uniformity. Furthermore, during the extraction of local blood flow images, this application constructs a basic blood flow response by combining the original image difference information and the filter domain difference information. It then combines nonlinear mapping processing and brightness normalization processing to generate local blood flow images. This enables the blood flow response to simultaneously display weak blood flow signals, suppress noise, and reduce interference from local brightness unevenness, thereby improving the continuity, clarity, and realism of the final fundus blood flow image.

[0068] To further illustrate the imaging effect of the fundus blood flow imaging method based on cascade registration provided in this application, the blood flow image generation results of this application are explained below in conjunction with comparative experiments.

[0069] Reference Figure 3 , Figure 3 These are comparison images of blood flow corresponding to different registration processing methods in the embodiments of this application. Figure 3The images, from left to right, show the blood flow images obtained without registration processing, using traditional single-reference full-sequence registration processing, and using the two-stage registration link processing described in this application. Figure 3 As can be seen, without registration processing, the background noise in the image is significant, and the continuity of the vascular structure is poor. After traditional single-reference full-sequence registration processing, the vascular structure is improved compared to the unregistered image, but artifacts and insufficient display of small vessels still exist in local areas. After using the two-stage registration link processing of this application, the overall continuity of the vascular network is better, the microvascular structure is displayed more completely, and the vessel edges in the magnified local areas are clearer. This comparative result shows that this application, by first performing local registration and local blood flow extraction in overlapping local time regions, and then performing global unified registration based on the local structural image and simultaneously transforming the local blood flow image, can improve the spatial consistency of blood flow images under complex eye-tracking conditions.

[0070] Reference Figure 4 , Figure 4 This is a comparison chart of the quantitative evaluation results of the traditional single-reference full-sequence registration scheme and the two-stage registration link of this application in the embodiments of this application. Figure 4 The study compared vascular continuity scores, artifact area ratios, blood flow map signal-to-noise ratios, and blood flow map contrast-to-noise ratios. A higher vascular continuity score indicates a more complete vascular structure; a lower artifact area ratio indicates less artifact interference; and higher blood flow map signal-to-noise ratios and contrast-to-noise ratios indicate better blood flow image quality. Figure 4 As can be seen, compared with the traditional single-reference full-sequence registration scheme, the two-stage registration chain of this application improves the vessel continuity score, blood flow map signal-to-noise ratio, and blood flow map contrast-to-noise ratio, while reducing the proportion of artifact area. These results demonstrate that the cascaded registration method of this application can reduce the impact of motion artifacts on blood flow images while maintaining the continuity of local blood flow images.

[0071] Reference Figure 5 , Figure 5 This is a comparison of the blood flow image effects corresponding to different blood flow extraction algorithms in the embodiments of this application. Figure 5 The image, from left to right, shows the difference algorithm, decorrelation algorithm, statistical variance algorithm, laser speckle contrast imaging algorithm, and the blood flow image obtained by the method of this application. Figure 5It can be seen that the image obtained by the difference algorithm has more bright background speckles and relatively insufficient blood vessel continuity. The decorrelation algorithm can display the main blood vessel structure, but the image uniformity is poor and it is easily affected by the background brightness distribution. The statistical variance algorithm does not extract enough blood vessel structure, and the overall blood vessel network is difficult to distinguish clearly. The laser speckle contrast imaging algorithm is easily affected by background brightness, and the blood vessel edges are relatively blurred. In contrast, the blood flow image obtained by the method of this application can more clearly display the capillary network around the macula, preserve microvessels more fully, suppress background noise and brightness unevenness interference, and has better overall imaging quality. This comparison result shows that the present application can improve the display integrity and image stability of the microvascular network in the blood flow image by constructing a local blood flow image based on the locally registered fundus laser image and performing fusion processing in a unified coordinate system.

[0072] In conclusion, Figures 3 to 5 The imaging effect of the proposed method is illustrated from three aspects: registration processing effect, quantitative evaluation index, and comparison of different blood flow extraction algorithms. By using a cascaded processing method that connects local registration with global unified registration, and a fusion output method based on local blood flow image sequences, this application can obtain fundus blood flow images with more continuous vascular structures, less artifact interference, and clearer microvascular display even in fundus laser image sequences with eye movement, local jitter, and low signal-to-noise ratio interference.

[0073] Figure 6 This is a structural block diagram of a fundus blood flow imaging system based on cascaded registration provided in one embodiment of this application. The system includes at least the following modules: The image sequence acquisition module is used to acquire a sequence of fundus laser images arranged in the acquisition time sequence; The local registration module is used to divide the fundus laser image sequence into multiple overlapping local time regions according to the acquisition time sequence, perform local registration on the fundus laser images in each local time region, and obtain local structural images and local blood flow images corresponding to the local time regions based on the locally registered fundus laser images. The global registration module is used to perform global unified registration based on each local structural image, determine the spatial transformation parameters corresponding to each local structural image, and synchronously apply each spatial transformation parameter to the corresponding local blood flow image to obtain a blood flow image sequence in a unified coordinate system. The blood flow image fusion module is used to fuse blood flow image sequences in a unified coordinate system to obtain fundus blood flow images.

[0074] For relevant details, please refer to the above method implementation examples.

[0075] Figure 7This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 701 and a memory 702.

[0076] Processor 701 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 701 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 701 may also include a main processor and a coprocessor. The main processor performs calculations for processing sequences such as local temporal region segmentation, local registration, local blood flow image extraction, global unified registration, synchronous application of spatial transformation parameters, and blood flow image fusion. The coprocessor assists the main processor in performing some calculations in image preprocessing, image registration, or image fusion. In some embodiments, processor 701 may also integrate a GPU (Graphics Processing Unit), which performs image processing operations such as pixel-level difference calculation, spatial transformation, interpolation, blood flow response enhancement, and blood flow image display rendering of fundus laser images. In some embodiments, processor 701 may further include an AI (Artificial Intelligence) processor for performing computational operations related to image feature extraction, image enhancement, or vascular structure recognition.

[0077] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the memory 702 is used to store a sequence of fundus laser images arranged in the acquisition sequence, local structural images, local blood flow images, spatial transformation parameters, a blood flow image sequence in a unified coordinate system, and the finally obtained fundus blood flow image. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one instruction, which is loaded and executed by the processor 701 to implement the cascaded registration-based fundus blood flow imaging method provided in the embodiments of this application.

[0078] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the fundus blood flow imaging method based on cascaded registration described in the above method embodiments.

[0079] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the cascaded registration-based fundus blood flow imaging method of the above-described method embodiments.

[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A fundus blood flow imaging method based on cascaded registration, characterized in that, The method includes: Acquire a sequence of fundus laser images arranged in chronological order of acquisition; Based on the fundus laser image sequence, multiple overlapping local time regions are divided according to the acquisition time sequence. Fundus laser images in each local time region are locally registered, and local structural images and local blood flow images corresponding to the local time regions are obtained based on the locally registered fundus laser images. Global unified registration is performed based on each of the local structural images to determine the spatial transformation parameters corresponding to each of the local structural images, and each of the spatial transformation parameters is synchronously applied to the corresponding local blood flow image to obtain a blood flow image sequence in a unified coordinate system. The blood flow image sequence under the unified coordinate system is fused to obtain the fundus blood flow image.

2. The fundus blood flow imaging method based on cascaded registration according to claim 1, characterized in that, The process of dividing the fundus laser image sequence into multiple overlapping local time regions according to the acquisition time sequence includes: According to the preset local window length and preset sliding step size, multiple consecutive frames of fundus laser images are selected sequentially from the fundus laser image sequence along the acquisition time sequence to obtain multiple local time regions; Each of the local time regions includes multiple frames of continuously acquired fundus laser images, and at least one identical fundus laser image is contained between two adjacent local time regions.

3. The fundus blood flow imaging method based on cascaded registration according to claim 1, characterized in that, The step of locally registering the fundus laser images within each local time region and obtaining a local structural image corresponding to the local time region based on the locally registered fundus laser images includes: For each local time region, one frame of fundus laser image is selected from multiple frames of fundus laser images within the local time region as a local reference image; Register the remaining fundus laser images within the local time region, excluding the local reference image, to the coordinate system where the local reference image is located, to obtain the locally registered image sequence corresponding to the local time region; The local reference image is determined as a local structural image corresponding to the local time region.

4. The fundus blood flow imaging method based on cascaded registration according to claim 1, characterized in that, Based on the locally registered fundus laser image, a local blood flow image corresponding to the local time region is obtained, including: Based on the original image difference information and filter domain difference information between locally registered fundus laser images, a basic blood flow response is constructed. The enhanced blood flow response is obtained by performing nonlinear mapping on the basic blood flow response; Based on the image intensity information of the locally registered fundus laser image, the enhanced blood flow response is subjected to brightness normalization processing to obtain a local blood flow image corresponding to the local time region.

5. The fundus blood flow imaging method based on cascaded registration according to claim 1, characterized in that, The step of performing global unified registration based on each of the local structural images to determine the spatial transformation parameters corresponding to each of the local structural images includes: Select one frame of local structure image from each of the aforementioned local structure images as a global reference image; For each frame of target local structure image other than the global reference image, feature point detection and feature point matching are performed on the target local structure image and the global reference image to obtain matching feature point pairs between the target local structure image and the global reference image. Based on the matching feature point pairs, the spatial transformation relationship of the target local structure image relative to the coordinate system of the global reference image is determined; Based on the spatial transformation relationship, the target local structure image is registered to the coordinate system where the global reference image is located, and the transformation parameters corresponding to the spatial transformation relationship are determined as the spatial transformation parameters corresponding to the target local structure image.

6. The fundus blood flow imaging method based on cascaded registration according to claim 1, characterized in that, The step of synchronously applying each of the spatial transformation parameters to the corresponding local blood flow image to obtain a blood flow image sequence in a unified coordinate system includes: Based on the relationship between generating a frame of local structural image and a frame of local blood flow image corresponding to the same local time region, a one-to-one correspondence between each of the local structural images and each of the local blood flow images is determined; For each frame of the target local blood flow image, spatial transformation parameters corresponding to the local structure image that has a one-to-one correspondence with the target local blood flow image are obtained; According to the obtained spatial transformation parameters, the target local blood flow image is spatially transformed to the coordinate system of the global reference image. The local blood flow images, transformed to the coordinate system of the global reference image, are arranged according to the acquisition sequence of the corresponding local time regions to obtain a blood flow image sequence in a unified coordinate system.

7. The fundus blood flow imaging method based on cascaded registration according to claim 1, characterized in that, The process of fusing the blood flow image sequence in the unified coordinate system to obtain the fundus blood flow image includes: In the blood flow image sequence under the unified coordinate system, determine the pixels with the same coordinate position in each blood flow image; The pixel values ​​of multiple pixels with the same coordinate position are fused to obtain the fused pixel value at the corresponding coordinate position; The fundus blood flow image is generated based on the fused pixel values ​​at each coordinate position.

8. A fundus blood flow imaging system based on cascaded registration, characterized in that, include: The image sequence acquisition module is used to acquire a sequence of fundus laser images arranged in the acquisition time sequence; The local registration module is used to divide the fundus laser image sequence into multiple overlapping local time regions according to the acquisition time sequence, perform local registration on the fundus laser images in each local time region, and obtain local structural images and local blood flow images corresponding to the local time regions based on the locally registered fundus laser images. The global registration module is used to perform global unified registration based on each of the local structural images, determine the spatial transformation parameters corresponding to each of the local structural images, and synchronously apply each of the spatial transformation parameters to the corresponding local blood flow images to obtain a blood flow image sequence in a unified coordinate system. The blood flow image fusion module is used to fuse blood flow image sequences under the unified coordinate system to obtain fundus blood flow images.

9. An electronic device, characterized in that, The device includes a processor and a memory; the memory stores a program that is loaded and executed by the processor to implement a cascaded registration-based fundus blood flow imaging method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement a fundus blood flow imaging method based on cascaded registration as described in any one of claims 1 to 7.

Images