An image registration data processing method and system

Abstract

The application provides an image registration data processing method and system, relates to the technical field of image registration, and obtains a plurality of structural feature images in a plurality of monochrome laser images; according to the plurality of structural feature images, a plurality of ghosting areas with excessive eye movement are obtained, each of the ghosting areas includes a corresponding ghosting floating image and a ghosting reference image; a feature point corresponding relationship is obtained; a plurality of feature point pairs with accuracy greater than a preset threshold are obtained, feature point matching processing is performed, and a processed ghosting floating image is obtained; pixel registration processing is performed, and a region registration image is obtained; a plurality of region registration images are aligned and fused, a reference image and a floating image are combined, and a registration image is obtained. The application solves the technical problems of the prior art, such as inconsistent image offset in the image acquisition process, low whole-image registration accuracy, and the like, and achieves the technical effects of correcting the partial image offset caused by patient eye movement and improving real-time registration accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image registration, and in particular to an image registration data processing method and system. BACKGROUND

[0002] In recent years, with the change of lifestyle and the increase of overuse of eyes, the demand for treatment of ophthalmic diseases is increasing, and the market scale is rapidly expanding. In the process of diagnosing ophthalmic diseases, since the blood vessels of the fundus are the only blood vessels that can be directly observed through the body surface of the human body, medical personnel checks whether the optic nerve, retina and other parts of the fundus have lesions through a fundus camera.

[0003] At present, through laser scanning confocal imaging technology, a pinhole is introduced at the imaging focal plane to realize that the laser point light source, the retina and the point detector are at the conjugate position, and the stray light of the non-focal plane of the retina is effectively filtered out. It can provide optical tomography capability and high-resolution dynamic imaging that the fundus camera does not have, and the lateral resolution reaches microns.

[0004] However, the images obtained by monochrome laser shooting are all gray-scale images, and the gray-scale images cannot present the fundus lesions. Therefore, multiple laser sources are usually used to image the same part of the fundus, and the fundus color photo is synthesized through image processing technology. Since different tissues of the fundus respond differently to different wavelengths of laser, the monochrome laser images are not consistent in terms of gray-scale distribution and tissue response. Therefore, an image registration algorithm based on feature point matching is usually used to extract and match the same structural features of two images. However, since the monochrome laser image of the laser scanning confocal imaging technology is very sensitive to the eye movement of the patient, it often presents a certain longitudinal scanning point as a dividing point, and the motion states of the left and right parts of the monochrome laser image are inconsistent during data acquisition. Therefore, the offset of the floating image and the reference image is not consistent. This leads to the fact that the final synthesized fundus color photo may have partial ghosting, which cannot guarantee the quality of the color photo. In addition, due to the large offset, it takes a long time to find the feature points, which cannot meet the real-time requirement. The prior art has the technical problems of inconsistent image offset of the patient during image acquisition, and low registration accuracy of the whole image. SUMMARY

[0005] The present application provides an image registration data processing method and system to solve the technical problems of inconsistent image offset of the patient during image acquisition, and low registration accuracy of the whole image in the prior art.

[0006] In view of the above problems, the present application provides an image registration data processing method and system.

[0007] In a first aspect, the present application provides an image registration data processing method, wherein the method comprises: obtaining a plurality of structural feature images in a plurality of monochromatic laser images, wherein the plurality of monochromatic laser images comprise a reference image and a floating image; obtaining a plurality of ghost regions with excessive eye movement according to the plurality of structural feature images, each of the plurality of ghost regions comprising a corresponding ghost floating image and a ghost reference image; for each of the plurality of ghost regions, obtaining a feature point correspondence relationship in the ghost floating image and the ghost reference image, wherein the feature point correspondence relationship comprises a plurality of pairs of feature points; obtaining a plurality of pairs of feature points in the feature point correspondence relationship with accuracy greater than a preset threshold; performing feature point matching processing on the ghost floating image to obtain a processed ghost floating image; performing pixel registration processing on the processed ghost floating image to obtain a region registration image; and performing alignment fusion processing on a plurality of region registration images in the plurality of ghost regions to obtain a registration image in combination with the reference image and the floating image.

[0008] In a second aspect, the present application provides an image registration data processing system, comprising: a feature image obtaining module configured to obtain a plurality of structural feature images in a plurality of monochromatic laser images, wherein the plurality of monochromatic laser images comprise a reference image and a floating image; a ghost region obtaining module configured to obtain a plurality of ghost regions with excessive eye movement according to the plurality of structural feature images, each of the plurality of ghost regions comprising a corresponding ghost floating image and a ghost reference image; a correspondence relationship obtaining module configured to, for each of the plurality of ghost regions, obtain a feature point correspondence relationship in the ghost floating image and the ghost reference image, wherein the feature point correspondence relationship comprises a plurality of pairs of feature points; a floating image obtaining module configured to obtain a plurality of pairs of feature points in the feature point correspondence relationship with accuracy greater than a preset threshold, and perform feature point matching processing on the ghost floating image to obtain a processed ghost floating image; a region image obtaining module configured to perform pixel registration processing on the processed ghost floating image to obtain a region registration image; and a registration image obtaining module configured to perform alignment fusion processing on a plurality of region registration images in the plurality of ghost regions to obtain a registration image in combination with the reference image and the floating image.

[0009] In a third aspect, the present application provides an eye fundus laser shadowgraph, comprising the image registration data processing system according to the second aspect.

[0010] The one or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0011] The method provided by the embodiment of the application obtains a plurality of structural feature images in a plurality of monochromatic laser images, wherein the plurality of monochromatic laser images include a reference image and a floating image, then obtains a plurality of ghosting areas with excessive eye movement according to the plurality of structural feature images, each of the plurality of ghosting areas includes a corresponding ghosting floating image and a ghosting reference image, then for each of the plurality of ghosting areas, a feature point correspondence relationship is obtained in the ghosting floating image and the ghosting reference image, wherein the feature point correspondence relationship includes a plurality of pairs of feature points, then a plurality of pairs of feature points with accuracy greater than a preset threshold in the feature point correspondence relationship are obtained, feature point matching processing is performed on the ghosting floating image, a processed ghosting floating image is obtained, then pixel registration processing is performed on the processed ghosting floating image, a region registration image is obtained, then a plurality of region registration images in the plurality of ghosting areas are aligned and fused, and a registration image is obtained in combination with the reference image and the floating image. The efficiency of the registration image is improved, and the technical effect of correcting the whole image offset caused by eye movement is achieved.

[0012] The above description is only a summary of the technical solutions of the application, in order to more clearly understand the technical means of the application, the specific embodiments of the application can be implemented according to the content of the description, and in order to make the above and other purposes, characteristics and advantages of the application more obvious and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 A flowchart of an image registration data processing method provided by the application is provided.

[0014] Figure 2 A flowchart of obtaining a plurality of structural feature images in a plurality of monochromatic laser images in an image registration data processing method provided by the application is provided.

[0015] Figure 3 A flowchart of obtaining a plurality of ghosting areas with excessive eye movement in an image registration data processing method provided by the application is provided.

[0016] Figure 4 A structural diagram of an image registration data processing system provided by the application is provided.

[0017] Explanation of reference signs: feature image obtaining module 11, ghosting area obtaining module 12, correspondence relationship obtaining module 13, floating image obtaining module 14, region image obtaining module 15, and registration image obtaining module 16. DETAILED DESCRIPTION

[0018] The application provides an image registration data processing method and system, which is used for solving the technical problems of inconsistent image offset and low whole image registration accuracy caused by eye movement of a patient during image acquisition in the prior art. The application effectively corrects the partial image offset caused by the eye movement of the patient, effectively solves the ghosting problem after synthesis, and improves the whole image registration accuracy and efficiency.

[0019] The acquisition, storage, use and processing of data in the technical solution of the application comply with the relevant provisions of national laws and regulations.

[0020] Hereinafter, the technical solutions in the application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application, and it should be understood that the application is not limited to the example embodiments described herein. Based on the embodiments of the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application. In addition, it should be noted that, for the convenience of description, only the parts related to the application are shown in the drawings, not all.

[0021] Embodiment one

[0022] As shown in the figure, the application provides an image registration data processing method, wherein the method comprises: Figure 1

[0023] Step S100: acquiring a plurality of structural feature images in a plurality of monochrome laser images, wherein the plurality of monochrome laser images comprise a reference image and a floating image;

[0024] Further, as shown in the figure, the plurality of structural feature images in the plurality of monochrome laser images are acquired, and the step S100 of the embodiment of the application further comprises: Figure 2

[0025] Step S110: performing Gaussian filter noise smoothing processing on the plurality of monochrome laser images to obtain a plurality of noise reduction processing images;

[0026] Step S120: judging whether the gray value of the plurality of noise reduction processing images is less than a preset gray value threshold; if yes, the noise reduction processing image is processed by using an adaptive histogram enhancement algorithm and a gamma curve stretching histogram processing, and if not, no processing is performed to obtain a plurality of processing images;

[0027] Step S130: performing blur enhancement processing on the plurality of processing images to obtain the plurality of structural feature images.

[0028] ​​Specifically, the fundus camera uses multiple laser sources to capture images of the patient's fundus from multiple angles and different positions to obtain the multiple monochrome laser images. The multiple monochrome laser images refer to multiple images of the fundus. The multiple structural feature images reflect the information of the blood vessels and the optic disc structure of the fundus from multiple different angles and different positions. The reference image refers to an image that remains unchanged during image registration of the multiple monochrome laser images. The floating image refers to an image that moves or rotates during image registration of the multiple monochrome laser images. The Gaussian filter noise smoothing processing is a weighted average of multiple monochrome laser images. Each pixel in the image is obtained by weighted average of itself and other pixel values in the neighborhood, thereby correcting the cumulative transmission error, suppressing noise, and obtaining the multiple noise reduction processing images. The multiple noise reduction processing images refer to images obtained after processing the noise in the image. The preset gray value threshold refers to the minimum value that the gray value of each pixel in the image needs to meet, which is set by the staff and is not limited here. When the gray value of the pixel in the image is lower than the preset gray value threshold, the image cannot clearly express the details of the image. The adaptive histogram enhancement algorithm processing is to consider the entropy and brightness mean difference of the image, and adaptively select a suitable threshold to divide the image into two sub-images for double histogram equalization and gray uniformization processing, thereby effectively enhancing the image and avoiding over-enhancement. The gamma curve stretching histogram processing is a nonlinear operation on the input image gray value, making the output image gray value and the input image gray value in exponential relationship, thereby enhancing the image. The blur processing is to superimpose the multiple processing images to enhance the features of the image. Then, the multiple structural feature images are extracted according to the results of the blur enhancement processing.

[0029] Specifically, by performing Gaussian filter noise smoothing processing on the multiple monochrome laser images, the inconsistency of the image gray scale detail distribution in the image can be processed, and the cumulative error in the image can be corrected. Further, by judging the gray value of the multiple noise reduction processing images, it can be determined whether the quality of the processed image can meet the requirements. The images can be processed in batches, which can meet the requirements for subsequent operations. If the requirements cannot be met, the gray value of the image can be improved by image enhancement processing. Then, the multiple processing images are obtained by blur enhancement processing, and the structural features in the image are extracted to lay the foundation for subsequent structural analysis and calibration.

[0030] Further, the multiple processing images are subjected to blur enhancement processing, and the step S130 of the embodiment of the application further includes:

[0031] Step S131: Perform blur filter processing on the multiple processed images to obtain multiple blurred processed images;

[0032] Step S132: Superimpose the multiple blurred images onto the multiple monochrome laser images according to a first preset ratio to obtain multiple first enhanced images;

[0033] Step S133: Superimpose the plurality of processed images and the plurality of first enhanced processed images according to a second preset ratio to obtain a plurality of second enhanced processed images;

[0034] Step S134: Obtain multiple grayscale thresholds based on the average grayscale values ​​of the multiple monochrome laser images;

[0035] Step S135: Extract the multiple second enhancement images according to the multiple grayscale value thresholds to obtain the multiple structural feature images.

[0036] Specifically, the multiple processed images undergoing blur filter processing refers to removing high-frequency components such as noise and boundaries from the images to achieve image blurring. The multiple blurred images refer to the images obtained after blurring multiple processed images to reduce the intensity of boundaries. The first preset ratio is the proportion of blurred processed images corresponding to the enhancement of multiple monochromatic laser images, i.e., the number of blurred processed images corresponding to one monochromatic laser image. The multiple first enhanced processed images are obtained after enhancing the monochromatic laser images. The second preset ratio is a pre-set superposition ratio between the images when multiple processed images are superimposed on the multiple first enhanced processed images. The second enhanced processed image refers to the image obtained after superimposing the processed image and the enhanced processed image according to a preset ratio. The average grayscale value is obtained by calculating the grayscale value corresponding to each pixel in each of the multiple monochromatic laser images and then performing average processing. Furthermore, based on the average grayscale value, the multiple grayscale value thresholds corresponding to the multiple monochromatic laser images are determined, thereby determining the grayscale thresholds corresponding to feature extraction for different monochromatic laser images.

[0037] Specifically, by performing blur processing on the plurality of processed images, the boundary of the image is not obvious enough, but at the same time, the larger and brighter pixel points in the image are retained, which can effectively remove noise. Therefore, by superimposing the plurality of blur-processed images on the plurality of monochrome laser images, the plurality of monochrome laser images are image-enhanced, and then the plurality of first enhanced processed images are superimposed with the plurality of processed images, so that the structure in the image is clearer. Further, by extracting the second enhanced processed image according to the plurality of gray value thresholds, feature extraction is performed according to different gray value thresholds, so that the plurality of structure feature images reflecting different structures are obtained. The technical effect of feature analysis of the image and obtaining a plurality of high-quality structure feature images is achieved, which lays the foundation for subsequent image registration.

[0038] Step S200: According to the plurality of structure feature images, a plurality of ghost regions with excessive eye movement are obtained, each of which includes a corresponding ghost floating image and a ghost reference image;

[0039] Further, as shown in Figure 3 According to the plurality of structure feature images, a plurality of ghost regions with excessive eye movement are obtained, and the embodiment step S200 of the application further includes:

[0040] Step S210: The plurality of structure feature images are longitudinally divided into a plurality of regions to obtain a plurality of {block1, block2, block3, …};

[0041] Step S220: According to the plurality of {block1, block2, block3, …}, the mutual information value difference of each reference image and each floating image in the plurality of regions is calculated to obtain a plurality of mutual information value difference sequences {d1, d2, d3, …};

[0042] Step S230: When the mutual information value difference starts to be less than or greater than the preset difference threshold continuously, the region is taken as the plurality of ghost regions.

[0043] Specifically, the plurality of ghost regions are regions with image overlap blur caused by eye movement during image acquisition, and the corresponding mutual information value difference of the region does not continuously meet the preset difference threshold. The ghost floating image is an image that is rotated or moved during image registration. The ghost reference image is a ghost image that remains unchanged during image registration. The plurality of regions are obtained by longitudinally dividing the plurality of structure feature images respectively. The plurality of mutual information value difference sequences are obtained by arranging the mutual information value difference between the reference image and the floating image in each region in order of the plurality of regions, and the plurality of mutual information value difference sequences correspond one-to-one to the plurality of regions.

[0044] Specifically, by equally dividing the plurality of structure feature images according to the longitudinal distance, a plurality of regions, i.e., the plurality of {block1, block2, block3, …} are obtained. Further, according to the mutual information value between the reference image and the floating image corresponding to each region, difference calculation is performed, and the mutual information value difference is analyzed. When the mutual information value difference starts to be continuously less than or greater than the preset difference threshold value, it indicates that the difference degree of the mutual information value between the reference image and the floating image corresponding to the region at this time cannot meet the requirements, and ghosting phenomenon occurs. Thus, the technical effects of accurately positioning the ghosting region and improving the image registration efficiency are achieved.

[0045] Step S300: For each ghosting region, the corresponding relationship of feature points in the ghosting floating image and the ghosting reference image is obtained, wherein the corresponding relationship of feature points includes a plurality of pairs of feature points.

[0046] Further, for each ghosting region, the corresponding relationship of feature points in the ghosting floating image and the ghosting reference image is obtained, and the embodiment of the present application step S300 further includes:

[0047] Step S310: For each ghosting region, the feature points of the ghosting reference image are extracted based on a feature point extraction algorithm to obtain a reference feature point set.

[0048] Step S320: The ghosting reference image and the ghosting floating image are respectively constructed into a reference image pyramid and a floating image pyramid through downsampling, wherein the ghosting reference image and the ghosting floating image are located at the bottom layer of the reference image pyramid and the floating image pyramid, respectively, and the image of the upper layer is obtained by 1 / 2 proportional downsampling based on the image of the lower layer.

[0049] Step S330: The plurality of feature points in the reference feature point set are corresponded to the highest layer in the reference image pyramid, and then the plurality of feature points are searched in the corresponding layer of the floating image pyramid to obtain a highest layer floating feature point set.

[0050] Step S340: According to the highest layer floating feature point set, pre-translation and residual calculation are performed on the next layer floating image to obtain a first feature point corresponding relationship of the next layer floating image.

[0051] Step S350: Based on the first feature point corresponding relationship, iterative calculation is performed on each layer floating image to obtain the corresponding relationship of feature points.

[0052] Specifically, the feature point extraction algorithm refers to an algorithm for extracting extreme points, i.e., key points, in an image to obtain the direction of the feature points. The reference feature point set refers to feature points in a ghost reference image that serve as a reference for image registration. The down-sampling reference image pyramid refers to reducing the reference image by reducing the original image while retaining effective information to reduce the dimension of the features, iteratively sampling down, thereby obtaining multiple layers, and the image is continuously reduced from the bottom up to form a reference image pyramid. For example, after the down-sampling step is performed, the length and width of the image are both reduced to one-half of the original, and the overall image is reduced to one-fourth of the original.

[0053] Specifically, by iteratively sampling, the multiple feature points in the reference feature point set are set in the highest layer of the reference image pyramid. Then, according to the one-to-one correspondence between the reference image and the floating image, searching is performed in the corresponding layer of the floating image pyramid, thereby finding the points corresponding to the multiple feature points in the highest layer of the reference image pyramid, and thereby obtaining the highest layer floating feature point set. The highest layer floating feature point set is the basic feature point for adjusting the image.

[0054] Specifically, according to the feature points in the highest layer floating feature point set, pre-translation and residual calculation are performed on the next layer floating image to obtain the accurate feature point correspondence in the next layer floating image. The first feature point correspondence indicates the feature point correspondence between the reference image and the floating image, and then the reference image pyramid is iterated layer by layer until the bottom of the reference image pyramid is reached, and the iteration is stopped, thereby obtaining the correspondence between the feature points of the original reference image and the floating image, i.e., the feature point correspondence. This achieves the technical effect of laying the foundation for accurate image registration and solving the problem of being unable to find corresponding feature points due to excessive offset between the reference image and the floating image.

[0055] Step S400: Obtain a plurality of pairs of feature points in the feature point correspondence whose accuracy is greater than a preset threshold, perform feature point matching processing on the ghost floating image, and obtain a processed ghost floating image.

[0056] Further, a plurality of pairs of feature points in the feature point correspondence whose accuracy is greater than a preset threshold are obtained, feature point matching processing is performed on the ghost floating image, and the embodiment step S400 of the present application further includes:

[0057] Step S410: based on the RANSAC algorithm, screening a plurality of pairs of feature points in the feature point correspondence relationship, and obtaining a plurality of pairs of feature points with accuracy greater than the preset threshold;

[0058] Step S420: based on the change matrix and the plurality of pairs of feature points, performing affine transformation processing on the ghost floating image to obtain the processed ghost floating image.

[0059] Specifically, there are a plurality of pairs of feature points in the feature point correspondence relationship. By screening the feature points according to the RANSAC algorithm, a plurality of pairs of feature points with accuracy greater than the preset threshold are collected with the preset threshold as the screening target. The change matrix is a change process matrix for adjusting the floating image according to the feature point correspondence relationship. According to the plurality of pairs of feature points on the ghost floating image and the change matrix, the ghost floating image is subjected to affine transformation processing, the positions of the feature points on the ghost floating image are adjusted through linear transformation, and then the ghost floating image is adjusted to obtain the processed ghost floating image. The technical effect of eliminating the influence of eye movement by transforming the image is achieved.

[0060] Step S500: performing pixel registration processing on the processed ghost floating image to obtain a regionally registered image;

[0061] Further, the pixel registration processing is performed on the processed ghost floating image, and the embodiment of the present application step S500 further comprises:

[0062] Step S510: based on the dense pyramid optical flow method, calculating the pixel-by-pixel optical flow (flox, floy) of the ghost floating image and the ghost reference image;

[0063] Step S520: according to the pixel-by-pixel optical flow (flox, floy), mapping the pixels in the ghost floating image to the processed ghost floating image to obtain the regionally registered image.

[0064] Specifically, the regionally registered image is an image obtained by supplementing image details after pixel registration processing of a graph. The dense pyramid optical flow method refers to obtaining the pixel-by-pixel optical flow (flox, floy) by calculating the offset between each pixel point in the ghost floating image and the ghost reference image through point-by-point matching of the image. The flox refers to the offset amount of the offset between the pixel points mapped on the x-axis, and the floy refers to the corresponding offset amount of the offset between the pixel points mapped on the y-axis. Further, the pixel points in the ghost floating image are moved according to the pixel-by-pixel optical flow (flox, floy) and mapped on the processed ghost floating image, thereby regionally registering details to obtain the regionally registered image. The technical effect of high-quality and efficient registration of images is achieved.

[0065] Step S600: aligning and fusing the plurality of regionally registered images in the plurality of ghost regions to obtain a registered image in combination with the reference image and the floating image.

[0066] Further, the plurality of regionally registered images in the plurality of ghost regions are aligned and fused, and the embodiment of the present application step S600 further comprises:

[0067] Step S610: aligning and fusing the plurality of regionally registered images to obtain a preliminary registered image;

[0068] Step S620: extracting an edge missing part ROI of the plurality of regionally registered images in the preliminary registered image to obtain a plurality of ROIs;

[0069] Step S630: Gaussian blurring processing the plurality of ROIs to obtain a plurality of blurred ROIs;

[0070] Step S640: adjusting the gray scale distribution of the reference image according to the pixel distribution of the floating image, and superimposing the plurality of blurred ROIs according to a third preset proportion to obtain img1;

[0071] Step S650: iteratively superimposing the plurality of blurred ROIs and the floating image according to the third preset proportion to obtain img2;

[0072] Step S660: pixel-by-pixel superimposing the img1, the img2 to the preliminary registered image to obtain the registered image.

[0073] Specifically, the registration image is a high-precision registration image obtained after eliminating the ghost caused by eye movement. The alignment fusion processing refers to aligning the multiple region registration images in the previous segmentation order and fusing the image content. The preliminary registration image refers to the image obtained by preliminarily processing the multiple region registration images. Further, the missing parts of the multiple region registration images in the preliminary registration image after alignment fusion are set as the region of interest, i.e., the ROI. The multiple ROIs reflect the missing parts of the preliminary registration image. Further, the multiple ROIs are subjected to Gaussian blur processing to eliminate noise and boundaries, and the multiple blurred ROIs are obtained. Further, the gray scale distribution of the reference image is adjusted according to the pixel distribution of the floating image, and the gray scale distribution of the reference image is adjusted according to the floating image. The multiple blurred ROIs are superimposed, and the third preset proportion is the proportion corresponding to the superposition of the reference image and the blurred ROI, i.e., the number of blurred ROIs superimposed on one reference image. Further, the floating image and the multiple blurred ROIs are iterated according to the third preset proportion, and the img2 is obtained. By pixel-by-pixel superimposition of the img1 and the img2 on the preliminary registration image, the edge missing parts in the preliminary registration image are supplemented, and the technical effect of improving the image precision is achieved.

[0074] The technical solutions provided in the present application have at least the following technical effects or advantages:

[0075] 1. The embodiment of the present application extracts multiple structural feature images in multiple monochrome laser images to obtain images that can reflect the structure of the fundus, including blood vessels and optic disc information structure, and deeply mines the multiple structural feature images to obtain multiple ghost regions caused by eye movement. By using the down-sampling construction method, the correspondence relationship between the feature points in the ghost floating image and the ghost reference image in each ghost region is collected, the target of providing a basis for subsequent registration is achieved, and then a plurality of feature points with accuracy greater than a preset threshold in the feature point correspondence relationship are selected. The feature point matching processing is performed on the ghost floating image to obtain a processed ghost floating image, and then the pixel registration processing is performed on the processed ghost floating image to obtain a region registration image. Further, the multiple region registration images in the multiple ghost regions are subjected to alignment fusion processing, and the registration image is obtained by combining the reference image and the floating image. The technical effects of improving the accuracy of image registration, correcting the image shift caused by eye movement, eliminating the influence of eye movement, and efficiently completing the full-image high-precision real-time registration are achieved.

[0076] 2. The embodiment of the present application smoothes noise by performing Gaussian filtering on multiple monochrome laser images to obtain multiple noise-processed images, improves the accuracy of subsequent analysis, and then determines whether the gray values of the multiple noise-processed images are less than a preset gray value threshold, thereby performing different processing on the images, if yes, the noise-processed images are processed using an adaptive histogram enhancement algorithm and a gamma curve stretching histogram processing, if no, no processing is performed, multiple processed images are obtained, and then the images are subjected to fuzzy enhancement processing to obtain multiple structural feature images. The technical effect of enhancing the image display effect and improving the accuracy and efficiency of subsequent image analysis is achieved.

[0077] Embodiment two

[0078] As Figure 4 shown, in order to more clearly explain the technical scheme of the image registration data processing method, the embodiment of the present application provides an image registration data processing system, specifically as follows:

[0079] The feature image obtaining module 11 is configured to obtain multiple structural feature images in the multiple monochrome laser images, wherein the multiple monochrome laser images include a reference image and a floating image.

[0080] The ghost region obtaining module 12 is configured to obtain multiple ghost regions of excessive eye movement according to the multiple structural feature images, wherein each ghost region includes a corresponding ghost floating image and a ghost reference image.

[0081] The corresponding relationship obtaining module 13 is configured to obtain a feature point corresponding relationship in the ghost floating image and the ghost reference image for each ghost region, wherein the feature point corresponding relationship includes multiple pairs of feature points.

[0082] The floating image obtaining module 14 is configured to obtain a plurality of pairs of feature points in the feature point corresponding relationship with accuracy greater than a preset threshold, perform feature point matching processing on the ghost floating image, and obtain a processed ghost floating image.

[0083] The region image obtaining module 15 is configured to perform pixel registration processing on the processed ghost floating image to obtain a region registration image.

[0084] The registration image obtaining module 16 is configured to perform alignment fusion processing on the multiple region registration images in the multiple ghost regions, combine the reference image and the floating image, and obtain a registration image.

[0085] Further, the system further comprises:

[0086] a denoising image obtaining unit, configured to perform Gaussian filter noise smoothing processing on the plurality of monochrome laser images to obtain a plurality of denoising processing images;

[0087] a gray value judgment unit, configured to judge whether the gray values of the plurality of denoising processing images are less than a preset gray value threshold; if yes, the denoising processing images are processed by using an adaptive histogram enhancement algorithm and a gamma curve stretching histogram processing, and if not, no processing is performed to obtain a plurality of processing images;

[0088] a structural feature image obtaining unit, configured to perform blur enhancement processing on the plurality of processing images to obtain the plurality of structural feature images.

[0089] Further, the system further comprises:

[0090] a blur processing image obtaining unit, configured to perform blur filter processing on the plurality of processing images to obtain a plurality of blur processing images;

[0091] an enhancement processing image obtaining unit, configured to superimpose the plurality of blur processing images on the plurality of monochrome laser images according to a first preset ratio to obtain a plurality of first enhancement processing images;

[0092] a second enhancement image obtaining unit, configured to superimpose the plurality of processing images and the plurality of first enhancement processing images according to a second preset ratio to obtain a plurality of second enhancement processing images;

[0093] a gray value threshold obtaining unit, configured to obtain a plurality of gray value thresholds according to the average gray value size of the plurality of monochrome laser images;

[0094] an image extracting unit, configured to extract the plurality of second enhancement processing images according to the plurality of gray value thresholds to obtain the plurality of structural feature images.

[0095] Further, the system further comprises:

[0096] a region division unit, configured to divide the plurality of structural feature images into a plurality of regions by longitudinal blocking to obtain a plurality of {block1, block2, block3, …};

[0097] a difference sequence obtaining unit configured to calculate a difference of mutual information values of each reference image and each floating image in the plurality of regions according to the plurality of {block1, block2, block3,...}, and obtain a plurality of mutual information value difference sequences {dl, d2, d3,...};

[0098] a ghost region setting unit configured to set a region where the difference of mutual information values starts to be continuously less than or greater than a preset difference threshold as the plurality of ghost regions.

[0099] Further, the system further comprises:

[0100] a reference feature set obtaining unit configured to extract feature points of a ghost reference image based on a feature point extraction algorithm for each ghost region, and obtain a reference feature point set;

[0101] an image pyramid constructing unit configured to construct a reference image pyramid and a floating image pyramid by down-sampling the ghost reference image and the ghost floating image respectively, wherein the ghost reference image and the ghost floating image are located at the bottom layer of the reference image pyramid and the floating image pyramid respectively, and an image of an upper layer is obtained by 1 / 2 scale down-sampling based on an image of a lower layer;

[0102] a highest layer feature point obtaining unit configured to correspond a plurality of feature points in the reference feature point set to a highest layer in the reference image pyramid, and search the plurality of feature points in a corresponding layer of the floating image pyramid, and obtain a highest layer floating feature point set;

[0103] a corresponding relationship obtaining unit configured to perform pre-translation and residual calculation on a floating image of a next layer according to the highest layer floating feature point set, and obtain a first feature point corresponding relationship of the floating image of the next layer;

[0104] an iterative calculation unit configured to perform iterative calculation on each layer of floating image based on the first feature point corresponding relationship, and obtain the feature point corresponding relationship.

[0105] Further, the system further comprises:

[0106] a feature point screening unit configured to screen a plurality of pairs of feature points in the feature point corresponding relationship based on a RANSAC algorithm, and obtain a plurality of pairs of feature points with accuracy greater than the preset threshold;

[0107] An affine transformation processing unit is configured to perform affine transformation processing on the ghosted floating image based on the transformation matrix and the pairs of feature points, to obtain a processed ghosted floating image.

[0108] Further, the system further comprises:

[0109] A light flow calculation unit is configured to calculate pixel-wise light flows (flox, floy) of the ghosted floating image and the ghosted reference image based on a dense pyramid optical flow method.

[0110] A regionally registered image obtaining unit is configured to map pixels in the ghosted floating image onto the processed ghosted floating image according to the pixel-wise light flows (flox, floy), to obtain a regionally registered image.

[0111] Further, the system further comprises:

[0112] A preliminary registered image obtaining unit is configured to perform alignment fusion processing on the plurality of regionally registered images, to obtain a preliminary registered image.

[0113] An edge missing portion obtaining unit is configured to extract edge missing portions ROIs of the plurality of regionally registered images in the preliminary registered image, to obtain a plurality of ROIs.

[0114] A Gaussian blur processing unit is configured to perform Gaussian blur processing on the plurality of ROIs, to obtain a plurality of blurred ROIs.

[0115] An img1 obtaining unit is configured to adjust a gray scale distribution of a reference image according to a pixel distribution of the floating image, and superimpose the plurality of blurred ROIs on the reference image according to a third preset ratio, to obtain img1.

[0116] An img2 obtaining unit is configured to iteratively superimpose the plurality of blurred ROIs on the floating image according to the third preset ratio, to obtain img2.

[0117] A pixel superimposition unit is configured to perform pixel-wise superimposition of the img1 and the img2 on the preliminary registered image, to obtain the registered image.

[0118] Embodiment Three

[0119] Based on the same inventive concept as the image registration data processing system in Embodiment Two, this embodiment also provides an ocular fundus laser retinal imaging device, which comprises the image registration data processing system as described in Embodiment Two.

[0120] Any of the above described methods or steps can be stored as computer instructions or programs in various types of computer memories, recognized by various types of computer processors, and implemented accordingly.

[0121] Based on the above specific embodiments of the present application, any improvements and modifications made by those skilled in the art to the present application without departing from the principles of the present application shall fall within the scope of the patent protection of the present application.

Claims

1. An image registration data processing method, characterized by, The method comprises Sequentially performing noise reduction, selective enhancement and blur enhancement processing on a plurality of monochrome laser images to obtain a plurality of structural feature images, wherein the plurality of monochrome laser images include a reference image and a floating image; According to the mutual information value difference sequence obtained after longitudinal blocking of the plurality of structural feature images, one or more regions in which the mutual information value difference is continuously not consistent with a preset threshold are identified as a plurality of ghosting regions generated by excessive eye movement, each ghosting region including a corresponding ghosting floating image and a ghosting reference image; For each ghosting region, obtain the feature point correspondence in the ghosting floating image and the ghosting reference image, wherein the feature point correspondence includes a plurality of pairs of feature points; Obtain a plurality of pairs of feature points in the feature point correspondence whose accuracy is greater than a preset threshold, perform feature point matching processing on the ghosting floating image to obtain a processed ghosting floating image; Perform pixel registration processing on the processed ghosting floating image to obtain a regionally registered image; Perform alignment fusion processing on a plurality of regionally registered images in the plurality of ghosting regions, combine the reference image and the floating image, and obtain a registered image.

2. The method of claim 1, wherein, Obtaining a plurality of structural feature images in a plurality of monochrome laser images comprises: Performing Gaussian filter noise smoothing processing on the plurality of monochrome laser images to obtain a plurality of noise reduction processing images; Judging whether the gray value of the plurality of noise reduction processing images is less than a preset gray value threshold; if yes, the noise reduction processing image is processed using an adaptive histogram enhancement algorithm and a gamma curve stretching histogram processing, and if not, no processing is performed to obtain a plurality of processing images; Performing blur enhancement processing on the plurality of processing images to obtain the plurality of structural feature images.

3. The method of claim 2, wherein, The blur enhancement processing on the plurality of processing images comprises: Performing blur filter processing on the plurality of processing images to obtain a plurality of blur processing images; Stacking the plurality of blur processing images onto the plurality of monochrome laser images according to a first preset ratio to obtain a plurality of first enhancement processing images; Stacking the plurality of processing images and the plurality of first enhancement processing images according to a second preset ratio to obtain a plurality of second enhancement processing images; Obtaining a plurality of gray value thresholds according to the average gray value size of the plurality of monochrome laser images; Extracting the plurality of second enhancement processing images according to the plurality of gray value thresholds to obtain the plurality of structural feature images.

4. The method of claim 1, wherein, According to the plurality of structural feature images, obtaining a plurality of ghosting regions of excessive eye movement comprises: Longitudinally blocking the plurality of structural feature images to divide them into a plurality of regions to obtain a plurality of {block1, block2, block3, …}; According to the plurality of {block1, block2, block3, …}, calculating the mutual information value difference of each reference image and each floating image in the plurality of regions to obtain a plurality of mutual information value difference sequences {d1, d2, d3, …}; When the mutual information value difference starts to be continuously less than or greater than a preset difference threshold, the region is the plurality of ghosting regions.

5. The method of claim 1, wherein, For each ghost region, the correspondence relationship between the feature points in the ghost floating image and the ghost reference image is obtained, comprising: For each ghost region, the feature points of the ghost reference image are extracted based on a feature point extraction algorithm to obtain a set of reference feature points; The ghost reference image and the ghost floating image are respectively constructed into a reference image pyramid and a floating image pyramid through downsampling, wherein the ghost reference image and the ghost floating image are located at the bottom layer of the reference image pyramid and the floating image pyramid respectively, and the image at the upper layer is obtained by 1 / 2 proportional downsampling based on the image at the lower layer; A plurality of feature points in the set of reference feature points are corresponded to the highest layer in the reference image pyramid, and then the plurality of feature points are searched in the corresponding layer of the floating image pyramid to obtain a set of highest layer floating feature points; According to the set of highest layer floating feature points, a first feature point correspondence relationship of the next layer floating image is obtained through pre-translation and residual calculation of the next layer floating image; Based on the first feature point correspondence relationship, an iterative calculation is performed for each layer of floating image to obtain the feature point correspondence relationship.

6. The method of claim 1, wherein, The accuracy of a plurality of feature point pairs in the feature point correspondence relationship is greater than a preset threshold, and a feature point matching process is performed on the ghost floating image, comprising: Based on the RANSAC algorithm, a plurality of feature point pairs in the feature point correspondence relationship are screened to obtain the plurality of feature point pairs with accuracy greater than the preset threshold; Based on the change matrix and the plurality of feature point pairs, an affine transformation process is performed on the ghost floating image to obtain the processed ghost floating image.

7. The method of claim 1, wherein, The pixel registration process is performed on the processed ghost floating image, comprising: Based on the dense pyramid optical flow method, the pixel-by-pixel optical flow (flox, floy) of the ghost floating image and the ghost reference image is calculated; According to the pixel-by-pixel optical flow (flox, floy), the pixels in the ghost floating image are mapped to the processed ghost floating image to obtain the region registration image.

8. The method of claim 1, wherein, The alignment fusion process is performed on a plurality of region registration images in the plurality of ghost regions, comprising: The plurality of region registration images are aligned and fused to obtain a preliminary registration image; The edge missing part ROI of the plurality of region registration images in the preliminary registration image is extracted to obtain a plurality of ROIs; The plurality of ROIs are subjected to Gaussian blur processing to obtain a plurality of blurred ROIs; The gray scale distribution of the reference image is adjusted according to the pixel distribution of the floating image, and the plurality of blurred ROIs are superimposed according to a third preset proportion to obtain img1; The plurality of blurred ROIs and the floating image are iterated according to the third preset proportion to obtain img2; The img1, the img2 are pixel-by-pixel superimposed to the preliminary registration image to obtain the registration image.

9. An image registration data processing system characterized by, The system comprises: The feature image obtaining module is configured to sequentially perform noise reduction, selective enhancement and blur enhancement processing on the plurality of monochromatic laser images to obtain a plurality of structural feature images, wherein the plurality of monochromatic laser images include a reference image and a floating image; The ghost region obtaining module is configured to identify one or more regions in which the difference in mutual information values does not continuously meet a preset threshold as a plurality of ghost regions caused by excessive eye movement according to a difference sequence of mutual information values obtained after longitudinal blocking of the plurality of structural feature images, each of the ghost regions including a corresponding ghost floating image and a ghost reference image; The corresponding relationship obtaining module is configured to obtain a feature point corresponding relationship in the ghost floating image and the ghost reference image for each of the ghost regions, wherein the feature point corresponding relationship includes a plurality of pairs of feature points; The floating image obtaining module is configured to obtain a plurality of pairs of feature points in the feature point corresponding relationship that have an accuracy greater than a preset threshold, perform feature point matching processing on the ghost floating image, and obtain a processed ghost floating image; The region image obtaining module is configured to perform pixel registration processing on the processed ghost floating image to obtain a region registration image; The registration image obtaining module is configured to perform alignment and fusion processing on a plurality of region registration images in the plurality of ghost regions, combine the reference image and the floating image, and obtain a registration image.

10. An ophthalmic fundus laser projection device, characterized by, The fundus laser shadowgraph includes the image registration data processing system of claim 9.

Images