A fluorescence lifetime microscopic image large field-of-view splicing method, system and electronic device

Abstract

The present application relates to the technical field of image processing, and discloses a fluorescence lifetime microscopic image large field-of-view splicing method, system and electronic equipment, the method comprising: constructing a tissue area mask for the image to be spliced, constructing a dark corner model in the tissue area to complete dark corner correction; determining a global brightness reference by counting the brightness index in the tissue area, realizing image group brightness uniformity through global scaling and single image brightness adaptive correction; further realizing adjacent image alignment through feature point matching, combining a minimum color difference seam line fusion method to complete image block splicing to obtain an image strip; extracting an overlapping area from the image strip to generate an effective content mask, estimating the relative displacement by using the phase correlation method after low-frequency suppression, and completing image strip splicing to output a large field-of-view spliced image by using the minimum color difference seam line fusion method, which improves the automation degree and robustness of splicing, and the spliced image is geometrically consistent and visually seamless, meeting the demand of medical research for high-resolution large field-of-view fluorescence lifetime microscopic images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method, system, and electronic device for large field-of-view stitching of fluorescence lifetime micrographs. Background Technology

[0002] Fluorescence lifetime microscopy (FLIM) is an important imaging technique in the biomedical field. However, due to the limitation of the spatial bandwidth product of the optical system, high-resolution imaging can only achieve small field-of-view acquisition. Image stitching technology, which fuses multiple small field-of-view images into a large field-of-view image, has become a key way to make this technology practical.

[0003] Image stitching technology is mature in the field of real-world imaging, but dedicated stitching solutions for fluorescence lifetime microscopy images are still scarce. Conventional stitching methods have many technical defects when applied directly, making it difficult to meet the accuracy requirements of microscopic imaging. On the one hand, fluorescence lifetime microscopy images are susceptible to vignetting due to the optical system, exhibiting a brightness distribution that is bright in the center and dark at the edges. Furthermore, fluctuations in the imaging light source and differences in exposure can lead to uneven brightness within the same group of images. Existing algorithms do not incorporate the tissue region characteristics of the microscopic image for targeted correction, but only perform local brightness smoothing on overlapping areas, failing to fundamentally eliminate stitching banding caused by vignetting and uneven brightness.

[0004] On the other hand, conventional image alignment methods rely on feature point matching techniques such as Harris and SIFT. Fluorescence lifetime microscopy images have limited texture, low contrast, and are subject to noise interference, making them prone to unstable feature point matching and low alignment accuracy. Methods used in the fusion stage, such as weighted averaging and pyramid fusion, either result in blurred or ghosting stitching boundaries, or the fusion effect depends heavily on alignment accuracy, making them unsuitable for the stitching requirements of microscopic images. Furthermore, existing methods lack layered stitching strategies adapted to microscopic images, leading to large cumulative stitching errors and image distortion, thus failing to achieve high-resolution, artifact-free, large-field-of-view fluorescence lifetime microscopy image stitching. Summary of the Invention

[0005] In view of this, in order to solve the problems of incomplete vignetting correction, poor brightness consistency, and low alignment and fusion accuracy of existing fluorescence lifetime microscopy image large field-of-view stitching technology, this invention provides a fluorescence lifetime microscopy image large field-of-view stitching method, system, and electronic device, which can realize high-quality large field-of-view fluorescence lifetime microscopy image stitching and meet the needs of medical research for high-resolution large field-of-view fluorescence lifetime microscopy images.

[0006] In a first aspect, the present invention provides a method for large-field stitching of fluorescence lifetime micrographs, comprising:

[0007] A tissue region mask is constructed for the images within the fluorescence lifetime microscopic image group to be stitched together. A vignetting model is constructed within the tissue region based on the mask. The correction coefficient is calculated in combination with the image center brightness reference to complete the vignetting correction in the tissue region.

[0008] For the image group after vignetting correction, the brightness index of each image is statistically analyzed in the organization area and a global brightness benchmark is determined. The brightness of the image group is unified through global brightness scaling and adaptive correction of single image brightness.

[0009] After unifying the brightness of the image, adjacent images are aligned by feature point matching based on spatial location. Combined with the stitch fusion method based on minimum color difference, the image blocks are horizontally or vertically stitched together to obtain the image strip.

[0010] The overlapping regions of the image bands are extracted and an effective content mask is generated. After low-frequency suppression, the relative displacement of adjacent image bands is estimated by the phase correlation method. The stitching of the image bands is completed by reusing the stitching method with the minimum color difference, and a large field-of-view fluorescence lifetime microscopic stitched image is output.

[0011] The large-field-of-view stitching method for fluorescence lifetime microscopy images provided in this invention first eliminates global brightness unevenness caused by optical vignetting by modeling and correcting vignetting through tissue region mask constraints. Then, it unifies the brightness of the entire image and individual images to achieve overall consistency and local balance in the brightness of the image group, laying a high-quality image foundation for subsequent stitching. Next, it achieves precise alignment of image blocks through feature point matching, and reduces stitching gaps by combining a suture fusion method with minimal color difference. Finally, it completes the large-field-of-view stitching by using phase correlation to achieve image band registration and reuse the fusion strategy. The entire process balances geometric alignment accuracy and visual fusion effect, effectively suppresses stitching artifacts, and maximizes the preservation of image structure and intensity continuity. It significantly improves the automation, robustness, and stitching quality of large-field-of-view stitching for fluorescence lifetime microscopy images, meeting the needs of medical research for high-resolution, seamless large-field-of-view images.

[0012] In one alternative implementation, the construction of the tissue region mask includes:

[0013] The image is converted to the Lab color space, and the color information of the a and b channels of the Lab color space is extracted and clustered. The first tissue region is obtained by identifying the background category.

[0014] The luminance information of the L luminance channel in the Lab color space is extracted, and the second tissue region is obtained based on adaptive threshold segmentation.

[0015] The union of the first and second organizational regions is taken as the organizational region mask.

[0016] This invention constructs a tissue region mask by fusing color clustering and brightness threshold segmentation, solving the problem of inaccurate differentiation between tissue and background in fluorescence lifetime micrographs using single segmentation methods. Based on clustering analysis of the a and b channels of the Lab color space, it accurately captures the chromaticity differences between tissue and background, effectively distinguishing regions with different chromaticity characteristics. Combined with adaptive threshold segmentation of the L brightness channel, it achieves accurate separation of tissue and background based on brightness differences. The union of the two regions is used as the final mask, which not only overcomes the shortcomings of single color clustering being susceptible to brightness interference and single brightness segmentation being susceptible to chromaticity similarity, but also maximizes the coverage of the real tissue region and effectively eliminates background interference. This mask defines a precise effective region for subsequent steps such as vignetting modeling and brightness correction, avoiding interference from background regions and significantly improving the accuracy and reliability of subsequent vignetting correction and brightness unification.

[0017] In one optional implementation, the step of constructing a vignetting model within the tissue region based on the mask, and calculating correction coefficients in conjunction with the image center brightness reference to complete vignetting correction within the tissue region, includes:

[0018] The brightness information within the tissue area is sampled, and outliers are removed from the sampled values ​​to obtain the filtered brightness values.

[0019] Logarithmic transformation is performed on the filtered brightness values, and a two-dimensional polynomial model is constructed. The coefficients of the two-dimensional polynomial model are estimated by the weighted least squares method to obtain a vignetting model that characterizes the attenuation trend of image brightness from the center to the edge.

[0020] The vignetting model is normalized based on the statistical brightness of the central region of the image, a brightness correction coefficient is calculated, and the correction coefficient is applied only to the tissue region of the image to complete the vignetting correction.

[0021] This invention samples the brightness of the tissue region and removes outliers to ensure that the modeling data accurately reflects the light attenuation trend and avoids interference from extremely bright or dark points in the model fitting. Logarithmic transformation combined with two-dimensional polynomial modeling accurately characterizes the low-frequency smooth attenuation characteristics of vignetting, resulting in high fitting accuracy. Normalization based on the center brightness and calculation of correction coefficients achieves accurate vignetting compensation, while applying the correction only to the tissue region, avoiding ineffective processing of the background region and preventing excessive amplification of edge noise. The corrected image has a uniform brightness distribution, effectively reducing the optical vignetting effect and fundamentally avoiding stitching artifacts caused by vignetting, providing a uniform brightness image foundation for subsequent image stitching.

[0022] In one optional implementation, the process of statistically analyzing the brightness indices of each image within the tissue region and determining a global brightness benchmark for the image group after vignetting correction, followed by global brightness scaling and adaptive brightness correction of individual images to achieve brightness uniformity for the image group, includes:

[0023] The image after vignetting correction is converted to grayscale, and the tissue region is obtained through preset adaptive threshold segmentation and morphological opening and closing operations.

[0024] The grayscale brightness distribution is statistically analyzed only within the organized area. The first preset percentile of the grayscale value is extracted as the main brightness index of the image, and the second preset percentile is used as a reference for the upper limit of brightness.

[0025] The global brightness benchmark is obtained by calculating the subject brightness index of all images using the truncated median method;

[0026] Based on the ratio of the global brightness benchmark to the brightness index of each image subject, calculate the global scaling factor and perform global linear scaling on the corresponding image;

[0027] Large-scale Gaussian smoothing is performed on the grayscale image of a single image to estimate the low-frequency brightness distribution. The adaptive correction factor of the single image is calculated within the organized region and its fluctuation range is limited to perform adaptive brightness correction of the single image, thereby achieving brightness uniformity of the image group.

[0028] This invention employs adaptive threshold segmentation and morphological operations to obtain stable tissue regions, ensuring the accuracy of brightness statistics. Specific percentiles are extracted as brightness indicators and upper limits to avoid outlier interference and accurately represent the true brightness of the image. The truncated median method determines the global benchmark, preventing extreme images from affecting the overall calibration fairness. Global linear scaling evens out the overall brightness between images, eliminating brightness discontinuities. Large-scale Gaussian smoothing combined with adaptive correction factor limiting achieves gentle brightness correction for individual images, smoothing out local unevenness without damaging tissue details. The final output image group exhibits globally consistent and locally smooth brightness, significantly improving the accuracy of subsequent feature point matching.

[0029] In one optional implementation, the image with unified brightness is aligned with adjacent images by feature point matching based on spatial location, and the image blocks are horizontally or vertically stitched together using a stitch fusion method based on minimum color difference to obtain an image strip, including:

[0030] The images are grouped and sorted in order according to the row and column index information in the image file name, and missing or invalid positions are filled with a completely black image.

[0031] Within the pre-defined width and height overlap area of ​​adjacent images, feature points and descriptors are extracted and feature matching is performed. The displacement difference of the matching point pairs is calculated and the median is used to estimate the translation parameters. A translation transformation matrix is ​​constructed to perform affine transformation on the images to achieve alignment of adjacent images. If feature matching fails or the image is empty, the original position of the image is kept unchanged.

[0032] The chromaticity difference of corresponding pixels in the overlapping area is calculated based on the Euclidean distance. The Euclidean distance is used as the cost function, and a dynamic programming method is used to search for the minimum cost path from top to bottom in the overlapping area as the optimal stitching seam. Constraints are applied to the edge area of ​​the seam.

[0033] A fusion mask is constructed based on the optimal stitching line, and pixel fusion is performed on the overlapping areas.

[0034] This invention sorts images by row and column indices and fills in invalid positions, ensuring the spatial structural integrity and orderly arrangement of image blocks and clarifying the matching relationship between adjacent images. It extracts feature points in overlapping areas and estimates translation parameters using the median, effectively eliminating interference from mismatched points and improving the accuracy and robustness of geometric alignment. A fallback strategy for feature matching failures ensures the continuity of the stitching process. A cost function is constructed based on the chromatic Euclidean distance between adjacent overlapping areas. Combined with dynamic programming to search for the optimal seam and apply edge constraints, the stitching boundary extends along the path with the least chromatic difference. Pixel fusion is then achieved through a fusion mask, effectively reducing stitching gaps and avoiding the blurring and ghosting problems of traditional fusion methods, thus ensuring the geometric consistency and visual continuity of the stitched image blocks.

[0035] In one optional implementation, the step of extracting overlapping regions from the image bands and generating an effective content mask, followed by low-frequency suppression and estimation of the relative displacement of adjacent image bands using a phase correlation method, includes:

[0036] The overlapping region of adjacent image band edges is extracted to construct an overlapping sub-image. The overlapping sub-image is then subjected to brightness threshold segmentation and morphological operations to generate an effective content mask.

[0037] The effective content mask is subjected to low-frequency suppression processing to highlight structural and edge information;

[0038] The phase correlation method is used to estimate the relative displacement of adjacent image bands in the horizontal and vertical directions for overlapping sub-images after low-frequency suppression.

[0039] This invention extracts overlapping regions at the edges of image bands to construct sub-images, reducing the registration calculation range and improving processing efficiency. By using brightness thresholding and morphological operations to generate an effective content mask, invalid regions such as black backgrounds are precisely removed, retaining only effective pixels of the tissue, thus eliminating interference from invalid regions on registration at the source. Low-frequency suppression processing highlights the structure of the tissue and high-frequency edge features, enhancing the feature recognition of registration and allowing the phase correlation method to accurately capture phase differences of homologous features. The phase correlation method based on frequency domain phase information has strong anti-interference capabilities and high registration accuracy, effectively estimating the relative displacement between image bands, laying a precise geometric alignment foundation for the seamless fusion of subsequent image bands.

[0040] In one optional implementation, the suture fusion method that reuses the minimum chromatic difference to complete image band stitching and output a large field-of-view fluorescence lifetime microscopic stitched image includes:

[0041] Based on the relative displacement, adjacent image bands are placed on the canvas in the same coordinate system. Non-overlapping areas are directly filled with the corresponding pixel values, while overlapping areas are fused using the minimum color difference stitching fusion method. The fusion operation is performed on all image bands one by one in a loop. After the stitching is completed, the black border is cropped, and a large field-of-view fluorescence lifetime microscopic stitched image is output.

[0042] This invention places image strips on the same coordinate system canvas based on relative displacement, ensuring the spatial geometric integrity of the large field-of-view image. Non-overlapping areas are directly filled with pixel values, maximizing the retention of effective organizational information without information loss. Overlapping areas reuse the validated stitching fusion method with minimal color difference, ensuring consistency and naturalness of the fusion effect while avoiding redundant algorithm development and improving stitching efficiency. Frame-by-frame cyclic fusion achieves orderly integration of multiple image strips, effectively reducing cumulative stitching errors. Black border cropping precisely removes invalid areas, ensuring the output image contains only effective organizational information without redundant pixels. The final output large field-of-view image is geometrically consistent, visually seamless, and content-effective, meeting the analytical needs of medical research for high-resolution, complete large field-of-view fluorescence lifetime microscopic images.

[0043] Secondly, the present invention provides a large field-of-view stitching system for fluorescence lifetime microscopic images, the system comprising:

[0044] The vignetting correction module is used to construct a tissue region mask for the images within the fluorescence lifetime microscopic image group to be stitched together. Based on the mask, a polynomial vignetting model is built in the tissue region, and the correction coefficient is calculated in combination with the image center brightness reference to complete the vignetting correction in the tissue region.

[0045] The brightness unification module is used to statistically analyze the brightness index of each image within the organized area and determine the global brightness benchmark for the image group after vignetting correction. The brightness unification of the image group is achieved through global brightness scaling and adaptive correction of single image brightness.

[0046] The image strip generation module is used to align adjacent images by matching feature points based on their spatial location after the brightness is unified. Combined with the stitch fusion method based on minimum color difference, it completes the horizontal or vertical stitching of image blocks to obtain image strips.

[0047] The image stitching output module is used to extract the overlapping area of ​​the image band and generate an effective content mask. After low-frequency suppression, the relative displacement of adjacent image bands is estimated by the phase correlation method. The stitching of the image bands is completed by reusing the stitch fusion method with the minimum color difference, and a large field-of-view fluorescence lifetime microscopic stitched image is output.

[0048] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the fluorescence lifetime microscopy image large field-of-view stitching method described in the first aspect or any corresponding embodiment thereof.

[0049] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the fluorescence lifetime microscopy image large field-of-view stitching method of the first aspect or any corresponding embodiment described above.

[0050] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the fluorescence lifetime microscopy image large field-of-view stitching method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0051] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating a method for large-field-of-view stitching of fluorescence lifetime micrographs according to an embodiment of the present invention.

[0053] Figure 2 This is a schematic diagram comparing the fluorescence lifetime of intestinal tissue before and after vignetting correction and brightness unification according to an embodiment of the present invention.

[0054] Figure 3 This is a schematic diagram of the splicing result according to an embodiment of the present invention;

[0055] Figure 4 This is a structural block diagram of a fluorescence lifetime microscopic image large field-of-view stitching system according to an embodiment of the present invention;

[0056] Figure 5 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0059] According to an embodiment of the present invention, a method for stitching large field-of-view fluorescence lifetime micrographs is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here. Figure 1 This is a flowchart of a large field-of-view stitching method for fluorescence lifetime micrographs according to an embodiment of the present invention, as follows: Figure 1 As shown, the specific steps are as follows:

[0060] Step S1: Construct a tissue region mask for the images within the fluorescence lifetime microscopic image group to be stitched together. Based on the mask, construct a vignetting model within the tissue region. Combine the image center brightness reference to calculate the correction coefficient and complete the vignetting correction in the tissue region.

[0061] Specifically, in this embodiment of the invention, after acquiring a group of fluorescence lifetime microscopic images to be stitched together, the process of constructing tissue region masks for the images within the group includes:

[0062] S11, convert the image to the Lab color space, extract the color information of the a and b channels of the Lab color space for clustering, and obtain the first organization region by identifying the background category.

[0063] Specifically, the original format of fluorescence lifetime microscopy images is the BGR color space, where the blue, green, and red channels are coupled data of brightness and chromaticity. The Lab color space, however, splits image information into an L channel (representing only brightness), an a channel (red-green hue), and a b channel (yellow-blue hue). Channels a and b are pure chromaticity channels, completely unrelated to brightness. Converting the image to Lab space allows for the extraction of a and b channels for clustering (e.g., using K-means clustering, K=3). The clustering algorithm divides pixels into several categories based on color similarity. In fluorescence microscopy images, there are significant inherent differences in the fluorescence staining colors of the background (such as slides or blank areas) and tissue areas (such as diseased tissue or biological sections). The algorithm naturally groups pixels with similar colors into the same category, achieving color separation between tissue and background.

[0064] After clustering is completed, based on a preset background color reference (such as the blank background color of the slide, the color characteristics of non-fluorescent areas), the background category is identified from the clustering results. This is the set of all pixels belonging to the background. All pixel categories other than the background category in the clustering results are merged, and the resulting pixel set is the first tissue region M1. This region is a tissue range segmented solely based on color features, completely eliminating the interference of uneven brightness. It can accurately fit the color boundaries of the tissue and avoid the problem of edge tissue being misjudged as background due to vignetting.

[0065] In one example, such as a fluorescence lifetime micrograph of a glioma section, the background is a light gray slide (a and b values ​​close to 0), and the glioma tissue is marked red with a fluorescent dye (large a value, small b value). After converting to Lab color space, the a and b channels are extracted and clustered. The algorithm clusters according to a given K value, calculates the chromaticity difference between the cluster color and the reference background, and sets the class that is close to the reference background as the background. That is, pixels with a and b values ​​close to 0 are clustered into the background class, and pixels with red features are clustered into the tissue class. After identifying the background class, the remaining set of red pixels is the first tissue region mask, which accurately segments the glioma tissue and is not affected by the brightness reduction caused by the dark corners of the image edges.

[0066] S12, extract the luminance information of the L luminance channel in the Lab color space, and obtain the second tissue region based on adaptive threshold segmentation.

[0067] Specifically, in fluorescence lifetime microscopy images, in addition to color differences, there are also significant brightness differences between tissue areas (such as stained biological sections) and background areas (such as blank slides). Tissues exhibit higher brightness due to fluorescence staining, while the background has lower brightness and a more uniform distribution. The L channel of the Lab color space has a value range of 0~100 (0 is pure black, 100 is pure white). Directly extracting this channel converts the image into a grayscale image containing only brightness information. Each pixel value corresponds to its brightness intensity, completely eliminating the chromaticity interference of the a and b channels and retaining only the characteristics of the brightness and darkness dimensions.

[0068] Furthermore, the Otsu algorithm used in this embodiment of the invention is an unsupervised segmentation method for brightness distribution. It automatically finds the optimal segmentation threshold based on the image brightness distribution. The specific process is as follows: The brightness histogram of the L-channel grayscale image is statistically analyzed, which represents the distribution of the number of pixels with different brightness values. The algorithm iterates through all possible thresholds, calculates the inter-class variance of pixels (background / tissue) on both sides of the threshold, and takes the threshold with the largest inter-class variance as the optimal segmentation threshold (the larger the inter-class variance, the more significant the brightness difference between the background and tissue). Using the optimal threshold as the boundary, pixels with brightness higher than the threshold in the L-channel image are determined as tissue pixels, and pixels with brightness lower than the threshold are determined as background pixels. The set of all tissue pixels is the second tissue region mask. For example, the average brightness of the L-channel of the background (slide) in the fluorescence lifetime micrograph of a tissue section is about 20, and the average brightness of the L-channel of the tissue due to fluorescence staining is about 60, with a clear bimodal brightness distribution. The Otsu algorithm automatically calculates the optimal threshold of about 35, and determines pixels with brightness > 35 as tissue, obtaining the second tissue region M2.

[0069] S13, take the union of the first tissue region and the second tissue region as the tissue region mask.

[0070] This step combines the first organization region M1 segmented by the color dimension and the second organization region M2 segmented by the brightness dimension, integrating the advantages of dual-dimensional segmentation to obtain a complete and precise organization region mask. This completely eliminates background interference and provides reliable regional constraints for subsequent vignetting correction and brightness unification.

[0071] In this embodiment of the invention, luminance data is preprocessed within the tissue region based on the mask, and a vignetting model is constructed. A correction coefficient is then calculated using the image center luminance reference to complete vignetting correction within the tissue region, including:

[0072] S14, sample the brightness information within the tissue area, and remove outliers from the sampled values ​​to obtain the filtered brightness values.

[0073] Specifically, this embodiment of the invention considers that vignetting is a low-frequency spatial variation trend. To reduce computational complexity and avoid the computational burden caused by redundant sampling, the image brightness component is downsampled at a fixed step size, retaining only equally spaced sampling points for model estimation. In the sampled data, to reduce the interference of extremely dark and bright points on the fitting results, based on the statistical distribution of the sampled brightness values, the 5th percentile and 97th percentile are used as thresholds (only as examples, not as a limitation) to remove abnormal brightness values, thereby ensuring that the data used in modeling mainly reflects the overall illumination trend.

[0074] S15, perform a logarithmic transformation on the filtered brightness values, construct a two-dimensional polynomial model, and estimate the coefficients of the two-dimensional polynomial model using the weighted least squares method to obtain a vignetting model that characterizes the attenuation trend of image brightness from the center to the edge.

[0075] Specifically, since the optical vignetting in fluorescence microscopy images is a nonlinear brightness change caused by lens optical path attenuation, it exhibits an exponential decrease in brightness from the center to the edge. Multinomial models excel at fitting linear / low-order nonlinear continuous changes. Applying a natural logarithmic transformation to the selected brightness values ​​converts the exponential nonlinear brightness attenuation into an approximately linear brightness change, making the vignetting attenuation trend more closely match the fitting characteristics of the multinomial model and significantly improving the model's accuracy in fitting the vignetting trend.

[0076] In one example, the original filtered brightness value is I, and the logarithmically transformed brightness value is I. log The transformation formula is: I log =ln(I). The constructed two-dimensional polynomial model is represented as:

[0077]

[0078] Where n is the order of the polynomial (e.g., n=3), β ij The parameters to be estimated are x and y, which are normalized spatial coordinates to eliminate differences in image size and improve the stability of the values.

[0079] Furthermore, weighted least squares is used to estimate the polynomial parameters at each sampling point (x). k y kThe values ​​of all two-dimensional polynomial basis functions are calculated at each point, and arranged row-wise to form a design matrix A (with dimensions m×n, where m is the number of effective luminance sampling points within the organization region, and n is the number of basis function terms in the two-dimensional polynomial. For each luminance sampling point, the values ​​of all basis functions of the two-dimensional polynomial are calculated, and the values ​​of all basis functions for that sampling point are arranged as row vectors. The row vectors of all sampling points are combined sequentially to form the design matrix A, where each row corresponds to a basis function value of a sampling point, and each column corresponds to a basis function term of the polynomial). Simultaneously, the corresponding logarithmic luminance values ​​are used to form an observation vector p (with dimensions m×1, where m is the number of effective sampling points. The luminance values ​​of each effective sampling point within the organization region are processed by removing outliers and performing a logarithmic transformation, then arranged sequentially according to the sampling point order to obtain the observation vector p, where each element corresponds to the logarithmic luminance value of a sampling point, which is the target value for model fitting). Weights are constructed based on the deviation of the logarithmic luminance value of each sampling point from the overall median. The deviation of the logarithmic luminance value of each sampling point from the overall luminance median is calculated and normalized using the median absolute deviation. Then, weights are assigned to each sampling point by calculating the deviation; sampling points with smaller deviations have larger weights, and those with larger deviations have smaller weights. Finally, a diagonal weight matrix W is constructed based on these weights. Matrix operations are then performed sequentially, and the parameter vector β is obtained by multiplying the two weights.

[0080] β=(A T WA) 1 A T Wp

[0081] Where A is the design matrix composed of two-dimensional polynomial basis functions, p is the logarithmic domain brightness observation vector, W is the diagonal matrix composed of sample weights, and β is the polynomial parameter vector to be estimated.

[0082] The core of the weighted least squares approach in this invention is to assign different weights to different sampling points: the central sampling point has a large weight (strengthening its influence on the model), while the edge sampling point has a small weight (weakening its noise interference). This makes the estimated model coefficients more consistent with the true attenuation trend of vignetting, adapting to the vignetting modeling requirements of batch fluorescence microscopy images, and the solution results are unique and optimal.

[0083] In other embodiments of vignetting correction in fluorescence lifetime microscopy images, the vignetting model can be determined using methods other than polynomial fitting, such as Gaussian functions, cosine functions, etc. n Functions such as exponential functions can accurately characterize the low-frequency distribution characteristics of optical vignetting, which are characterized by high brightness at the center and smooth decay towards the edges. They can adapt to the differences in vignetting morphology under different imaging conditions and can be selected according to the actual distribution characteristics of the vignetting in the image.

[0084] S16, using the statistical brightness of the central region of the image as the central brightness reference, calculate the brightness correction coefficient for the vignetting model, and apply the correction coefficient to the tissue region of the image to complete the vignetting correction.

[0085] Specifically, the central region of the image is the area with the least vignetting, the most stable brightness, and the lowest noise. It serves as the natural brightness benchmark for the entire image. The selection range can be customized according to the image resolution (usually a circular / square area centered on the geometric center of the image, such as a 50×50 pixel area in the center of a 512×512 image). This ensures that there is no significant brightness attenuation within the region and that it represents the true target brightness of the image. The original brightness values ​​within the selected central region are statistically analyzed, and the mean or median is taken as the central brightness benchmark.

[0086] Vignette Model I model The pixel value (x, y) represents the brightness attenuation caused by vignetting at the corresponding location within the tissue area, with the center brightness reference I. ref For reference, the vignetting model is normalized pixel by pixel to directly obtain the brightness correction coefficient: Where (x, y) are the pixel coordinates within the tissue region, this formula is valid only when the pixel belongs to the tissue region. This embodiment of the invention sets a reasonable upper limit k for the correction coefficient. max The upper limit is adaptively set according to the degree of vignetting attenuation and noise level of the image (such as taking the 95th percentile value of all correction coefficients, or an empirical value of 2~5). The principle is to avoid excessively large coefficients that would amplify noise, while fully compensating for the brightness of the edge vignetting.

[0087] The brightness correction factor is a pixel-level brightness gain factor, and its value is positively correlated with the degree of vignetting attenuation, perfectly matching the spatial attenuation trend of vignetting.

[0088] 1. Image center region: Vignetting attenuation is extremely weak, I model (x,y)≈I ref Therefore, k corr (x,y)≈1, the correction coefficient is close to 1, which means that no significant brightness gain is needed in this area, only a small compensation is required;

[0089] 2. Image mid-area: Vignetting gradually increases, I model (x,y) ref Therefore 1 <k corr (x,y) <k max The correction coefficient increases with the degree of attenuation, thus allocating an appropriate brightness gain to the region.

[0090] 3. Image edge regions: Vignetting is strongest, I model (x,y) I ref Therefore, k​corr (x,y) is maximized, allocating the maximum brightness gain to low-brightness areas at the edges, and accurately compensating for the brightness loss caused by vignetting.

[0091] Step S2: For the image group after vignetting correction, the brightness index of each image is statistically analyzed within the organization area and a global brightness benchmark is determined. The brightness of the image group is unified through global brightness scaling and adaptive correction of single image brightness.

[0092] Specifically, since vignetting correction only solves the brightness attenuation problem of the center being bright and the edges being dark due to the optical system within a single image, even continuous acquisition areas of the same slice in the same group of fluorescence lifetime microscopic images to be stitched together will have different overall brightness levels between images due to slight fluctuations in the imaging equipment (such as changes in light source power and slight deviations in exposure time). Furthermore, after vignetting correction, some images still have local slight brightness unevenness in the tissue area.

[0093] This step involves determining a global baseline through statistical analysis of tissue region brightness, globally scaling to even out the overall brightness between images, and adaptive image correction to smooth out brightness differences within individual images. This ensures that the tissue regions of the entire set of images to be stitched achieve consistent global brightness, eliminating brightness banding between images. It also ensures uniform brightness within the tissue regions of each image, resolving residual brightness inconsistencies within individual images, ultimately outputting a set of globally uniform and individually smooth images. Specifically, this step includes the following processes:

[0094] S21, convert the vignetting-corrected image into a grayscale image, and obtain the tissue region through preset adaptive threshold segmentation and morphological opening and closing operations.

[0095] Specifically, the corrected image is converted into a grayscale image to directly represent the brightness intensity of the image. Otsu adaptive thresholding is used to obtain an initial binary mask, which initially achieves the separation of tissue and background. However, there is a problem that there are small holes in the tissue area caused by imaging noise and uneven staining. Therefore, morphological closing and opening operations are combined to fill the holes and remove small noise areas, resulting in a tissue area with regular shape, complete area and stable noise interference.

[0096] S22, only statistically analyze the grayscale brightness distribution within the organization area, extract the first preset percentile of the grayscale value as the main brightness index of the image, and the second preset percentile as the upper limit reference of brightness.

[0097] Specifically, grayscale brightness distribution is statistically analyzed only within the tissue region. To reduce the influence of extremely dark backgrounds and outlier highlights, the 60th percentile of the grayscale value is extracted as the main brightness index of the image, rather than the mean / median. This accurately represents the core brightness level of the vast majority of pixels in the tissue region, avoiding the influence of extremely dark pixels and avoiding being shifted by a small number of bright pixels. It is the optimal statistical measure to reflect the overall brightness of the image. The 95th percentile is extracted as a brightness upper limit reference to define the brightness threshold of the tissue region. Pixels exceeding this value are judged as abnormal bright noise (such as imaging noise or fluorescence overexposure points). During subsequent brightness correction, this upper limit can be used to avoid over-brightening, ensuring the rationality of brightness adjustment and the stability of image details.

[0098] S23, the truncated median method is used to calculate the main brightness index of all images to obtain the global brightness benchmark.

[0099] The purpose of using the truncated median method in this embodiment of the invention is to remove extreme outliers from the main brightness index set and calculate a global brightness benchmark that represents the true brightness level of the vast majority of samples in the entire image set. This ensures that subsequent global brightness scaling can fairly and reasonably equalize the brightness of all images, guaranteeing the accuracy and consistency of the brightness uniformity of the entire image set. For example, by arranging the main brightness indices of all images in ascending order to obtain an ordered index set, and based on the imaging characteristics of fluorescence lifetime microscopy images, a fixed truncation ratio is preset (usually 5% to 20% on both sides, which can be flexibly adjusted according to the number of images, such as 10%). The leftmost low brightness extreme value and the rightmost high brightness extreme value in the ordered set are removed to obtain the truncated effective brightness index set.

[0100] S24. Calculate the global scaling factor based on the ratio of the global brightness reference to the brightness index of each image subject, and perform global linear scaling on the corresponding image.

[0101] Specifically, global linear scaling essentially adjusts the overall brightness of the organized areas of each image to the global brightness benchmark level through pixel-level linear brightness gain / attenuation, following the principle of brightness ratio matching: using the ratio of the global brightness benchmark to the brightness index of the main body of a single image as the scaling factor, and performing linear multiplication on the brightness values ​​of all pixels in the image, so that brighter images are reduced in brightness and darker images are brightened in brightness, ultimately achieving global uniformity of brightness for the entire group of images.

[0102] For the i-th image in the entire image set, with the global brightness reference as I... global Molecules, the brightness index of the main subject of the image I i Using the denominator, the specific global scaling factor k is calculated. i The formula is:

[0103]

[0104] If k i =1: The brightness of the main subject in a single image is consistent with the global baseline, no brightness adjustment is required, and the brightness remains unchanged after scaling;

[0105] If k i >1: The overall image is too dark. The scaling factor is the brightness gain coefficient to brighten the image.

[0106] If k i <1: If a single image is too bright overall, the scaling factor is a brightness attenuation coefficient to reduce overall brightness. The value of the scaling factor is proportional to the brightness deviation between the single image and the global baseline, ensuring the accuracy and specificity of brightness adjustment.

[0107] The calculated global scaling factor k i The brightness values ​​I of all pixels in the grayscale image of the i-th image. raw Perform a pixel-wise linear multiplication operation on (x, y) to obtain the scaled pixel brightness value I. scale (x,y), the formula is: I scale (x,y)=I raw (x,y)×k i .

[0108] S25 performs large-scale Gaussian smoothing on the grayscale image of a single image to estimate the low-frequency brightness distribution, calculates the adaptive correction factor of the single image within the organized region and limits its fluctuation range, performs adaptive brightness correction on the single image, and achieves brightness uniformity of the image group.

[0109] In this embodiment of the invention, a large-scale Gaussian smoothing method is used to estimate the low-frequency brightness distribution of a single image grayscale image, filtering out high-frequency details and noise in the image, and retaining only the low-frequency trend that reflects the overall brightness change of the tissue region. This trend is the true local brightness distribution benchmark of the image, avoiding interference from high-frequency information in the correction and judgment.

[0110] Using low-frequency brightness distribution as a reference, a local correction factor is calculated pixel-by-pixel within the tissue region. Darker areas are assigned a factor greater than 1 for slight darkening, while brighter areas are assigned a factor less than 1 for slight brightening, precisely smoothing out local brightness differences. The fluctuation range of the correction factor is limited to a small range (e.g., ±5%) to achieve gentle adaptive correction, eliminating brightness unevenness while avoiding over-correction that could lead to loss of tissue details or noise amplification, thus ensuring the integrity of image texture features. By applying the correction factor only to the tissue region for pixel-by-pixel brightness adjustment, the brightness within the tissue region of each image becomes smooth and uniform, achieving dual brightness consistency for the entire image set and for individual images.

[0111] A schematic diagram comparing the fluorescence lifetime of intestinal tissue before and after vignetting correction and brightness unification is shown below. Figure 2 As shown, Figure 2 The original image in (a) exhibits vignetting characteristics, with a bright center and dark edges, and uneven brightness in the tissue area, with blurred boundaries between the background and tissue; after correction... Figure 2 Image (b) effectively improves the vignetting effect, achieves global uniformity and local balance of brightness in the tissue area, and makes the boundary between the background and the tissue clearer. The internal details of the tissue are fully preserved. This image directly verifies the effectiveness of the correction step of this method, proving that it can solve the problems of optical vignetting and uneven brightness from the root, and lay a high-quality image foundation for subsequent seamless stitching.

[0112] Step S3: After the brightness is unified, adjacent images are aligned by feature point matching based on spatial location. Combined with the stitch fusion method based on minimum color difference, the image blocks are horizontally or vertically stitched together to obtain an image strip.

[0113] This step, based on spatial feature point matching, accurately identifies common features in adjacent images and estimates translation parameters. Affine transformation is used to achieve precise geometric alignment of image blocks, resolving structural misalignment issues caused by acquisition offsets in microscopic image stitching. A minimum chromatic difference stitching method is employed, fusing overlapping areas along paths with minimal chromaticity differences. Combined with the previous brightness unification effect, this ensures a natural transition in chromaticity and brightness at the stitching boundaries. Image bands are formed by stitching image blocks in horizontal / vertical directions, achieving layered processing during the stitching process. Invalid / missing image positions are filled in, ensuring the integrity of the stitching structure and providing a reliable intermediate carrier for the final image band stitching and large field-of-view image output. Specifically, the steps include:

[0114] Step S31: Group the images according to the row and column index information in the image file name and sort them in order, and fill in the missing or invalid positions with a completely black image.

[0115] During fluorescence lifetime microscopy image acquisition, the sample area is acquired point by point according to a two-dimensional grid, and the acquired images are named in order according to the row and column index (e.g., imgX.Y represents the Xth row and Yth column). However, in actual acquisition, due to equipment failure, sample area occlusion, acquisition errors, etc., some positions of the image may be missing or the imaging may be invalid. At the same time, the images imported in batches may be out of order due to storage and transmission problems. If they are directly stitched together, it will result in the spatial position of the image being misaligned and the stitched structure being incomplete.

[0116] Based on the row and column index of the file name, the original two-dimensional spatial acquisition order of the image is restored, ensuring that the images are grouped and sorted according to the actual acquired row and column positions, avoiding splicing misalignment caused by disordered order; for missing / invalid image positions, they are filled with a completely black image to ensure the integrity of the two-dimensional grid structure of the image to be spliced, so that subsequent horizontal and vertical splicing can be performed in a continuous row and column order, avoiding spatial position gaps during the splicing process.

[0117] Step S32: In the preset width or height overlapping area of ​​adjacent images, extract feature points and descriptors and perform feature matching, calculate the displacement difference of the matching point pair and take the median to estimate the translation parameter, construct the translation transformation matrix to perform affine transformation on the image to achieve adjacent image alignment, if feature matching fails or the image is empty, keep the original position of the image unchanged.

[0118] Specifically, a preset width and height (adjustable according to acquisition parameters) overlap region of adjacent images is determined. Feature point and descriptor extraction is performed only within this overlap region. The ORB feature algorithm is used to quickly detect scale- and rotation-invariant feature points within the overlap region, adapting to the characteristics of fluorescence microscopy images. It can effectively extract stable features such as tissue edges and textures. For the detected feature points, a binary descriptor is generated using an improved version of the BRIEF descriptor. It features fast calculation speed and high matching efficiency, and can accurately characterize the local features of each feature point, providing a basis for subsequent feature matching.

[0119] Even with feature matching in fluorescence microscopy images, a small number of mismatched points may still exist. The displacement difference of these points deviates from the normal range. Using the median has strong robustness to outliers (displacement differences of mismatched points), accurately reflecting the true displacement trend of the vast majority of valid matching point pairs, and ensuring the accuracy of translation parameter estimation. By performing an affine transformation on the image through the translation transformation matrix, matrix operations are performed on all pixel coordinates of the subsequent image with the transformation matrix to obtain the corrected pixel coordinates. The pixel values ​​are then completed using an interpolation algorithm (such as bilinear interpolation) to complete the geometric correction of the image, ensuring that the homologous tissue features of adjacent images accurately overlap, achieving misalignment-free alignment, ensuring that the tissue structure of the image is not distorted, and significantly reducing computational complexity and improving alignment efficiency.

[0120] If the number of valid matching point pairs in the feature point matching results of adjacent images is less than a preset threshold (determined as a matching failure), or if the image to be aligned is a completely black empty image, no affine transformation is performed on the image, preserving its original spatial position. By setting a fallback strategy for matching failures / empty images, the overall stitching is prevented from being interrupted due to anomalies in a single set of images, thus improving the engineering practicality of the algorithm.

[0121] Step S33: Calculate the chromaticity difference of corresponding pixels in the overlapping area based on the Euclidean distance, use the Euclidean distance as the cost function, and use dynamic programming to search for the minimum cost path from top to bottom in the overlapping area as the optimal stitching seam, and apply constraints to the edge area of ​​the seam.

[0122] Specifically, the overlapping areas are converted from BGR to Lab color space, and the chromaticity difference between adjacent overlapping areas is calculated using only the a and b chromaticity channels. Using Euclidean distance as the cost function transforms the stitching path selection problem into a path search problem within the cost matrix. The total cost of the stitching seam is the sum of the cost values ​​of all pixels along the path; the smaller the total cost, the more natural the visual transition after stitching. Based on the cost matrix, a dynamic programming algorithm is used to search for the globally minimum cost path along a top-down direction (row direction). This path is the optimal stitching seam, ensuring a natural chromaticity transition throughout the stitched area without any obvious local boundaries.

[0123] During the dynamic programming search process, edge constraints are applied to the position of the optimal stitching seam to prevent the seam from getting too close to the left and right edges of the overlapping area. The specific constraint logic is as follows: set an edge protection zone for the overlapping area (such as reserving 5% to 10% of the width on the left and right as the edge area), and apply a penalty cost to the pixels in the edge area in the cost matrix (such as increasing the cost by a fixed value). If the path of the dynamic programming search attempts to enter the edge protection zone, its accumulated cost will increase significantly, and the algorithm will automatically avoid the area, so that the optimal stitching seam always falls in the middle effective area of ​​the overlapping area.

[0124] The purpose of constraints is to ensure that edge pixels in overlapping areas are prone to feature mismatch due to sampling deviations and minor geometric alignment errors. If the seam is close to the edge, it can easily lead to structural misalignment and color abrupt changes at the splicing boundary. Restricting the seam to the middle effective area can maximize the stability and naturalness of the splicing and avoid splicing traces caused by edge factors.

[0125] Step S34: Construct a fusion mask according to the stitch fusion method with minimum color difference, and perform pixel fusion on the overlapping area.

[0126] Specifically, this embodiment of the invention uses horizontal fusion as an example (in other embodiments, vertical image stitching can be performed first, followed by horizontal stitching). It is performed on the overlapping areas of single image blocks. The fused image blocks are stitched into continuous image bands in row and column order. The fused image bands have no stitching marks and high geometric consistency, which not only ensures the visual continuity of the image bands themselves, but also provides a high-quality intermediate carrier for the vertical stitching between subsequent image bands, effectively reducing the cumulative error of large field-of-view stitching.

[0127] Step S4: Extract the overlapping region of the image band and generate an effective content mask. After low-frequency suppression, estimate the relative displacement of adjacent image bands using the phase correlation method. Reuse the stitch fusion method with minimum color difference to complete the image band stitching and output a large field-of-view fluorescence lifetime microscopic stitched image.

[0128] Specifically, based on the image bands obtained from the previous horizontal stitching, vertical stitching is performed. First, overlapping areas of the image bands are extracted and an effective content mask is generated. Invalid areas such as black backgrounds are removed, retaining only effective tissue pixels. Then, low-frequency suppression is applied to highlight structural edge information, improving the accuracy of displacement estimation using the phase correlation method and achieving geometric alignment between image bands. Simultaneously, the minimum color difference suture fusion method is reused to seamlessly merge overlapping areas of the image bands. This process is repeated to complete the stitching of all image bands and crop black edges, outputting a large field-of-view image. This layered stitching method significantly reduces accumulated errors, ensuring geometric consistency, visual continuity, and integrity of tissue features in the stitched images, meeting the high-precision requirements for qualitative and quantitative analysis of lesions in medical research. Specifically, the process includes the following steps:

[0129] Step S41: Extract the overlapping region of the edges of adjacent images to construct an overlapping sub-image, and perform brightness threshold segmentation and morphological operations on the overlapping sub-image to generate an effective content mask.

[0130] Specifically, based on the preset parameters of image band acquisition and stitching, the overlap width of adjacent image band edges is determined, and the overlapping area of ​​the two image band edges is accurately extracted. This area is then separated from the original image band to construct an independent overlapping sub-image. Brightness threshold segmentation (a fixed threshold or adaptive threshold adapted to the microscopic image features) is performed on the overlapping sub-image. The significant difference between the high brightness of effective organization pixels and the zero brightness of invalid background pixels (completely black) is utilized to achieve preliminary binary separation of the two. Morphological opening and closing operations are sequentially performed on the initial binary mask obtained by brightness threshold segmentation to remove noise and optimize the mask shape, resulting in an effective content mask with regular edges, complete regions, and no noise interference.

[0131] Step S42: Perform low-frequency suppression processing on the effective content mask to highlight structural and edge information.

[0132] Specifically, by performing low-frequency suppression on the overlapping sub-images after effective content masking, low-frequency uniform brightness areas and background noise in the image are filtered out, highlighting high-frequency effective features such as the structural contours and edge textures of the tissue. This allows the phase correlation method to accurately capture these highly recognizable feature information when estimating the relative displacement of image bands, avoiding interference from low-frequency indifferent areas with displacement estimation results, and significantly improving the accuracy, robustness, and sensitivity of the phase correlation method registration. At the same time, low-frequency suppression is only performed on effective tissue areas, which does not destroy the integrity of tissue features and ensures the alignment accuracy when stitching image bands.

[0133] Step S43: The phase correlation method is used to estimate the relative displacement of adjacent image bands in the horizontal and vertical directions for the overlapping sub-images after low-frequency suppression.

[0134] Specifically, since image bands are composed of stitched-together single image blocks, minute translational misalignments can easily accumulate during acquisition and block stitching, leading to horizontal / vertical geometric deviations between adjacent image bands. The phase correlation method, which estimates displacement based on frequency domain phase information, is more adaptable to areas with less image texture and uniform brightness, and exhibits excellent resistance to noise and local brightness variations. Based on pre-processed, clean, overlapping sub-images, the phase correlation method accurately calculates the horizontal (x-axis) and vertical (y-axis) pure translational displacements of adjacent image bands in the frequency domain. These parameters are pixel-level / sub-pixel-level precise values, directly reflecting the spatial misalignment distance and direction between the two image bands.

[0135] Step S44: Place adjacent image bands in the same coordinate system on the canvas according to the relative displacement. The non-overlapping areas are directly filled with the corresponding pixel values, and the overlapping areas are fused using the stitching fusion method with the minimum color difference.

[0136] Specifically, based on the total resolution and row and column arrangement of all image strips to be stitched, a global coordinate system blank canvas is constructed that can accommodate all image strips. The resolution of the canvas is determined by the size, number, and overlap ratio of the image strips, ensuring that the effective pixels of all image strips can be completely mapped.

[0137] One image band is used as the reference image band and fixed in its initial position in the global coordinate system. Then, based on the horizontal / vertical relative displacement parameters (Δx, Δy) estimated by the phase correlation method, the other image band to be stitched is subjected to spatial coordinate mapping, precisely placing it in the corresponding position in the global coordinate system. This achieves geometrically accurate alignment of the two image bands in the same coordinate system, eliminating all translational misalignments. After the adjacent image bands are aligned in the global coordinate system, overlapping areas (the core area for feature matching and fusion) and non-overlapping areas (the effective areas unique to each image band) are formed. For the non-overlapping areas, the original pixel values ​​of the image bands are directly filled into the corresponding coordinate positions on the global canvas. For the overlapping areas of adjacent image bands, the stitching fusion method with the minimum color difference from the image block stitching stage is completely reused without any algorithmic adjustments, ensuring the consistency and stability of the fusion effect.

[0138] Step S45: Perform the fusion operation on all images one by one in a loop. After the stitching is completed, crop the black borders of the result and output a large field-of-view fluorescence lifetime microscopic stitched image.

[0139] Specifically, registration and fusion operations are performed on all image bands sequentially according to a preset row and column order, gradually integrating all scattered image bands into a single global coordinate system canvas to form a complete large-field-of-view image draft. Black borders are then cropped on the stitched large-field-of-view image draft, precisely removing all completely black invalid areas and retaining only areas containing valid biological tissue information. The final large-field-of-view fluorescence lifetime micrograph is output, exhibiting geometric consistency, visual seamlessness, and the removal of invalid areas. A schematic diagram of the stitched result is shown below. Figure 3 As shown, this achieves the goal of small field-of-view acquisition and large field-of-view stitching, meeting the practical needs of qualitative and quantitative analysis of diseased tissues in medicine.

[0140] This embodiment also provides a fluorescence lifetime microscopy image large field-of-view stitching system, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as described herein. As used below, the term "module" can be a combination of software and / or hardware that performs a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0141] This embodiment provides a large field-of-view stitching system for fluorescence lifetime microscopic images, such as... Figure 4 As shown, it includes:

[0142] The vignetting correction module 41 is used to construct a tissue region mask for the images in the fluorescence lifetime microscopic image group to be spliced, construct a vignetting model in the tissue region based on the mask, and calculate the correction coefficient in the tissue region by combining the image center brightness reference to complete the vignetting correction in the tissue region.

[0143] The brightness unification module 42 is used to statistically analyze the brightness index of each image in the organized area and determine the global brightness benchmark for the image group after vignetting correction. The brightness unification of the image group is achieved through global brightness scaling and local brightness correction of single images.

[0144] The image strip generation module 43 is used to align adjacent images by matching feature points according to their spatial positions after the brightness is unified, and to complete the horizontal or vertical stitching of image blocks by combining the stitch fusion method based on minimum color difference to obtain the image strip.

[0145] The image stitching output module 44 is used to extract the overlapping area of ​​the image band and generate an effective content mask. After low-frequency suppression, the relative displacement of adjacent image bands is estimated by the phase correlation method. The stitching of the image band is completed by reusing the stitch fusion method with minimum color difference, and a large field-of-view fluorescence lifetime microscopic stitched image is output.

[0146] In some alternative implementations, the vignetting correction module 41 includes:

[0147] The first tissue region acquisition unit is used to convert the image to the Lab color space, extract the color information of the a channel and b channel of the Lab color space for clustering, and obtain the first tissue region by identifying the background category;

[0148] The second tissue region acquisition unit is used to extract the luminance information of the L luminance channel in the Lab color space and obtain the second tissue region based on adaptive threshold segmentation.

[0149] The tissue region mask generation unit is used to take the union of the first tissue region and the second tissue region as the tissue region mask.

[0150] In some alternative implementations, the vignetting correction module 41 includes:

[0151] The luminance value sampling unit is used to sample the luminance information in the tissue area and remove outliers from the sampled values ​​to obtain the filtered luminance values.

[0152] The vignetting model construction unit is used to perform a logarithmic transformation on the filtered brightness values ​​and construct a two-dimensional polynomial model. The coefficients of the two-dimensional polynomial model are estimated by the weighted least squares method to obtain a vignetting model that characterizes the trend of image brightness decay from the center to the edge.

[0153] The vignetting correction unit is used to normalize the vignetting model based on the statistical brightness of the central region of the image, calculate the brightness correction coefficient, and apply the correction coefficient only to the tissue area of ​​the image to complete the vignetting correction.

[0154] In some alternative implementations, the brightness unification module 42 includes:

[0155] The grayscale conversion unit is used to convert the vignetting-corrected image into a grayscale image and obtain the tissue region through preset adaptive threshold segmentation and morphological opening and closing operations.

[0156] The brightness distribution statistics unit is used to statistically analyze the gray-scale brightness distribution only within the organization area, extract the first preset percentile of the gray-scale value as the main brightness index of the image, and the second preset percentile as a reference for the upper limit of brightness.

[0157] The global brightness benchmark calculation unit is used to calculate the main brightness index of all images using the truncated median method to obtain the global brightness benchmark;

[0158] A global linear scaling unit is used to calculate a global scaling factor and perform global linear scaling on the corresponding image based on the ratio of the global brightness reference to the brightness index of each image subject.

[0159] The brightness unification unit is used to perform large-scale Gaussian smoothing on the grayscale image of a single image to estimate the low-frequency brightness distribution. It calculates the adaptive correction factor of a single image within the organized region and limits its fluctuation range, thereby achieving brightness unification of the image group through adaptive brightness correction of a single image.

[0160] In some alternative implementations, the image band generation module 43 includes:

[0161] The image sorting unit is used to group and sort images according to the row and column index information in the image file name, and fill in missing or invalid positions with a completely black image.

[0162] The adjacent image alignment unit is used to extract feature points and descriptors and perform feature matching within the preset width and height overlap area of ​​adjacent images, calculate the displacement difference of the matching point pairs and take the median to estimate the translation parameters, construct the translation transformation matrix to perform affine transformation on the image to achieve adjacent image alignment, and if feature matching fails or the image is empty, the original position of the image remains unchanged.

[0163] The stitching unit calculates the chromaticity difference of corresponding pixels in the overlapping area based on the Euclidean distance, uses the Euclidean distance as the cost function, and uses a dynamic programming method to search for the minimum cost path from top to bottom in the overlapping area as the optimal stitching seam, and applies constraints to the edge area of ​​the seam.

[0164] The overlapping region pixel fusion unit is used to construct a fusion mask according to the stitch fusion method with minimum color difference and to perform pixel fusion on the overlapping region.

[0165] In some alternative implementations, the stitched image output module 44 includes:

[0166] The overlapping sub-image extraction unit is used to extract the overlapping region of the edges of adjacent images to construct an overlapping sub-image, and to perform brightness threshold segmentation and morphological operations on the overlapping sub-image to generate an effective content mask.

[0167] A low-frequency suppression unit is used to perform low-frequency suppression processing on the effective content mask to highlight structural and edge information;

[0168] The relative displacement calculation unit is used to estimate the relative displacement of adjacent image bands in the horizontal and vertical directions by using the phase correlation method on the overlapping sub-images after low-frequency suppression.

[0169] In one alternative implementation, the stitched image output module 44 includes:

[0170] The image stitching output unit is used to place adjacent image strips on the same coordinate system canvas according to the relative displacement. Non-overlapping areas are directly filled with corresponding pixel values, and overlapping areas are fused using the minimum color difference stitching fusion method. The fusion operation is performed on all image strips one by one in a loop. After the stitching is completed, the black border is cropped on the result, and a large field-of-view fluorescence lifetime microscopic stitched image is output.

[0171] The fluorescence lifetime microscopy image large field-of-view stitching system provided in this embodiment of the invention can execute the fluorescence lifetime microscopy image large field-of-view stitching method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0172] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. See below for details. Figure 5 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0173] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0174] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the nuclear security network design data operation and maintenance delivery and automatic configuration method of the embodiments of the present invention.

[0175] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0176] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the nuclear security network design data operation and maintenance delivery and automatic configuration method shown in the above embodiments is implemented.

[0177] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0178] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for large field-of-view stitching of fluorescence lifetime micrographs, characterized in that, include: A tissue region mask is constructed for the images within the fluorescence lifetime microscopic image group to be stitched together. A vignetting model is constructed within the tissue region based on the mask. The correction coefficient is calculated in combination with the image center brightness reference to complete the vignetting correction in the tissue region. For the image group after vignetting correction, the brightness index of each image is statistically analyzed in the organization area and a global brightness benchmark is determined. The brightness of the image group is unified through global brightness scaling and adaptive correction of single image brightness. After unifying the brightness of the image, adjacent images are aligned by feature point matching based on spatial location. Combined with the stitch fusion method based on minimum color difference, the image blocks are horizontally or vertically stitched together to obtain the image strip. The overlapping regions of the image bands are extracted and an effective content mask is generated. After low-frequency suppression, the relative displacement of adjacent image bands is estimated by the phase correlation method. The stitch fusion method with minimum color difference is reused to complete the image band stitching and output a large field-of-view fluorescence lifetime microscopic stitched image. The step of extracting overlapping regions from the image bands and generating an effective content mask, followed by low-frequency suppression and estimation of the relative displacement of adjacent image bands using a phase correlation method, includes: The overlapping region of adjacent image band edges is extracted to construct an overlapping sub-image. The overlapping sub-image is then subjected to brightness threshold segmentation and morphological operations to generate an effective content mask. The effective content mask is subjected to low-frequency suppression processing to highlight structural and edge information; The phase correlation method is used to estimate the relative displacement of adjacent image bands in the horizontal and vertical directions for overlapping sub-images after low-frequency suppression. The method of reusing the minimum chromatic difference suture fusion is used to complete image band stitching and output a large field-of-view fluorescence lifetime microscopic stitched image, including: Based on the relative displacement, adjacent image bands are placed on the canvas in the same coordinate system. Non-overlapping areas are directly filled with the corresponding pixel values, while overlapping areas are fused using the minimum color difference stitching fusion method. The fusion operation is performed on all image bands one by one in a loop. After the stitching is completed, the black border is cropped, and a large field-of-view fluorescence lifetime microscopic stitched image is output.

2. The method according to claim 1, characterized in that, The construction of the tissue region mask includes: The image is converted to the Lab color space, and the color information of the a and b channels of the Lab color space is extracted and clustered. The first tissue region is obtained by identifying the background category. The luminance information of the L luminance channel in the Lab color space is extracted, and the second tissue region is obtained based on adaptive threshold segmentation. The union of the first and second organizational regions is taken as the organizational region mask.

3. The method according to claim 1 or 2, characterized in that, Based on the mask, a vignetting model is constructed within the tissue region. Correction coefficients are calculated using the image center brightness reference to complete vignetting correction within the tissue region, including: The brightness information within the tissue area is sampled, and outliers are removed from the sampled values ​​to obtain the filtered brightness values. Logarithmic transformation is performed on the filtered brightness values, and a two-dimensional polynomial model is constructed. The coefficients of the two-dimensional polynomial model are estimated by the weighted least squares method to obtain a vignetting model that characterizes the attenuation trend of image brightness from the center to the edge. The vignetting model is normalized based on the statistical brightness of the central region of the image, a brightness correction coefficient is calculated, and the correction coefficient is applied only to the tissue region of the image to complete the vignetting correction.

4. The method according to claim 1, characterized in that, The image group after vignetting correction involves statistically analyzing the brightness indices of each image within the tissue region and determining a global brightness benchmark. Brightness uniformity of the image group is achieved through global brightness scaling and adaptive brightness correction of individual images, including: The image after vignetting correction is converted to grayscale, and the tissue region is obtained through preset adaptive threshold segmentation and morphological opening and closing operations. The grayscale brightness distribution is statistically analyzed only within the organized area. The first preset percentile of the grayscale value is extracted as the main brightness index of the image, and the second preset percentile is used as a reference for the upper limit of brightness. The global brightness benchmark is obtained by calculating the subject brightness index of all images using the truncated median method; Based on the ratio of the global brightness benchmark to the brightness index of each image subject, calculate the global scaling factor and perform global linear scaling on the corresponding image; Large-scale Gaussian smoothing is performed on the grayscale image of a single image to estimate the low-frequency brightness distribution. The adaptive correction factor of the single image is calculated within the organized region and its fluctuation range is limited to perform adaptive brightness correction of the single image, thereby achieving brightness uniformity of the image group.

5. The method according to claim 1, characterized in that, The image, after brightness unification, is aligned with adjacent images by feature point matching based on spatial location. Combined with a stitching fusion method based on minimum color difference, image blocks are horizontally or vertically stitched together to obtain an image strip, including: The images are grouped and sorted in order according to the row and column index information in the image file name, and missing or invalid positions are filled with a completely black image. Within the pre-defined width and height overlap area of ​​adjacent images, feature points and descriptors are extracted and feature matching is performed. The displacement difference of the matching point pairs is calculated and the median is used to estimate the translation parameters. A translation transformation matrix is ​​constructed to perform affine transformation on the images to achieve alignment of adjacent images. If feature matching fails or the image is empty, the original position of the image is kept unchanged. The chromaticity difference of corresponding pixels in the overlapping area is calculated based on the Euclidean distance. The Euclidean distance is used as the cost function, and a dynamic programming method is used to search for the minimum cost path from top to bottom in the overlapping area as the optimal stitching seam. Constraints are applied to the edge area of ​​the seam. A fusion mask is constructed based on the stitch fusion method with minimum color difference, and pixel fusion is performed on the overlapping areas.

6. A large field-of-view stitching system for fluorescence lifetime microscopic images, characterized in that, include: The vignetting correction module is used to construct a tissue region mask for the images within the fluorescence lifetime microscopic image group to be stitched together, construct a vignetting model in the tissue region based on the mask, and calculate correction coefficients in the tissue region in combination with the image center brightness reference to complete vignetting correction in the tissue region. The brightness unification module is used to statistically analyze the brightness index of each image within the organized area and determine the global brightness benchmark for the image group after vignetting correction. The brightness unification of the image group is achieved through global brightness scaling and adaptive correction of single image brightness. The image strip generation module is used to align adjacent images by matching feature points based on their spatial location after the brightness is unified. Combined with the stitch fusion method based on minimum color difference, it completes the horizontal or vertical stitching of image blocks to obtain image strips. The image stitching output module is used to extract the overlapping area of ​​the image band and generate an effective content mask. After low-frequency suppression, the relative displacement of adjacent image bands is estimated by the phase correlation method. The stitching of the image band is completed by reusing the stitch fusion method with the minimum color difference and outputting a large field of view fluorescence lifetime microscopic stitched image. The stitched image output module includes: The overlapping sub-image extraction unit is used to extract the overlapping region of the edges of adjacent images to construct an overlapping sub-image, and to perform brightness threshold segmentation and morphological operations on the overlapping sub-image to generate an effective content mask. A low-frequency suppression unit is used to perform low-frequency suppression processing on the effective content mask to highlight structural and edge information; The relative displacement calculation unit is used to estimate the relative displacement of adjacent image bands in the horizontal and vertical directions by using the phase correlation method on the overlapping sub-images after low-frequency suppression. The stitched image output module includes: The image stitching output unit is used to place adjacent image strips on the same coordinate system canvas according to the relative displacement. Non-overlapping areas are directly filled with corresponding pixel values, and overlapping areas are fused using the minimum color difference stitching fusion method. The fusion operation is performed on all image strips one by one in a loop. After the stitching is completed, the black border is cropped on the result, and a large field-of-view fluorescence lifetime microscopic stitched image is output.

7. An electronic device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the large field-of-view stitching method for fluorescence lifetime microscopy images as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the fluorescence lifetime microscopy image large field-of-view stitching method according to any one of claims 1 to 5.

Images