A multi-scale color correction method for Fourier layered imaging

Abstract

This invention discloses a multi-scale color correction method for Fourier layered imaging, relating to the fields of microscopic imaging and image processing technology. The method acquires low-resolution original image sequences of red, green, and blue channels corresponding to multi-angle illumination, constructs a three-level resolution pyramid (fine, medium, and coarse), and adaptively divides the image into blocks based on local gray-level variance. It extracts the local average brightness and local contrast of image sub-blocks, filters significant background blocks using K-means clustering, calculates confidence weights, and weightedly fuses them to obtain illumination vectors. Based on these illumination vectors, it performs channel-by-channel white balance correction on the original image sequence. Finally, it spatially aligns the corrected low-resolution color image with a high-resolution grayscale image, performs frequency domain deconvolution, and feature fusion to generate a high-resolution color microscopic image. This method requires no external reference, corrects global color shift, reduces the impact of spatially non-uniform illumination on illumination estimation, and preserves high-frequency details, making it suitable for digital pathology, cell analysis, and biological slide observation.

Description

Technical Field

[0001] This invention relates to the field of Fourier layered microscopy and microscopic image processing technology, and particularly to a multi-scale color correction method for Fourier layered imaging. Background Technology

[0002] Fourier transform imaging technology, combined with multi-angle LED array illumination and phase retrieval algorithms, can overcome the limitations of numerical aperture and field of view in traditional optical systems, achieving large field of view and high-resolution microscopic imaging at low cost. This has significant application value in fields such as digital pathology diagnosis, cell analysis, and biological slide observation. In clinical pathology testing scenarios, stable color reproduction is a core prerequisite for sample typing and lesion identification; mainstream imaging systems typically use tri-color LED arrays as the illumination source.

[0003] Due to hardware limitations, the luminous efficiency, spectral distribution, and spatial intensity distribution of the red, green, and blue channels naturally differ, leading to global color casts and spatially non-uniform local color differences in acquired and reconstructed images. This is typically manifested as blue-purple distortion in the background, which can obscure the fine structures of biological samples and affect pathological interpretation. Current solutions to this color cast problem mainly include hardware optimization, physical reference correction using white paper or standard color charts, and post-processing with traditional algorithms. Hardware optimization increases acquisition time and affects the high-throughput characteristics of the imaging system; physical reference correction requires mechanical switching mechanisms or manual operation, making it difficult to adapt to fully automated imaging scenarios; traditional global color correction algorithms, such as the white point algorithm and grayscale world algorithm, rely on the global statistical characteristics of the entire image and do not consider the spatially non-uniform illumination characteristics caused by LED lighting. They are mainly used to compensate for global color casts and have limitations in handling local color differences under spatially non-uniform illumination conditions, potentially causing overexposure in highlight areas, over-compensation of color, and loss of detail in certain situations.

[0004] For example, Chinese patent application CN111062889A (published on April 24, 2020) discloses a light intensity correction method for Fourier layered microscopy imaging technology. It corrects the brightness of low-resolution images by setting the range of image intensity multiple changes and iteratively finding the brightness multiple value, and then reconstructs the corrected images at high resolution.

[0005] Furthermore, existing correction methods do not fully consider the impact of color correction intervention timing on the phase recovery iteration process. An unreasonable correction timing can lead to continuous color shift interfering with iterative convergence and reducing image quality. In addition, a single fixed block local correction method may produce block effect and affect the visual consistency of the image.

[0006] Therefore, how to correct global color shift, reduce the impact of spatial non-uniform illumination on illumination estimation, and adapt to fully automated high-throughput imaging without the need for an external reference body has become an urgent technical problem to be solved. Summary of the Invention

[0007] The main objective of this invention is to provide a multi-scale color correction method for Fourier layered imaging, which aims to correct global color shift, reduce the impact of spatial non-uniform illumination on illumination estimation, and adapt to fully automated high-throughput imaging without the need for an external reference body.

[0008] To achieve the above objectives, this invention proposes a multi-scale color correction method for Fourier layered imaging, comprising the following steps: S1. Obtain the original low-resolution red, green, and blue three-channel images corresponding to multi-angle illumination acquired by the Fourier stacked microscopy system, construct a three-level resolution pyramid of fine, medium, and coarse, and adaptively divide each level of image into blocks based on local gray-level variance to obtain multiple non-overlapping image sub-blocks. S2. Extract the local average brightness and local contrast features of each image sub-block, use the K-means unsupervised clustering algorithm to divide all image sub-blocks into tissue sample regions and background regions, and select the image sub-blocks in the background regions as a set of significant background blocks. S3. Calculate the confidence weight of each of the salient background blocks, and combine the confidence weight to perform weighted fusion of the color information of all the salient background blocks to obtain an initial illumination vector characterizing the light source characteristics of the imaging system, and perform green channel normalization constraint on the initial illumination vector to obtain the illumination vector. S4. Perform channel-by-channel white balance correction on the low-resolution RGB original image sequence based on the illumination vector to obtain a low-resolution color image after color shift correction. S5. Spatial alignment, frequency domain regularization deconvolution processing, and color-structure feature fusion are performed on the low-resolution color image after color deviation correction and the high-resolution grayscale image obtained by Fourier stacking iteration reconstruction to generate a high-resolution color microscopic image.

[0009] Preferably, in step S1, the adaptive segmentation of each level of image based on local gray-level variance includes: determining a preset threshold T according to the set of local gray-level variances of each level of image, wherein the preset threshold T satisfies the following formula:

[0010] in, For the preset threshold, This represents the mean of the local grayscale variance of the current image level. This represents the standard deviation of the local grayscale variance of the current image level. The preset coefficient is used; regions with local gray-level variance greater than the preset threshold T are divided into first-size sub-blocks, and regions with local gray-level variance less than or equal to the preset threshold T are divided into second-size sub-blocks; wherein, the size of the first-size sub-block is... pixels, the size of the second size sub-block is Pixel.

[0011] Preferably, in step S2, the number of cluster centers of the K-means unsupervised clustering algorithm is... The local contrast features are characterized by pixel standard deviation; the K-means unsupervised clustering algorithm automatically divides all image sub-blocks into a first cluster and a second cluster. The first cluster is a tissue sample cluster with lower average brightness and higher variance among the two clusters, corresponding to the tissue sample region; the second cluster is a background cluster with higher average brightness and lower variance among the two clusters, corresponding to the background region.

[0012] Preferably, in step S3, calculating the confidence weight of each of the salient background blocks includes: calculating the first... The feature space Euclidean distance from each of the salient background blocks to the corresponding background cluster centers of the background regions. And calculate the confidence weight of the significant background block according to the following formula:

[0013] in, For the first Confidence weights for each significant background block and All are significant background block numbers and their values ​​range from 1 to 2. , This represents the total number of salient background blocks in the set of salient background blocks. and The first The and the first The Euclidean distance in the feature space corresponding to each salient background block It is a positive number.

[0014] Preferably, in step S3, the weighted fusion calculates the initial illumination vector using the following formula:

[0015] In the formula, Let be the initial illumination vector. The total number of salient background blocks in the set of salient background blocks. Number the prominent background blocks and , For the first Confidence weights for each significant background block For the first The average color vector of a significant background block; applying a green channel normalization constraint to the initial illumination vector means: if the initial illumination vector for The illumination vector ultimately used for white balance correction Satisfy the following formula:

[0016] in, , and These are the red channel component, green channel component, and blue channel component of the initial illumination vector, respectively.

[0017] Preferably, in step S4, the channel-by-channel white balance correction is performed before the phase retrieval iterative reconstruction is performed using the Fourier stacking algorithm, and the white balance correction is performed by performing pixel-by-pixel and color-channel-by-color channel division on the low-resolution RGB original image sequence using the following formula:

[0018] In the formula, These are the pixel values ​​of the image after color cast correction. The original image pixel values ​​in the low-resolution RGB original image sequence. This indicates a channel-by-channel division operation. Let be the illumination vector.

[0019] Preferably, in step S5, the spatial alignment, frequency domain regularization deconvolution processing, and color-structure feature fusion include the following sub-steps: S501. The low-resolution color image after color shift correction is enlarged using a bicubic interpolation algorithm to make its image size consistent with that of the high-resolution grayscale image, thus completing spatial alignment. S502. The red, green, and blue channels of the magnified color image are decoupled through the following linear transformation to extract the low-frequency color features of the image. Combined with the point spread function of the Fourier layered microscopy imaging system, regularized deconvolution processing is performed in the frequency domain to eliminate edge blurring caused by interpolation magnification:

[0020]

[0021] in, For the luminance component, and Low-frequency color characteristics, , and These are the pixel values ​​of the red, green, and blue channels of the magnified color image. It is a linear transformation matrix. The first one after deconvolution processing The low-frequency color feature is located in the frequency domain. The value at that location, For the magnified version of the first The low-frequency color feature is located in the frequency domain. The value at that location, Here is the frequency domain representation of the point spread function. for conjugate, This is the regularization parameter.

[0022] Preferably, in step S5, the spatial alignment, frequency domain regularization deconvolution processing, and color-structure feature fusion further include the following sub-steps: S503. Using the high-resolution grayscale image as a high-frequency structural constraint prior, construct the following joint optimization model: fuse the deconvolutioned low-frequency color features with the structural features of the high-resolution grayscale image, and restore them to red, green, and blue three-channel pixel values ​​through inverse mapping, outputting the final high-resolution color micrograph:

[0023]

[0024] in, The fusion feature to be solved, The optimal solution for the joint optimization model. The color components extracted from the fusion features. The low-frequency color features after deconvolution processing The brightness component is extracted from the fusion features. For the high-resolution grayscale image, For gradient operators, and These are the weighting coefficients. The linear transformation matrix The inverse matrix. The luminance and chrominance components in the inverse mapping are respectively derived from the optimal solution. pass and Extracted.

[0025] Preferably, the high-resolution grayscale image is obtained by iterative reconstruction using a wavelet-optimized alternating projection algorithm. The wavelet-optimized alternating projection algorithm uses the Daubechies4 wavelet basis for two-level wavelet decomposition, performs soft thresholding on the high-frequency wavelet coefficients after each alternating projection update, and sets the number of iterations to 10.

[0026] Preferably, the Fourier layered microscopy imaging system includes an illumination module, which employs... The WS2812B type red, green and blue light-emitting diode array has a center-to-center spacing of 3mm between each light-emitting diode, a vertical distance of 45mm between the red, green and blue light-emitting diode array and the sample plane, and peak wavelengths of the red, green and blue light-emitting diodes are 620nm, 525nm and 450nm respectively.

[0027] The above technical solution has the following advantages: This invention employs a three-level resolution pyramid (fine, medium, and coarse) and an adaptive block-segmentation strategy, combined with K-Means unsupervised clustering to extract salient background blocks. It then estimates the illumination vector by integrating the color and brightness statistics of different background blocks to correct global color cast and reduce the impact of spatially non-uniform illumination on illumination estimation. Simultaneously, confidence-weighted illumination estimation reduces the influence of a single block on illumination estimation while fusing information from multiple salient background blocks. Green channel normalization is applied to the initial illumination vector to improve the algorithm's stability. Furthermore, this method performs color correction at the low-resolution RGB original image sequence stage, reducing the interference of color cast on the phase retrieval iteration process and improving the quality of super-resolution reconstruction. The correction process requires no external reference objects such as white paper or color charts and can run fully automatically. Through feature fusion between low-resolution color images and high-resolution grayscale images, it preserves high-frequency details such as sample edges and cell textures while correcting global color cast. Attached Figure Description

[0028] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the architecture of the multi-scale color correction method for Fourier layered imaging provided in this embodiment of the invention.

[0029] Figure 2 This is a comparison chart of simulation results for a color correction algorithm based on the Macbeth color chart. Figure 2 (a) in the image is the baseline truth map of the Macbeth color chart. Figure 2 Image (b) is the distorted image obtained by applying a global RGB gain bias and a spatially correlated light intensity bias to the reference ground truth image. Figure 2 (c) in the figure represents the white point algorithm used for processing. Figure 2 The result obtained by correcting the distorted image shown in (b) is shown in the figure. Figure 2 (d) in the figure represents the result of using the gray world algorithm. Figure 2 The result obtained by correcting the distorted image shown in (b) is shown in the figure. Figure 2(e) in the figure represents the method of the present invention for... Figure 2 The result obtained by correcting the distorted image shown in (b) is shown in the figure.

[0030] Figure 3 Comparison of imaging results from different color correction methods for urine microscopic samples of formed elements.

[0031] Figure 4 Comparison of imaging results of different color correction methods for cross-section samples of pine rootstock. Detailed Implementation

[0032] The following is combined Figures 1 to 4 The present invention will be further described in detail below with reference to specific embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0033] The core of this invention is to provide a multi-scale color correction method for Fourier layered imaging, which aims to solve the global color shift problem existing in the current Fourier layered microscopy imaging, reduce the impact of spatial non-uniform illumination on illumination estimation, overcome the defects of traditional global color correction algorithms such as loss of detail, insufficient correction, and the inability of physical reference schemes to be fully automated, improve color restoration while preserving sample structural details, reduce the impact of block processing on illumination estimation, and adapt to fully automated high-throughput microscopy imaging scenarios.

[0034] Example 1 This embodiment provides a multi-scale color correction method for Fourier layered imaging. To clearly illustrate the specific implementation process of this method, the hardware platform on which this embodiment relies is first described. This embodiment is based on a self-built Fourier layered microscopy imaging system, which mainly includes an optical module, an image sensor, an illumination module, a microcontroller, and a computing module.

[0035] The optical module features a 2x objective lens with a numerical aperture of 0.05. An image sensor, used to capture optical images, has an effective pixel resolution of 5120×5120. The illumination module employs... The system uses a WS2812B type RGB LED array, with a center-to-center spacing of 3mm between each LED. The vertical distance between the RGB LED array and the sample plane is set to 45mm, and the peak wavelengths of the red, green, and blue LEDs are 620nm, 525nm, and 450nm, respectively. The system employs an STM32F103C8T6 microcontroller for timing control of the RGB LED array and synchronous triggering with the image sensor. The computation module uses a wavelet-optimized alternating projection algorithm to complete the subsequent high-resolution image reconstruction.

[0036] Combination Figure 1In addition to the aforementioned Fourier layered microscopy imaging system, this embodiment provides a multi-scale color correction method for Fourier layered imaging, the specific execution steps of which are described in detail below.

[0037] Before performing step S1, a low-resolution original image sequence of red, green, and blue channels corresponding to multi-angle illumination is first obtained using a Fourier layered microscopy system.

[0038] Specifically, the microcontroller controls the RGB LED array to sequentially illuminate LEDs at different spatial positions according to a preset timing sequence, thereby achieving multi-angle oblique incidence plane wave illumination. When an LED illuminates at each angle, the image sensor synchronously triggers the capture of a low-resolution RGB raw image of the sample at the current illumination angle. By traversing and illuminating all LEDs in the entire RGB LED array, a sequence of low-resolution RGB raw images corresponding to multi-angle illumination is ultimately obtained.

[0039] like Figure 1 As shown, step S1 involves constructing a three-level resolution pyramid and performing multi-scale adaptive segmentation. For the acquired low-resolution RGB original image sequence, in order to decompose and extract color and texture features in a multi-scale space, this embodiment constructs a three-level resolution pyramid (fine, medium, and coarse) for each frame of the original image sequence. Then, the system calculates the local gray-level variance of each level of the image layer by layer. The local gray-level variance reflects the texture complexity and information content of different regions of the image. To balance computational efficiency with the integrity of local details, this embodiment performs adaptive segmentation of each level of the image based on the local gray-level variance.

[0040] The specific segmentation strategy is as follows: a preset threshold T is determined based on the local gray-level variance set of each image level, and the preset threshold T satisfies the following formula:

[0041] In the formula, For the preset threshold, This represents the mean of the local grayscale variance of the current image level. This represents the standard deviation of the local grayscale variance of the current image level. This is a preset coefficient. For regions where the local grayscale variance is greater than this preset threshold T, they are identified as complex tissue sample regions such as cells or stained tissues, and are divided into sub-blocks of the first size. The specific size of this first-size sub-block is... Pixels; and for regions where the local grayscale variance is less than or equal to the preset threshold T, they are determined to be background regions with no sample occlusion and a flat image, and are divided into second-size sub-blocks, the specific size of which is [missing information]. Pixels. In this way, all regions of the entire image are divided into multiple image sub-blocks of corresponding sizes, and all the divided image sub-blocks present a non-overlapping juxtaposition structure in spatial structure, thus completing the multi-scale adaptive slicing process of the entire image.

[0042] like Figure 1 As shown, step S2 involves feature extraction and clustering to filter significant background blocks.

[0043] After adaptive segmentation, the system needs to identify background blocks without tissue samples from all image sub-blocks to provide a benchmark for subsequent illumination estimation. To this end, this embodiment first extracts two-dimensional statistical features from each image sub-block, specifically local average brightness and local contrast features. The local contrast features are characterized by calculating the pixel standard deviation within each image sub-block.

[0044] Subsequently, the system uses the K-Means unsupervised clustering algorithm to automatically classify all image sub-blocks with extracted features. In this embodiment, the number of cluster centers for the K-Means unsupervised clustering algorithm is set. This automatically divides all image sub-blocks into two distinct clusters: the first cluster and the second cluster. The first cluster has low brightness and high variance, corresponding to the tissue sample region containing biological cells and stained tissues, i.e., the tissue sample cluster. The second cluster has high brightness and low variance, corresponding to the background region without sample occlusion, i.e., the background cluster.

[0045] After clustering, the system performs a removal operation, discarding tissue sample clusters belonging to the first cluster to reduce the interference of biological sample color, staining process, and complex tissue structure on subsequent light source estimation. The system only selects image sub-blocks belonging to the background cluster of the second cluster and forms a salient background block set from these image sub-blocks, which serves as the input source for subsequent calculation of the illumination vector.

[0046] like Figure 1 As shown, step S3 is to calculate the confidence weight and solve for the illumination vector.

[0047] Due to spatial differences in light intensity distribution and fluctuations in the spectral performance of individual LEDs within an RGB LED array, even within the background area, different sub-blocks will exhibit variations in color and brightness. To comprehensively utilize the color and brightness information of different background blocks and reduce the impact of a single background block on illumination estimation, this embodiment introduces a confidence-weighted mechanism based on feature spatial distance.

[0048] Specifically, the system first calculates the first in the set of salient background blocks. The feature space Euclidean distance from each salient background patch to the corresponding background cluster center of the background region. The smaller the Euclidean distance in the feature space, the closer the features of the salient background block are to the cluster center of the background cluster, and the higher the confidence level of the background color information it contains. Therefore, this embodiment calculates the confidence weight of the salient background block according to the following formula:

[0049] In the formula, For the first Confidence weights for each significant background block and All are significant background block numbers and their values ​​range from 1 to 2. , This represents the total number of salient background blocks in the set of salient background blocks. and The first The and the first The Euclidean distance in the feature space corresponding to each salient background block It is a positive number.

[0050] After obtaining the confidence weight for each salient background block, the system combines this confidence weight to perform weighted fusion of the color information of all salient background blocks, thereby calculating an initial illumination vector that characterizes the light source characteristics of the current imaging system. The weighted fusion process is calculated using the following formula:

[0051] In the formula, The initial illumination vector obtained from the calculation. This represents the total number of salient background blocks in the set of salient background blocks. Number the prominent background blocks and , For the first The confidence weights of each significant background block, and Indicates the first The average color vectors of the red, green, and blue channels of each salient background patch are used. This fusion process combines the color information of multiple salient background patches and performs a weighted summation to reduce the impact of a single background patch on the illumination estimation results.

[0052] To ensure the numerical stability of the subsequent correction process and avoid correction collapse under abnormal lighting data, this embodiment applies a green channel normalization constraint to the initial lighting vector after obtaining it. If the initial lighting vector... for Then the normalized illumination vector Satisfy the following formula:

[0053] In the formula, , and These are the red, green, and blue channel components of the initial illumination vector, respectively.

[0054] like Figure 1 As shown, step S4 is the white balance correction for low-resolution images.

[0055] In this embodiment, before performing phase retrieval iterative reconstruction using the Fourier stacked algorithm, i.e., during the low-resolution RGB original image sequence stage, channel-by-channel white balance correction is performed on the original low-resolution RGB image sequence based on the solved illumination vector. Performing color correction before iteration reduces the negative interference of LED array color shift on the subsequent convergence process of the Fourier stacked phase retrieval iterative process, helps improve the accuracy of phase retrieval, and enhances the quality of the final super-resolution reconstruction.

[0056] In the specific correction process, the system iterates through every pixel in the original low-resolution RGB image and uses the finally determined illumination vector to perform pixel-by-pixel and color-channel division operations on the original low-resolution RGB image sequence using the following formula:

[0057] In the formula, The pixel values ​​of the low-resolution color image obtained after color cast correction. These are the original image pixel values ​​from a low-resolution RGB original image sequence. This indicates a channel-by-channel division operation, while This is the illumination vector after normalization constraints. Through this pixel-by-pixel, channel-by-channel division mapping, the relative gain of each color channel in the original image is uniformly corrected, thereby reducing global spectral color shift in the low-resolution stage and outputting a color-shifted low-resolution color image.

[0058] like Figure 1 As shown, step S5 involves spatial alignment, frequency domain regularization deconvolution processing, and color-structure feature fusion to generate a high-resolution color micrograph.

[0059] After color correction of the low-resolution image, in order to obtain a microscopic image that possesses both super-resolution spatial detail and true color, the system needs to perform multi-feature fusion between the color-corrected low-resolution color image and the high-resolution grayscale image obtained through Fourier layer iterative reconstruction. Prior to this, the high-resolution grayscale image was obtained by the computation module calling the wavelet-optimized alternating projection algorithm to perform alternating projection phase recovery iterative reconstruction on the low-resolution RGB image sequence after white balance correction in step S4. To ensure the convergence stability and reconstruction efficiency of the iterative results, this embodiment sets the number of iterations to 10.

[0060] The wavelet-optimized alternating projection algorithm uses the Daubechies4 wavelet basis for two-level wavelet decomposition. After each alternating projection update, the wavelet high-frequency coefficients of the current reconstructed image are processed according to the following soft thresholding function:

[0061] In the formula, These are wavelet high-frequency coefficients. These are the high-frequency coefficients after soft thresholding. Soft threshold, This is the threshold adjustment coefficient. This is an estimate of the noise standard deviation, where the subscript... Used to characterize noise terms, This represents the total number of pixels in the currently processed image. Represents the natural logarithm function. Represents a symbolic function. This represents the function that takes the maximum value. The image after soft thresholding then enters the next alternating projection update, until 10 iterations are completed.

[0062] The specific execution process of multi-feature fusion includes the following three cascaded sub-steps.

[0063] Sub-step S501 is spatial alignment. Since the grayscale image output by Fourier stacked imaging after iterative reconstruction has higher spatial resolution and image size than the low-resolution color image, the system first uses a bicubic interpolation algorithm to upsample and enlarge the color-shifted low-resolution color image. The enlarged color image is consistent with the high-resolution grayscale image in image size and field of view geometry, thus completing the spatial alignment between the two.

[0064] Sub-step S502 is frequency domain regularization deconvolution processing. Because ordinary bicubic interpolation magnification introduces blurring and high-frequency smoothing effects at image edges and textures, leading to color boundary diffusion, this embodiment first decouples the red, green, and blue channels of the magnified color image through linear transformation, thereby extracting the low-frequency color features of the image, specifically satisfying the following formula:

[0065] In the formula, For the luminance component, and Low-frequency color characteristics, , and These are the pixel values ​​of the red, green, and blue channels of the magnified color image. This is a linear transformation matrix. Next, combining the point spread function of the Fourier layered microscopy imaging system, regularized deconvolution processing is performed in the frequency domain, specifically satisfying the following formula:

[0066] In the formula, The first one after deconvolution processing The low-frequency color feature is located in the frequency domain. The value at that location, For the magnified version of the first The low-frequency color feature is located in the frequency domain. The value at that location, The frequency domain representation of the known point spread function of the imaging system. for conjugate, This is the regularization parameter. Regularization during deconvolution can reduce edge blurring and color overflow caused by interpolation amplification, thus clarifying color boundaries.

[0067] Sub-step S503 is color-structure feature fusion. Although the boundaries of low-frequency color features are clarified after deconvolution, they still lack high-frequency spatial structure texture. Therefore, in this embodiment, the high-resolution grayscale image obtained by Fourier stacked iterative reconstruction is used as the high-frequency structure constraint prior to construct the following joint optimization model:

[0068] In the formula, The fusion feature to be solved, The optimal solution for the joint optimization model. The color components extracted from the fusion features. The low-frequency color features after deconvolution processing The brightness component is extracted from the fusion features. For high-resolution grayscale images, For gradient operators, and These are the weighting coefficients.

[0069] In this joint optimization model, the low-frequency color features after deconvolution are fused with the high-frequency structural features of the high-resolution grayscale image. The optimal solution... The luminance and chromaticity components are respectively obtained through and The extracted values ​​are used as input for the inverse mapping. Finally, the fused multi-feature data is restored to red, green, and blue three-channel pixel values ​​through the following inverse mapping:

[0070] In the formula, The linear transformation matrix The inverse matrix of the matrix. Through the above processing, a high-resolution color micrograph with spatial resolution, true color, and details of sample edges and cell texture is output.

[0071] The solution in this embodiment can reduce the global blue-purple color shift caused by the WS2812B RGB LED array and reduce the impact of differences in different background areas on illumination vector estimation. The entire correction process does not require the introduction of external physical references such as white paper or standard color charts. The entire process can be run automatically and at high throughput by the microcontroller and computing module, while preserving the staining layers and microstructure of biological samples, providing image support for clinical digital pathology diagnosis and high-throughput cell observation.

[0072] Example 2 This embodiment, based on the Fourier in-slice microscopy system and basic steps described in Embodiment 1, conducts simulation experiments on a standard color chart used for color correction evaluation to verify the correction accuracy of the multi-scale color correction method. To simulate complex color shifts caused by hardware limitations in real imaging environments, the experiment first artificially adds global RGB gain deviation and spatially related light intensity deviation to the standard color chart image to simulate the non-uniform color shift caused by the RGBLED array of the Fourier in-slice microscopy system. Subsequently, the experiment uses the method provided in this invention, the traditional white point algorithm, and the gray-world algorithm to conduct correction experiments, and selects four indicators for evaluation: color difference value calculated based on the CIEDE2000 color difference formula, peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and background gray-level standard deviation.

[0073] This embodiment uses the above four indicators to compare the processing results of the method of the present invention, the white point algorithm, and the gray-world algorithm to evaluate color deviation, image distortion, structure preservation, and background uniformity; at the same time, it observes the gray-level changes at the boundaries of image blocks to check for obvious block artifacts. Figure 2 As shown, Figure 2 (a) in the figure is the baseline truth map of the Macbeth color chart; Figure 2 (b) in the middle is in Figure 2The distorted image obtained by applying global RGB gain bias and spatial location-related light intensity bias to the reference truth image shown in (a); Figure 2 (c) Figure 2 (d) and Figure 2 (e) represents the methods used in this invention, namely the white point algorithm, the gray world algorithm, and the method of this invention. Figure 2 The image shown in (b) is the result obtained after color correction of the distorted image. By... Figure 2 (c) to Figure 2 (e) in the middle are respectively with Figure 2 By comparing (a) in the above, we can evaluate the effect of different color correction methods on the restoration of image color deviation, structural information and background uniformity.

[0074] The corresponding quantitative indicators are shown in Table 1. As can be seen from Table 1, the PSNR of the method of the present invention is 23.42, SSIM is 0.8672, CIEDE2000 is 7.63, and background Std is 0.2468. Among the methods compared, the PSNR and SSIM of the method of the present invention are higher, while CIEDE2000 is lower.

[0075] Table 1 Comparison of Quantitative Indicators of Different Correction Algorithms

[0076] Example 3 This embodiment further applies the method provided by the present invention to imaging tests of various real samples. The test samples specifically include the USAF 1951 standard resolution plate, clinically commonly used urine formed element stained samples, and pine root and stem cross-section plant cell samples. To establish a multi-dimensional performance comparison, four control groups were set up during the experiment: an uncorrected group, a white paper physical correction group, a white point algorithm group, and a grayscale world algorithm group as controls. The comparison results of the urine formed element stained samples and the pine root and stem cross-section plant cell samples are as follows: Figure 3 and Figure 4 As shown.

[0077] Under the test conditions of this embodiment, the uncorrected image exhibits a blue-purple color cast in various field-of-view regions, and the microstructural details of the samples are blurred. Traditional global white point algorithms and grayscale world algorithms can improve the global color cast of the entire image, but color inconsistencies are still observed under spatially non-uniform lighting conditions, and overexposure and loss of detail are observed in local highlight areas of the image. The white paper physical correction scheme can improve global color reproduction, but color differences caused by local LED light intensity fluctuations are still observed. Furthermore, this physical correction scheme relies on hardware switching, making it difficult to achieve fully automatic operation without human intervention.

[0078] In contrast, after processing using the method provided by this invention, because a three-level resolution pyramid (fine, medium, and coarse) is constructed at the low-resolution stage and adaptive full-image segmentation is achieved based on local gray-level variance, and the number of cluster centers is set to [value missing]. The K-means unsupervised clustering algorithm automatically identifies high-brightness and low-variance background clusters and selects them as a set of significant background blocks, thereby reducing the interference of staining in biological tissue regions such as urine samples on illumination estimation. Furthermore, confidence weights are used to weight and fuse the color information of multiple significant background blocks to reduce the impact of a single background block on illumination estimation.

[0079] The final super-resolution color image reconstruction results demonstrate that the method of this invention can reduce the global color cast of low-resolution RGB original images and improve the uniformity of background color across the entire field of view in the reconstructed image. Simultaneously, sharp edges, cellular microtexture, and staining levels of the samples are preserved. The entire correction and super-resolution reconstruction process requires no external reference objects such as white paper or color charts, and requires no manual intervention, making it suitable for fully automated, high-throughput clinical imaging and digital pathology diagnosis scenarios.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-scale color correction method of Fourier ptychographic imaging, comprising obtaining a sequence of low-resolution red, green, and blue channel raw images corresponding to multi-angle illumination collected by a Fourier ptychographic microscopic imaging system, correcting the low-resolution images, and reconstructing high-resolution images, characterized in that, Includes the following steps: S1. Construct a three-level resolution pyramid (fine, medium, and coarse) for the original image sequence of the low-resolution red, green, and blue channels, and adaptively divide each level of image into blocks based on the local gray-level variance to obtain multiple non-overlapping image sub-blocks. S2. Extract the local average brightness and local contrast features of each image sub-block, use the K-means unsupervised clustering algorithm to divide all image sub-blocks into tissue sample regions and background regions, and select the image sub-blocks in the background regions as a set of significant background blocks. S3. Calculate the confidence weight of each of the salient background blocks, and combine the confidence weight to perform weighted fusion of the color information of all the salient background blocks to obtain an initial illumination vector characterizing the light source characteristics of the imaging system, and perform green channel normalization constraint on the initial illumination vector to obtain the illumination vector. S4. Perform channel-by-channel white balance correction on the low-resolution RGB original image sequence based on the illumination vector to obtain a low-resolution color image after color shift correction. S5. Spatial alignment, frequency domain regularization deconvolution processing, and color-structure feature fusion are performed on the low-resolution color image after color deviation correction and the high-resolution grayscale image obtained by Fourier stacking iteration reconstruction to generate a high-resolution color microscopic image.

2. The multi-scale color correction method for Fourier layered imaging according to claim 1, characterized in that, In step S1, the adaptive segmentation of each image level based on local gray-level variance includes: determining a preset threshold T based on the set of local gray-level variances of each image level, wherein the preset threshold T satisfies the following formula: in, For the preset threshold, This represents the mean of the local grayscale variance of the current image level. This represents the standard deviation of the local grayscale variance of the current image level. The preset coefficient is used; regions with local gray-level variance greater than the preset threshold T are divided into first-size sub-blocks, and regions with local gray-level variance less than or equal to the preset threshold T are divided into second-size sub-blocks; wherein, the size of the first-size sub-block is... pixels, the size of the second size sub-block is Pixel.

3. The multi-scale color correction method for Fourier layered imaging according to claim 1, characterized in that, In step S2, the number of cluster centers of the K-means unsupervised clustering algorithm The local contrast features are characterized by pixel standard deviation; the K-means unsupervised clustering algorithm automatically divides all image sub-blocks into a first cluster and a second cluster. The first cluster is a tissue sample cluster with lower average brightness and higher variance among the two clusters, corresponding to the tissue sample region; the second cluster is a background cluster with higher average brightness and lower variance among the two clusters, corresponding to the background region.

4. The multi-scale color correction method for Fourier layered imaging according to claim 1, characterized in that, In step S3, calculating the confidence weight of each of the salient background blocks includes: calculating the first... The feature space Euclidean distance from each of the salient background blocks to the corresponding background cluster centers of the background regions. And calculate the confidence weight of the significant background block according to the following formula: in, For the first Confidence weights for each significant background block and All are significant background block numbers and their values ​​range from 1 to 2. , This represents the total number of salient background blocks in the set of salient background blocks. and The first The and the first The Euclidean distance in the feature space corresponding to each salient background block It is a positive number.

5. The multi-scale color correction method for Fourier layered imaging according to claim 1, characterized in that, In step S3, the weighted fusion calculates the initial illumination vector using the following formula: In the formula, Let be the initial illumination vector. The total number of salient background blocks in the set of salient background blocks. Number the prominent background blocks and , For the first Confidence weights for each significant background block For the first The average color vector of a significant background block; applying a green channel normalization constraint to the initial illumination vector means: if the initial illumination vector for The illumination vector ultimately used for white balance correction Satisfy the following formula: in, , and These are the red channel component, green channel component, and blue channel component of the initial illumination vector, respectively.

6. The multi-scale color correction method for Fourier layered imaging according to claim 1, characterized in that, In step S4, the channel-by-channel white balance correction is performed before the phase retrieval iterative reconstruction is performed using the Fourier stacking algorithm, and the white balance correction is performed by performing pixel-by-pixel and color channel-by-color channel division on the low-resolution RGB original image sequence using the following formula: In the formula, These are the pixel values ​​of the image after color cast correction. The original image pixel values ​​in the low-resolution RGB original image sequence. This indicates a channel-by-channel division operation. Let be the illumination vector.

7. The multi-scale color correction method for Fourier layered imaging according to claim 1, characterized in that, In step S5, the spatial alignment, frequency domain regularization deconvolution processing, and color-structure feature fusion include the following sub-steps: S501. The low-resolution color image after color shift correction is enlarged using a bicubic interpolation algorithm to make its image size consistent with that of the high-resolution grayscale image, thus completing spatial alignment. S502. The red, green, and blue channels of the magnified color image are decoupled through the following linear transformation to extract the low-frequency color features of the image. Then, combined with the point spread function of the Fourier layered microscopy imaging system, regularized deconvolution processing is performed in the frequency domain: in, For the luminance component, and Low-frequency color characteristics, , and These are the pixel values ​​of the red, green, and blue channels of the magnified color image. It is a linear transformation matrix. The first one after deconvolution processing The low-frequency color feature is located in the frequency domain. The value at that location, For the magnified version of the first The low-frequency color feature is located in the frequency domain. The value at that location, Here is the frequency domain representation of the point spread function. for conjugate, This is the regularization parameter.

8. The multi-scale color correction method for Fourier layered imaging according to claim 7, characterized in that, In step S5, the spatial alignment, frequency domain regularization deconvolution processing, and color-structure feature fusion further include the following sub-steps: S503. Using the high-resolution grayscale image as a high-frequency structural constraint prior, construct the following joint optimization model: fuse the deconvolutioned low-frequency color features with the structural features of the high-resolution grayscale image, and restore them to red, green, and blue three-channel pixel values ​​through inverse mapping, outputting the final high-resolution color micrograph: in, The fusion feature to be solved The optimal solution for the joint optimization model. The color components extracted from the fusion features. The low-frequency color features after deconvolution processing The brightness component is extracted from the fusion features. For the high-resolution grayscale image, For gradient operators, and These are the weighting coefficients. The linear transformation matrix The inverse matrix, wherein the luminance and chrominance components in the inverse mapping are respectively derived from the optimal solution. pass and Extracted.

9. The multi-scale color correction method for Fourier layered imaging according to claim 1, characterized in that, The high-resolution grayscale image is obtained by iterative reconstruction using a wavelet-optimized alternating projection algorithm. The wavelet-optimized alternating projection algorithm uses the Daubechies4 wavelet basis for two-level wavelet decomposition. After each alternating projection update, soft thresholding is performed on the high-frequency wavelet coefficients, and the number of iterations is set to 10.

10. The multi-scale color correction method for Fourier layered imaging according to claim 1, characterized in that, The Fourier layered microscopy imaging system includes an illumination module, which employs... The WS2812B type red, green and blue light-emitting diode array has a center-to-center spacing of 3mm between each light-emitting diode, a vertical distance of 45mm between the red, green and blue light-emitting diode array and the sample plane, and peak wavelengths of the red, green and blue light-emitting diodes are 620nm, 525nm and 450nm respectively.

Images