A fluorescent immunohistochemical automatic exposure value method and system

Abstract

The present application relates to the technical field of image data processing, in particular to a fluorescent immunohistochemical automatic exposure value method and system. The method comprises: controlling the excitation light source to continuously collect two frames of probe frames with a preset micro-dose parameter, and obtaining a net signal image matrix by using a pre-stored dark field image to denoise; extracting the gradient features of the first net signal matrix to construct a morphology and light intensity combined confidence matrix combined with the gray value, representing the probability of each pixel point being an effective biological tissue signal; using the confidence matrix to statistically calculate the fluorescence decay ratio of the net signal matrix by weighting, and inversely deducing the theoretical maximum fluorescence intensity of the sample before light bleaching occurs; and calculating the optimal exposure time according to the theoretical intensity, the target gray value and the probe time. The scheme of the present application can realize zero-loss protection of pathological samples, accurately extract effective signals from a low signal-to-noise ratio background, and improve the analysis accuracy.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology. More specifically, this invention relates to an automated exposure method and system for fluorescence immunohistochemistry. Background Technology

[0002] Multiplex fluorescence immunohistochemistry (MHI) technology enables in-situ detection of multiple biomarkers in the tumor microenvironment by labeling the same tissue section with multiple fluorescent dyes. In digital pathological scanning, automatic exposure is a crucial step determining the accuracy of quantitative image analysis, directly impacting the subsequent assessment of key protein expression levels and the reliability of diagnostic results. Current technologies commonly employ a trial-and-error feedback method to determine exposure time: an initial exposure time is set, the histogram is analyzed, and if unsuitable, the time is adjusted for repeated exposure. This process typically requires multiple full-power excitations to lock in the optimal value. However, fluorescent dyes are highly sensitive to light; each trial exposure causes irreversible photochemical quenching of fluorescent molecules. By the time the parameters are finally determined for formal imaging, the true signal of the sample has often significantly attenuated, leading to a lower quantitative score.

[0003] To reduce photobleaching, some existing technologies attempt to perform pre-scans for extremely short periods, significantly shortening the exposure time of the excitation light to reduce sample damage. After acquiring images under extremely short exposures, these improved techniques typically process the image data using traditional methods based on overall image average grayscale or simple thresholding. The aim is to determine the sample's brightness distribution by analyzing these low-dose pre-scan images, thereby calculating the exposure parameters required for subsequent formal acquisition.

[0004] However, under extremely short exposures, photon shot noise and sensor dark current noise dominate in the image, resulting in an extremely low signal-to-noise ratio. Traditional methods based on average grayscale of the entire image or simple threshold segmentation struggle to extract the weak, effective fluorescence signal from the strong background noise, causing the calculated exposure time to deviate significantly from the actual requirements and failing to meet the demands of high-precision pathological analysis. Summary of the Invention

[0005] The purpose of this invention is to provide an automated exposure method and system for fluorescence immunohistochemistry, which can effectively avoid signal loss caused by trial and error exposure and accurately calculate exposure parameters in low signal-to-noise ratio environments. To this end, this invention provides solutions in the following two aspects.

[0006] In a first aspect, the present invention provides an automated exposure method for fluorescence immunohistochemistry, comprising:

[0007] The excitation light source is controlled to continuously acquire data from the current field of view using preset micro-dose parameters, obtaining a first probe frame and a second probe frame. The first and second probe frames are then processed using pre-stored dark-field images to obtain a denoised first net signal image matrix and a second net signal image matrix. Gradient features of the first net signal image matrix are extracted, and a morphology-light intensity joint confidence matrix is ​​constructed by combining pixel grayscale values. This confidence matrix characterizes the probability that each pixel represents a valid biological tissue fluorescence signal. The first and second net signal image matrices are then weighted using the confidence matrix to calculate a weighted fluorescence attenuation ratio. Based on the weighted fluorescence attenuation ratio and sensor characteristics, the theoretical maximum fluorescence intensity of the sample before photobleaching is inferred. Finally, the optimal exposure time for formal acquisition is calculated based on the theoretical maximum fluorescence intensity, the preset target grayscale value, and the probe exposure time in the micro-dose parameters.

[0008] In this way, by using dual-frame micro-dose acquisition and subsequent mathematical back-calculation, the bombardment of the sample by photons can be minimized, avoiding the irreversible signal loss caused by traditional multi-round trial and error exposure, and thus protecting precious pathological samples.

[0009] Preferably, when constructing the morphology and light intensity joint confidence matrix, the expression for calculating the signal confidence weight of each pixel is as follows: In the formula, Representing coordinates The signal confidence weight of each pixel; Represents the coordinates in the first net signal image matrix The pixel grayscale value at that location; Represents the logarithmic gain coefficient; This represents the local gradient magnitude of the pixel. This represents the gradient penalty coefficient.

[0010] Thus, by introducing a confidence index combining morphology and light intensity, using gradient features to suppress isolated noise, and using grayscale logarithms to enhance the identification of weak signal areas, it is possible to accurately locate real biological tissue signal regions under harsh imaging conditions of low light and high noise.

[0011] Preferably, the calculation of the weighted fluorescence attenuation ratio specifically includes: using the signal confidence weight as a weighting coefficient, performing a weighted summation on the first net signal image matrix and the second net signal image matrix respectively, and dividing the weighted sum of the second net signal image matrix by the weighted sum of the first net signal image matrix to obtain the weighted fluorescence attenuation ratio.

[0012] In this way, by using confidence weights for weighted statistics, the calculation of attenuation rate focuses on tissue regions with high confidence, automatically eliminating the interference of background noise on the calculation results and ensuring the accuracy of attenuation assessment.

[0013] Preferably, the expression for the theoretical maximum fluorescence intensity of the reverse-engineered sample before photobleaching is as follows: In the formula, This represents the theoretical maximum fluorescence intensity of the sample at time zero, obtained through reverse calculation. This represents the currently observed effective highlight signal value; Indicates the weighted fluorescence decay ratio; This represents the nonlinearity correction factor for the photoelectric conversion of the sensor. This indicates the photobleaching kinetic index.

[0014] Thus, by combining the first-order photochemical dynamics principle and sensor characteristics to perform mathematical modeling, the initial true state of the sample can be restored from the slightly attenuated signal, providing an accurate benchmark for subsequent exposure calculations.

[0015] Preferably, the expression for calculating the optimal exposure time ultimately used for formal data acquisition is as follows: In the formula, Indicates the optimal exposure time; This indicates the probe exposure time when the probe frame is acquired; This indicates the preset target peak gray level of the image; This represents the theoretical maximum fluorescence intensity of the sample at time zero, obtained through reverse calculation. This represents the safety margin coefficient.

[0016] Preferably, when calculating the weighted fluorescence attenuation ratio, if the weighted sum of the first net signal image matrix is ​​less than a preset minimum energy threshold, the weighted fluorescence attenuation ratio is directly set to 1, and the current field of view is marked as the background field of view.

[0017] Thus, by setting a minimum energy threshold, invalid or erroneous calculations can be avoided in background fields with no or extremely weak signals, thereby improving the robustness of the system.

[0018] Preferably, the local gradient magnitude is calculated using the Sobel operator, specifically by summing the absolute values ​​of the grayscale differences of the pixels in the horizontal and vertical directions.

[0019] Preferably, the step of processing the first probe frame and the second probe frame using the pre-stored dark field image specifically includes: subtracting the gray value of the corresponding coordinate of the dark field image from the gray value of the probe frame, and comparing the result with 0 to take the maximum value, so as to eliminate sensor thermal noise.

[0020] Preferably, the effective high-brightness signal value is obtained by: sorting the values ​​in the confidence matrix, selecting the coordinate points corresponding to the top preset proportion of the largest values, and calculating the average gray value of these coordinate points in the first net signal image matrix.

[0021] In the second aspect, an automated fluorescence immunohistochemistry exposure and value acquisition system includes:

[0022] The processor; the memory storing computer instructions for automatic exposure and value acquisition in fluorescence immunohistochemistry, which, when executed by the processor, cause the system to perform the aforementioned automatic exposure and value acquisition method in fluorescence immunohistochemistry.

[0023] The beneficial effects of this invention are as follows: by combining dual-frame micro-dose acquisition with mathematical modeling, the photobleaching loss caused by multiple full-power trial and error in traditional methods is eliminated, achieving zero-loss protection for precious pathological samples; at the same time, by constructing a morphology and light intensity joint confidence matrix and utilizing the nonlinear combination of gradient and brightness, the true signal intensity of biological tissues can be accurately identified and restored from low signal-to-noise ratio images, thereby calculating the optimal exposure time without damage in one go based on the theoretical initial intensity, which significantly improves the accuracy and diagnostic reliability of fluorescence immunohistochemical quantitative analysis. Attached Figure Description

[0024] Figure 1 The flowchart illustrating the steps of the automated exposure method for fluorescence immunohistochemistry in this embodiment is shown in the schematic diagram.

[0025] Figure 2 This is a comparison chart of the time-series changes in fluorescence signal intensity between existing technologies and the present invention;

[0026] Figure 3 This is a schematic diagram illustrating the principle of effective signal extraction under low light conditions based on confidence weighting. Detailed Implementation

[0027] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0028] like Figure 1 As shown in this embodiment, an automated exposure method for fluorescence immunohistochemistry includes the following steps:

[0029] Step S1: Control the excitation light source to continuously acquire the current field of view with preset micro-dose parameters, acquire the first probe frame and the second probe frame, and process the first probe frame and the second probe frame using the pre-stored dark field image to obtain the first net signal image matrix and the second net signal image matrix after denoising.

[0030] Specifically, the excitation source of the microscope system is controlled to continuously acquire data from the current field of view using preset micro-dose parameters. These micro-dose parameters refer to controlling the excitation source shutter to open at an extremely low energy density of 5% to 10% of full power, or setting an extremely short fixed exposure time, such as 10 milliseconds. The system completes two consecutive exposure acquisitions within a very short time interval, obtaining two frames containing the original grayscale data.

[0031] To eliminate the interference of inherent dark current noise and thermal noise from the sensor on subsequent weak signal analysis, the system uses pre-stored dark-field images to process the acquired first and second probe frames. The dark-field images are acquired with the optical shutter closed, using the same exposure time and gain settings as the probe frames.

[0032] The processing procedure is as follows: For each pixel in the image matrix, the gray value of that pixel in the probe frame is subtracted from the gray value of the corresponding coordinate point in the dark field image. If the subtraction result is less than zero, the result is corrected to zero, thus obtaining the first and second net signal image matrices after denoising. For example, assuming the original gray value of the first probe frame at a certain pixel coordinate is 120, and the corresponding dark field noise value is 10, then the denoised net signal value is 110; if the original gray value is 5 and the dark field noise value is 8, then the calculation result is taken as 0, to prevent negative value anomalies.

[0033] Thus, by acquiring the raw data stream containing fluorescence decay information while minimizing photon bombardment of the sample, and combining it with dark-field correction, we can obtain net signal data with sensor noise removed, laying the foundation for subsequent accurate analysis.

[0034] Step S2: Extract the gradient features of the first net signal image matrix, and construct a morphology and light intensity joint confidence matrix by combining pixel gray values. The confidence matrix is ​​used to characterize the probability that each pixel is a valid biological tissue fluorescence signal.

[0035] Specifically, in this step, the system uses the Sobel operator to calculate the absolute value of the gray-level difference of each pixel in the horizontal and vertical directions, and adds these two absolute values ​​to obtain the local gradient magnitude. The confidence matrix is ​​constructed following the principle of high weight for high-brightness and smooth-gradient regions, and low weight for low-brightness or high-gradient regions, aiming to distinguish real biological tissue signals from discrete noise points. The expression for calculating the confidence weight of each pixel signal is as follows:

[0036] ;

[0037] In the formula, Representing coordinates The signal confidence weight of each pixel; Represents the coordinates in the first net signal image matrix The pixel grayscale value at that location; Represents the logarithmic gain coefficient; This represents the local gradient magnitude of the pixel. This represents the gradient penalty coefficient.

[0038] For example, let's set the logarithmic gain coefficient to 0.1 and the gradient penalty coefficient to 0.5. Assuming pixel A is a real signal with a grayscale value of 200, and its surrounding gradient is relatively flat, resulting in a gradient magnitude of 10, the numerator is approximately 608, the denominator is 6, and the final confidence weight is approximately 101.3. Conversely, assuming pixel B is noise, although its grayscale value is also 200, its isolated bright spot results in a gradient magnitude as high as 150, leading to a denominator of 76 and a final confidence weight of only 8. It is evident that this calculation method significantly increases the confidence weight of real signal points compared to noise points, effectively suppressing the weight of noise.

[0039] Thus, by constructing the above algorithm, it is possible to effectively identify which pixels are real biological tissue fluorescence and which are random noise, thereby achieving accurate signal extraction.

[0040] Step S3: Use the confidence matrix to perform weighted statistics on the first net signal image matrix and the second net signal image matrix, calculate the weighted fluorescence attenuation ratio, and based on the weighted fluorescence attenuation ratio and sensor characteristics, infer the theoretical maximum fluorescence intensity of the sample before photobleaching.

[0041] Specifically, using signal confidence weights as weighting coefficients, the net signal image matrices of the two frames are weighted and summed. Then, the weighted sum of the second frame is divided by the weighted sum of the first frame to obtain the weighted fluorescence attenuation ratio. This process focuses the attenuation rate calculation on high-confidence tissue regions. Subsequently, the calculation is performed by reverse derivation using the first-order photochemical kinetics principle, as shown in the following formula:

[0042] ;

[0043] In the formula, This represents the theoretical maximum fluorescence intensity of the sample at time zero, obtained through reverse calculation. This represents the currently observed effective highlight signal value; Indicates the weighted fluorescence decay ratio; This represents the nonlinearity correction factor for the photoelectric conversion of the sensor. This indicates the photobleaching kinetic index.

[0044] For example, assuming the weighted fluorescence attenuation ratio obtained after full-image weighted calculation is 0.95, the selected effective bright signal value is 100, the sensor nonlinearity correction factor is set to 1.0, and the photobleaching kinetic index is set to 1.1, then the intermediate term is calculated as follows: The value in parentheses is 1.0526, and the final calculated theoretical maximum fluorescence intensity is approximately 105.8. This indicates that although the current observed value is only 100, the true intensity of the sample before being irradiated by the micro-dose probe should be 105.8.

[0045] Thus, by using the confidence matrix to eliminate noise interference and accurately calculating the fluorescence decay rate, the theoretical maximum intensity before photobleaching can be deduced, thereby restoring the signal destroyed by the laser.

[0046] Step S4: Calculate the optimal exposure time for formal data acquisition based on the theoretical maximum fluorescence intensity, the preset target gray value, and the probe exposure time in the micro-dose parameters.

[0047] Specifically, the system sets a preset target peak gray level for the image, typically 55000 for a 16-bit camera, while introducing a constant as a safety margin factor. The formula for calculating the optimal exposure time is as follows:

[0048] ;

[0049] In the formula, Indicates the optimal exposure time; This indicates the probe exposure time when the probe frame is acquired; This indicates the preset target peak gray level of the image; This represents the theoretical maximum fluorescence intensity of the sample at time zero, obtained through reverse calculation. This represents the safety margin coefficient.

[0050] Following the calculation example above, if the probe exposure time is 10 milliseconds, the theoretical maximum fluorescence intensity is 105.8, the target peak grayscale is 55000, and the safety margin is 0.9, then the optimal exposure time calculated by the system is approximately 4678 milliseconds. The system will directly use this time to control the camera for the formal exposure, eliminating the need for a test shot, thus maximizing the protection of the pathological sample while ensuring image signal strength.

[0051] In this way, by calculating the final physical exposure time based on the theoretical intensity derived from the reverse calculation, the optimal exposure time without loss can be calculated in one go, thus achieving zero-loss exposure decision-making.

[0052] The following is in conjunction with the appendix Figure 2 and attached Figure 3 The technical effects of the present invention will be further explained.

[0053] Figure 2 The figure shows a comparison of the time-series changes in fluorescence signal intensity between the prior art and the present invention. The curve of the prior art shows a step-like decline, with each tentative exposure corresponding to a sharp drop in the curve, representing the consumption of fluorescence signal; while the curve of the present invention shows an almost straight line, with only a slight drop in the very short prediction phase, and then directly enters the final imaging stage, with the signal intensity remaining at a high level of the initial value.

[0054] Figure 3 This diagram illustrates the principle of effective signal extraction under low light conditions based on confidence weighting. The figure shows the original histogram of the pre-scanned image, which contains high background noise peaks. It also shows the effective signal distribution area after processing by this invention, representing the accurately restored real biological signal. The function curve in the figure illustrates the trend of the confidence weighting function, which is S-shaped. By assigning low weights, the background noise peaks are suppressed and filtered out, while by assigning high weights, the effective biological signal is preserved.

[0055] The present invention also provides an automated exposure and value acquisition system for fluorescence immunohistochemistry. The system includes a processor and a memory, the memory storing computer program instructions. When the processor executes the computer program instructions, it implements the automated exposure and value acquisition method for fluorescence immunohistochemistry described above according to the present invention.

[0056] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and therefore will not be described in detail here.

[0057] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented by computer-readable / executable instructions stored or otherwise maintained on such a computer-readable medium.

[0058] In the description of this specification, "multiple" means at least two, such as two, three or more, etc., unless otherwise expressly and specifically defined.

[0059] While various embodiments of the invention have been shown and described in this specification, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention.

Claims

1. An automated exposure method for fluorescence immunohistochemistry, characterized in that, include: The excitation light source is controlled to continuously acquire the current field of view with preset micro-dose parameters, and the first probe frame and the second probe frame are acquired. The first probe frame and the second probe frame are processed using the pre-stored dark field image to obtain the first net signal image matrix and the second net signal image matrix after denoising. Gradient features of the first net signal image matrix are extracted, and a morphology and light intensity joint confidence matrix is ​​constructed by combining pixel gray values. The confidence matrix is ​​used to characterize the probability that each pixel is a valid biological tissue fluorescence signal. The first net signal image matrix and the second net signal image matrix are weighted and statistically analyzed using the confidence matrix to calculate the weighted fluorescence attenuation ratio. Based on the weighted fluorescence attenuation ratio and sensor characteristics, the theoretical maximum fluorescence intensity of the sample before photobleaching is inferred. Based on the theoretical maximum fluorescence intensity, the preset target gray value, and the probe exposure time in the micro-dose parameters, the optimal exposure time for the final formal acquisition is calculated.

2. The automated exposure method for fluorescence immunohistochemistry values ​​according to claim 1, characterized in that, When constructing the joint confidence matrix of morphology and light intensity, the expression for calculating the signal confidence weight of each pixel is as follows: ； In the formula, Representing coordinates Signal confidence weights at each pixel; Represents the coordinates in the first net signal image matrix The pixel grayscale value at that location; Represents the logarithmic gain coefficient; This represents the local gradient magnitude of the pixel. This represents the gradient penalty coefficient.

3. The automated exposure method for fluorescence immunohistochemistry values ​​according to claim 2, characterized in that, The calculation of the weighted fluorescence attenuation ratio specifically includes: using the signal confidence weight as a weighting coefficient, performing a weighted summation on the first net signal image matrix and the second net signal image matrix respectively, and dividing the weighted sum of the second net signal image matrix by the weighted sum of the first net signal image matrix to obtain the weighted fluorescence attenuation ratio.

4. The automated exposure and value acquisition method for fluorescence immunohistochemistry according to claim 3, characterized in that, The expression for the theoretical maximum fluorescence intensity of the inverse sample before photobleaching is as follows: ； In the formula, This represents the theoretical maximum fluorescence intensity of the sample at time zero, obtained through reverse calculation. This represents the currently observed effective highlight signal value; Indicates the weighted fluorescence decay ratio; This represents the nonlinearity correction factor for the photoelectric conversion of the sensor. This indicates the photobleaching kinetic index.

5. The automated exposure and value acquisition method for fluorescence immunohistochemistry according to claim 4, characterized in that, The expression for calculating the optimal exposure time used in the final data acquisition is as follows: ； In the formula, Indicates the optimal exposure time; This indicates the probe exposure time when the probe frame is acquired; This indicates the preset target peak gray level of the image; This represents the theoretical maximum fluorescence intensity of the sample at time zero, obtained through reverse calculation. This represents the safety margin coefficient.

6. The automated exposure method for fluorescence immunohistochemistry values ​​according to claim 3, characterized in that, When calculating the weighted fluorescence attenuation ratio, if the weighted sum of the first net signal image matrix is ​​less than the preset minimum energy threshold, the weighted fluorescence attenuation ratio is directly set to 1, and the current field of view is marked as the background field of view.

7. The automated exposure method for fluorescence immunohistochemistry values ​​according to claim 2, characterized in that, The local gradient magnitude is calculated using the Sobel operator, specifically by summing the absolute values ​​of the grayscale differences of the pixels in the horizontal and vertical directions.

8. The automated exposure method for fluorescence immunohistochemistry values ​​according to claim 1, characterized in that, The process of using the pre-stored dark field image to process the first probe frame and the second probe frame specifically includes: subtracting the gray value of the corresponding coordinate of the dark field image from the gray value of the probe frame, and comparing the result with 0 to take the maximum value, so as to eliminate sensor thermal noise.

9. The automated exposure method for fluorescence immunohistochemistry according to claim 4, characterized in that, The effective high-brightness signal value is obtained by sorting the values ​​in the confidence matrix, selecting the coordinate points corresponding to the largest preset proportion, and calculating the average gray value of these coordinate points in the first net signal image matrix.

10. An automated exposure and value acquisition system for fluorescence immunohistochemistry, characterized in that, include: processor; A memory storing computer instructions for automatic exposure and value acquisition in fluorescence immunohistochemistry, which, when executed by the processor, cause the system to perform the automatic exposure and value acquisition method for fluorescence immunohistochemistry according to any one of claims 1-9.

Images