Method and device for processing single molecule positioning microscope image
By employing steps such as bandpass filtering, background homogenization, and contrast adjustment, single-molecule localization microscope images are processed automatically, solving the problems of low signal-to-noise ratio and reliance on manual intervention, and achieving high-precision and consistent single-molecule localization.
Patent Information
- Application Number
- CN202511866287.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-11
- Publication Date
- 2026-04-21
AI Technical Summary
In current single-molecule localization microscopy image processing, the low signal-to-noise ratio and reliance on manual intervention lead to inconsistent results and poor repeatability, affecting localization accuracy and the fidelity of super-resolution images.
By employing steps such as bandpass filtering, background homogenization, contrast adjustment, and Gaussian point spread function fitting, single-molecule localization microscope images are automatically processed to remove noise and background interference, improve image quality, and accurately calculate single-molecule coordinates.
It achieves high-precision single-molecule localization without human intervention, improves localization accuracy and image reliability, and ensures consistency and fidelity of results.
Smart Images

Figure CN121904150A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computers, and more specifically to a method and apparatus for processing single-molecule localization microscope images. Background Technology
[0002] Single-Molecule Localization Microscopy (SMLM), encompassing techniques such as Photoactivated Localization Microscopy (PALM), is a fundamental method in the field of super-resolution microscopy. This type of technique achieves nanometer-scale spatial resolution by separating the fluorescence emission of sparse subsets of photoactivated fluorophores over time and then precisely calculating and locating the point spread function (PSF) of these fluorophores in consecutive image frames. The final resolution of the reconstructed super-resolution image essentially depends on the accuracy and precision of these single-molecule localization processes.
[0003] However, accurate identification of single-molecule coordinates in SMLM images is highly dependent on the quality of the raw data. Various interference factors present during actual imaging significantly weaken the signal-to-noise ratio of the image, thus affecting the reliability of localization. To address these issues, existing analysis workflows typically rely on researchers repeatedly adjusting detection thresholds, fitting models, and related parameters based on experience. This trial-and-error approach, heavily reliant on human intervention, is not only inefficient but also prone to introducing subjective bias, leading to inconsistent results and poor repeatability under different user or experimental conditions. Ultimately, this limits the accuracy of single-molecule localization and the fidelity of super-resolution images. Summary of the Invention
[0004] This application provides a method and apparatus for processing single-molecule localization microscope images, thereby improving the accuracy of single-molecule localization.
[0005] This application provides a method for processing single-molecule localization microscope images, including:
[0006] The original single-molecule localization microscope image was subjected to bandpass filtering to obtain a bandpass filtered image;
[0007] The background is homogenized by performing background homogenization on the bandpass filtered image to obtain a background homogenized image;
[0008] A contrast-enhanced image is obtained by performing contrast adjustment processing on the background homogenized image.
[0009] For each pixel in the contrast-enhanced image, the brightness value of the pixel is compared with that of its neighboring pixels. The neighboring pixels are other pixels located within a preset range of the pixel, with the pixel as the center.
[0010] When the brightness value of a pixel is greater than the brightness value of its adjacent pixels, and the brightness value of a pixel is greater than a preset brightness value, the pixel is determined as a candidate molecule position.
[0011] Gaussian point diffusion function is used to fit the pixel region corresponding to the candidate molecule position to obtain the single molecule coordinates. The pixel region to be fitted is a pixel region with a preset shape and size that is cropped from the contrast-enhanced image with the candidate molecule position as the center.
[0012] This application also provides an apparatus for processing single-molecule localization microscope images, comprising:
[0013] The filtering unit is used to perform bandpass filtering on the original single-molecule localization microscope image to obtain a bandpass filtered image.
[0014] The homogenization unit is used to perform background homogenization processing on the bandpass filtered image to obtain a background homogenized image.
[0015] The adjustment unit is used to perform contrast adjustment processing on the background homogenization image to obtain a contrast-enhanced image;
[0016] The comparison unit is used to compare the brightness value of each pixel in the contrast-enhanced image with that of its neighboring pixels. The neighboring pixels are other pixels located within a preset range of the pixel, centered on the pixel.
[0017] The candidate determination unit is used to determine a pixel as a candidate molecule position when the brightness value of a pixel is greater than the brightness value of its adjacent pixels and the brightness value of the pixel is greater than a preset brightness value.
[0018] The coordinate determination unit is used to fit the Gaussian point diffusion function to the pixel region to be fitted corresponding to the candidate molecule position to obtain the single molecule coordinates. The pixel region to be fitted is a pixel region with a preset shape and size that is cropped from the contrast-enhanced image with the candidate molecule position as the center.
[0019] This application also provides an electronic device, including a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to execute steps in any of the methods for processing single-molecule localization microscope images provided in this application.
[0020] This application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute steps in any of the methods for processing single-molecule localization microscope images provided in this application.
[0021] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of any of the methods for processing single-molecule localization microscope images provided in this application.
[0022] This application embodiment can perform bandpass filtering on the original single-molecule localization microscope image to obtain a bandpass filtered image; perform background homogenization on the bandpass filtered image to obtain a background homogenized image; perform contrast adjustment on the background homogenized image to obtain a contrast-enhanced image; for each pixel in the contrast-enhanced image, compare the brightness value of the pixel with that of its neighboring pixels, where neighboring pixels are other pixels located within a preset range centered on the pixel; when the brightness value of the pixel is greater than that of its neighboring pixels and is greater than a preset brightness value, the pixel is determined as a candidate molecule position; perform Gaussian point diffusion function fitting on the pixel region to be fitted corresponding to the candidate molecule position to obtain single-molecule coordinates, where the pixel region to be fitted is a pixel region with a preset shape and size extracted from the contrast-enhanced image centered on the candidate molecule position.
[0023] In this application, the original single-molecule localization microscope image can be bandpass filtered to remove high-frequency noise and low-frequency background interference, resulting in a bandpass filtered image. Next, the bandpass filtered image undergoes background homogenization to eliminate background inhomogeneity, yielding a background homogenized image. Subsequently, contrast adjustment processing further enhances the image quality, generating a contrast-enhanced image. This series of preprocessing steps effectively reduces the influence of various interference factors during imaging, achieving a standardized image preprocessing workflow. For each pixel in the contrast-enhanced image, adjacent pixels centered on that pixel and within a preset range are first identified. Then, the brightness values of that pixel are compared with those of its adjacent pixels. When the brightness value of the pixel is greater than that of its adjacent pixels and also greater than a preset brightness value, the pixel is determined as a candidate molecule location. Next, taking the candidate molecule position as the center, a pixel region with a preset shape and size is cropped from the contrast-enhanced image. The Gaussian point diffusion function is then applied to the pixel region to be fitted corresponding to the candidate molecule position to accurately calculate the single molecule coordinates. The entire detection process requires no manual intervention, ensuring high precision and consistency and improving the accuracy of single molecule localization. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1a This is a flowchart illustrating a method for processing single-molecule localization microscope images provided in an embodiment of this application;
[0026] Figure 1b These are comparison images of bandpass filtering provided in the embodiments of this application. The left image is the original single-molecule localization microscope image, and the right image is the bandpass filtered image.
[0027] Figure 1c This is a schematic diagram of the interface for adjusting filter parameters provided in an embodiment of this application;
[0028] Figure 1d This is a schematic diagram of the background subtraction plugin provided in an embodiment of this application;
[0029] Figure 1e This is a schematic diagram of the background subtraction effect provided in the embodiments of this application. The left image is a bandpass filtered image, and the right image is a background homogenized image.
[0030] Figure 1f This is an interface diagram of manually adjusting contrast and brightness in the Fiji software provided in this application embodiment;
[0031] Figure 1g These are the effect diagrams of contrast optimization provided in the embodiments of this application. The left image is the background uniformization image, and the right image is the contrast enhancement image.
[0032] Figure 1h This is a schematic diagram of the ThunderSTORM plugin configuration for sub-pixel single-molecule localization provided in an embodiment of this application;
[0033] Figure 1i These are comparison images of the positioning effects provided in the embodiments of this application. The left image is a traditional wide-field PALM image, and the right image is an image processed by ThunderSTORM.
[0034] Figure 1j This is an enlarged schematic diagram of the ThunderSTORM output results provided in the embodiments of this application;
[0035] Figure 1k This is a schematic diagram of the TrackMate plugin used for single-molecule tracking and verification based on Fiji image processing provided in an embodiment of this application;
[0036] Figure 2This is a schematic diagram of the structure of a device for processing single-molecule localization microscope images provided in an embodiment of this application; Detailed Implementation
[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0038] This application provides a method and apparatus for processing single-molecule localization microscope images.
[0039] Specifically, the device for processing single-molecule localization microscope images can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer (PC); the server can be a single server or a server cluster consisting of multiple servers.
[0040] In some embodiments, the apparatus for processing single-molecule localization microscope images can also be integrated into multiple electronic devices. For example, the apparatus for processing single-molecule localization microscope images can be integrated into multiple servers, and the method for processing single-molecule localization microscope images of this application can be implemented by multiple servers.
[0041] In some embodiments, the server may also be implemented as a terminal.
[0042] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.
[0043] In this embodiment, a method for processing single-molecule localization microscope images is provided, such as... Figure 1a As shown, the specific process of this method for processing single-molecule localization microscope images can be as follows:
[0044] 101. Bandpass filtering is performed on the original single-molecule localization microscope image to obtain a bandpass filtered image.
[0045] Raw single-molecule localization microscopy images refer to unprocessed images of single-molecule samples captured directly by a microscope. These images contain various optical information about the single molecule, such as brightness and position, but may also contain background noise and other interfering factors. In specific experimental environments, raw single-molecule localization microscopy images capture the distribution of fluorescently labeled single molecules at a specific time point using a high-sensitivity camera. These images typically have high resolution, displaying details at the single-molecule level, but are also limited by the imaging system itself, such as optical distortion and noise. In practice, raw single-molecule localization microscopy images are acquired using electron-multiplying charge-coupled devices (EMCCD) or scientific-grade CMOS (sCMOS) cameras. These devices can capture weak fluorescence signals emitted by single molecules under low-light conditions, and the resulting images form the basis for subsequent processing and analysis.
[0046] A bandpass filtered image is a raw single-molecule localization microscope image that has been processed by bandpass filtering. The purpose is to remove low-frequency background structures and high-frequency noise from the image while retaining information in the intermediate frequencies, thereby enhancing the visibility of the target object in the image.
[0047] In some embodiments, in order to remove low-frequency background structures and suppress high-frequency noise, the original single-molecule localization microscope image is bandpass filtered to obtain a bandpass filtered image, including:
[0048] Based on the first and second filtering parameters in the target bandpass filter, the original single-molecule localization microscope image is bandpass filtered to obtain a bandpass filtered image. The first filtering parameter is used to remove low-frequency background structures, and the second filtering parameter is used to suppress high-frequency noise.
[0049] A target bandpass filter is an image processing tool specifically designed to selectively allow a range of frequencies to pass while suppressing other frequency components, thereby optimizing image quality. The target bandpass filter for single-molecule localization microscopy image processing is customized based on the characteristics of the microscope image. It considers the characteristics of the imaging system and the type of information to be extracted from the image, ensuring that irrelevant information is effectively removed while retaining the crucial single-molecule signal.
[0050] The first filtering parameter is a key parameter used to adjust the bandpass filter, primarily setting the low-frequency cutoff frequency, i.e., determining which low-frequency signals will be filtered out. In single-molecule localization microscopy image processing, the first filtering parameter is typically used to remove low-frequency background structures from the image, such as large-scale uniform or slowly varying background brightness, which may mask the weak signals of a single molecule.
[0051] The second filter parameter is another important filter adjustment parameter, mainly used to determine the high-frequency cutoff frequency of the bandpass filter, that is, to specify which high-frequency signals will be filtered out. When processing single-molecule localization microscope images, the role of the second filter parameter is to suppress high-frequency noise in the image, which may come from random fluctuations or other non-specific signal sources during the imaging process.
[0052] For example, original single-molecule localization microscope images (such as...) Figure 1b (As shown in the left image) First, the image is processed using a Fast Fourier Transform (FFT) bandpass filter in Fiji image processing software, as follows: Figure 1c As shown, this bandpass filter does not use the default parameters but is specifically optimized for single-molecule localization microscope images: "Filter Large" is the first filtering parameter, set to 40 pixels (for removing low-frequency background structures), and "Filter Small" is the second filtering parameter, set to 2 pixels (for suppressing high-frequency camera noise). After bandpass filtering the original single-molecule localization microscope image, the following is obtained: Figure 1b The right image shows a bandpass filtered image that effectively suppresses background noise while preserving the single-molecule signal.
[0053] This specific 40 / 2 pixel combination represents an optimized balance that simultaneously mitigates two main and distinct noise sources in wide-field single-molecule microscopy (SMLM). Unlike conventional default parameters, this precise parameter pairing achieves optimal noise suppression while preserving the integrity of the molecular signal—a balance not disclosed in existing technologies. The "Filter Large" parameter value of 40 matches the spatial scale of low-frequency background fluorescence and illumination inhomogeneities prevalent in wide-field SMLM, effectively suppressing these structural artifacts without attenuating the signal of the real single molecule. The "Filter Small" parameter value of 2 pixels is equally crucial: this value is precisely set slightly smaller than the typical full width at half maximum (FWHM) of the diffraction-limited spot, thus strongly suppressing high-frequency camera readout noise and photon shot noise while avoiding distortion of the basic shape of the point spread function (PSF).
[0054] 102. Perform background homogenization processing on the bandpass filtered image to obtain a background homogenized image.
[0055] Background homogenization refers to processing the original image to make the brightness or intensity distribution of the background part (i.e., non-target signal area) in the image uniform.
[0056] In some embodiments, weaker single-molecule fluorescence signals in the original single-molecule localization microscope image are mapped to a higher grayscale range, significantly improving their contrast with the background. This ensures that subsequent detection algorithms can reliably capture these signals and reduce omissions. Background homogenization processing is then performed on the bandpass-filtered image to obtain a background-homogenized image, including:
[0057] The pixel values of each pixel in the bandpass filtered image are mapped to a preset grayscale range to obtain a mapped image. The preset grayscale range is larger than the original grayscale range of the pixels in the bandpass filtered image.
[0058] The mapped image is then subjected to background homogenization processing to obtain a background homogenized image.
[0059] The preset grayscale range is a pre-set grayscale range used to guide the subsequent grayscale transformation process and ensure that key information in the image is displayed optimally.
[0060] Image mapping refers to the process of redistributing the gray values of a bandpass filtered image to a new gray range in order to improve image quality or highlight certain features.
[0061] In some embodiments, in order to improve the quality of a bandpass filtered image, the pixel values of each pixel in the bandpass filtered image are mapped to a preset grayscale range to obtain a mapped image, including:
[0062] The first pixel difference is obtained by determining the difference between the maximum and minimum gray values in the bandpass filtered image;
[0063] For each pixel in the bandpass filtered image, the difference between the pixel's gray value and the minimum gray value is determined to obtain the second pixel difference;
[0064] Divide the second pixel difference by the first pixel difference to obtain the normalization coefficient;
[0065] Multiply the normalization coefficient by the maximum value of the preset grayscale range to obtain the mapped pixel value;
[0066] The mapped image is obtained based on the mapped pixel values of all pixels.
[0067] It is understandable that the grayscale mapping process is equivalent to the "autoscale" operation in the field of image processing. It linearly stretches the original grayscale distribution of the image to a preset display range (such as 0–255), maximizing the use of dynamic range without changing the relative relationship of single-molecule signals, and significantly enhancing the visibility of weak fluorescence features.
[0068] Automatic scaling is an image processing technique used to adjust the brightness and contrast of an image to best display key information within it.
[0069] In this process, the original grayscale values of the image are reassigned to a new, wider grayscale range, such as from 0 to 255. The purpose of this is to fully utilize the available grayscale levels and improve the overall image quality. For example, in a bandpass-filtered image, by determining the difference between the maximum and minimum grayscale values and normalizing based on the difference between the grayscale value of each pixel and the minimum grayscale value, the grayscale values of all pixels are ultimately mapped to the new grayscale range, thereby improving the visual effect of the image. By automatically scaling the bandpass-filtered image, even weaker single-molecule fluorescence signals can be mapped to a higher grayscale value range, thus creating a clear contrast with the background. Specifically, this process ensures that even weak fluorescence points that were originally located at lower grayscale levels are appropriately magnified and highlighted by calculating the first pixel difference, the second pixel difference, and the normalization coefficient, making them easier for detection algorithms to identify and capture.
[0070] In some embodiments, in order to remove morphological background and homogenize the image background, a background homogenization process is performed on the bandpass filtered image to obtain a background homogenized image, including:
[0071] The rolling ball algorithm is used to predict the background of the bandpass filtered image, and the estimated background value corresponding to each pixel in the bandpass filtered image is obtained. The radius of the sphere used in the rolling ball algorithm is selected according to the resolution of the microscope image and the expected background size to be subtracted.
[0072] For each pixel in the bandpass filtered image, the pixel value is subtracted from its corresponding background estimate to obtain a background-normalized pixel, and a background-normalized image is obtained based on each background-normalized pixel.
[0073] Understandably, the rolling ball algorithm estimates the background value of each pixel by simulating a "sphere" rolling on the image surface. During the rolling process, the sphere contacts the image surface and records the height (grayscale value) of the contact point, which is then used as the background estimate. For a bandpass filtered image, assuming a sphere starts "rolling" from the current pixel, recording the grayscale values of all contact points, the background estimate for that pixel is obtained, representing the minimum height the rolling ball can reach at that position. Background homogenization aims to eliminate background gradients or irregular changes in the image caused by uneven illumination, optical system defects, etc., making the foreground signal (such as single-molecule fluorescent dots) in the bandpass filtered image clearer. Specifically, after the rolling ball algorithm estimates the background for each pixel, a background estimate image of the same size as the bandpass filtered image is obtained. This background estimate image reflects the local background intensity level at various locations in the image, typically including background changes caused by uneven illumination, optical system defects, etc. The background homogenized pixel value is obtained by subtracting the corresponding background estimate from the original grayscale value of each pixel. This step helps to highlight details in a bandpass filtered image, especially those obscured by an uneven background. Details that were previously difficult to observe due to inconsistent or slow changes in background brightness are now more apparent, and a uniform background image should reveal clearer foreground objects.
[0074] When performing background homogenization on a bandpass-filtered image, the bandpass-filtered image is then processed using Fiji's Subtract Background plugin. The key point is that... Figure 1d As shown, a rolling ball algorithm with a radius of 50 pixels was applied. This specific parameter is optimized for bandpass filtered images, allowing for the removal of residual non-uniform illumination without weakening the single-molecule signal. The effectiveness of this step is reflected in... Figure 1e In: with the input image ( Figure 1e Compared to the left image, the output image ( Figure 1e The background of the left image is more uniform, and the signal-to-noise ratio is further improved.
[0075] The effectiveness of the rolling ball algorithm hinges on the parameter of the sphere radius. The precise value of 50 pixels is not obvious because it is specifically optimized for the preprocessed image output from step 101. Since the 40-pixel bandpass filter in step 101 has already suppressed large-scale fluctuations, a sphere radius of 50 pixels can accurately locate and handle the remaining medium-scale background fluctuations.
[0076] The parameter of the sphere's radius, combined with the preceding bandpass filtered image, achieves an unconventional balancing effect not mentioned in existing techniques: if the radius is significantly increased, it will fail to respond to these residual fluctuations; if the radius is too small, it will incorrectly include and subtract high-intensity pixel regions of single molecules from the background model. This carefully considered parameter selection enables the method to achieve robust performance under various imaging conditions.
[0077] 103. Perform contrast adjustment processing on the background homogenized image to obtain a contrast-enhanced image.
[0078] Among them, contrast-enhanced images are processed by adjusting the contrast of a background homogenized image to increase the difference between different brightness levels in the contrast-enhanced image, thereby making the details in the contrast-enhanced image more clearly visible.
[0079] In some embodiments, in order to achieve precise image contrast adjustment, a background homogenization image is subjected to contrast adjustment processing to obtain a contrast-enhanced image, including:
[0080] In response to a contrast adjustment operation on a background homogenization image, an image adjustment page is displayed, which includes a grayscale histogram of the background homogenization image.
[0081] Based on the distribution information of the gray-level histogram, the contrast of the background homogenized image is adjusted to obtain a contrast-enhanced image.
[0082] The image adjustment page is a user interface component used to display and adjust the visual properties of an image (such as contrast and brightness). This page may contain a preview area of the original or processed image, as well as a series of control options and charts to help users intuitively understand and adjust image parameters.
[0083] A grayscale histogram is a statistical distribution chart of the number of pixels at each gray level in an image. It shows the frequency of each gray level in the image, with the horizontal axis representing gray values (from 0 to the maximum gray value, such as 255) and the vertical axis representing the number or proportion of pixels corresponding to the gray value.
[0084] Distribution information refers to the statistical features extracted from the grayscale histogram, which can be used to guide image contrast adjustment.
[0085] It is understandable that, such as Figure 1f As shown, the contrast and brightness adjustments performed through the Fiji interface are not merely for visual enhancement, but a crucial image calibration step. This operation effectively amplifies the grayscale difference between the single-molecule fluorescence signal and the homogenized background by appropriately scaling the grayscale histogram, thereby maximizing their numerical separability. Figure 1gThe comparison results show that the processed image provides the optimal input for subsequent localization algorithms—it has higher contrast and signal-to-noise ratio, which is an important prerequisite for achieving high-fidelity, sub-pixel-level single-molecule localization.
[0086] 104. For each pixel in the contrast-enhanced image, compare the brightness value of the pixel with that of its neighboring pixels. The neighboring pixels are other pixels located within a preset range of the pixel, centered on the pixel.
[0087] The preset range specifies a rectangular or circular region centered on the current pixel, where all pixels within the region are considered neighboring pixels. Adjusting this range controls the scale of local contrast analysis, thus affecting the sensitivity and accuracy of the final detection results. For example, the preset range can be a 4-neighborhood, an 8-neighborhood, or a larger window, etc.
[0088] In some embodiments, to further optimize the image, for each pixel in the contrast-enhanced image, a brightness value comparison process is performed between the pixel and its neighboring pixels, including:
[0089] The brightness distribution of the contrast-enhanced image is matched with a preset feature template to obtain matching information. The preset feature template is similar in spatial shape to the Gaussian distribution light spot presented by a single molecule.
[0090] When the matching information indicates that there is image content in the contrast-enhanced image that does not match the preset feature template, the image content is suppressed to obtain an updated contrast-enhanced image;
[0091] For each pixel in the updated contrast-enhanced image, the brightness value of the pixel is compared with that of its neighboring pixels.
[0092] The preset feature template simulates a Gaussian-distributed light spot formed by a single molecule during imaging. Such a template helps identify portions of the image that may represent single-molecule signals and can be used to enhance the contrast of these signals relative to background noise.
[0093] For example, a preset feature template can be implemented using B-spline wavelet filtering, with an order of 5 and a scale of 2. This means the template uses a fifth-order B-spline function for wavelet transform, while selecting a scale parameter suitable for capturing target features. This configuration can help accurately identify and match subtle structures in images, especially those with Gaussian-like distribution characteristics. Image content that does not match the preset feature template.
[0094] Image content that does not match the preset feature template refers to the parts of the contrast-enhanced image that do not match the preset feature template. This includes areas that do not conform to the expected shape, structure, or similar brightness distribution characteristics.
[0095] 105. When the brightness value of a pixel is greater than the brightness value of its adjacent pixels, and the brightness value of the pixel is greater than a preset brightness value, the pixel is determined as a candidate molecule position.
[0096] The preset brightness value serves as a fixed brightness standard, used to compare the actual brightness values of each pixel in the image. Only when a pixel's brightness value exceeds this preset brightness value, and the pixel is brighter than its neighboring pixels, can it be selected as a candidate molecule location. This helps ensure that the selected pixels are not only the brightest in their local area but also reach a certain brightness level, thereby improving the reliability and accuracy of the detection results.
[0097] Candidate molecule position refers to the position of a pixel whose brightness value is higher than all pixels in its defined neighborhood (e.g., within an 8-neighborhood) and also exceeds a preset brightness value.
[0098] Understandably, when the preset range is 8 neighborhoods, for any pixel P in the contrast-enhanced image, its neighboring pixels include the eight surrounding pixels directly adjacent to P in the horizontal, vertical, and diagonal directions, forming a 3×3 pixel window centered on P. During brightness value comparison processing, it is determined one by one whether the brightness value of pixel P is strictly greater than the brightness values of these eight neighboring pixels, and whether the brightness value of P exceeds a preset brightness value. Only when both conditions are met simultaneously is pixel P marked as a candidate molecule position.
[0099] In some embodiments, when the brightness value of a pixel is not greater than the brightness value of its neighboring pixels, or when the brightness value of a pixel is not greater than a preset brightness value, the pixel is determined not to be a candidate molecule position.
[0100] 106. Perform Gaussian point diffusion function fitting on the pixel region to be fitted corresponding to the candidate molecule position to obtain single molecule coordinates. The pixel region to be fitted is a pixel region with a preset shape and size that is cropped from the contrast-enhanced image with the candidate molecule position as the center.
[0101] The preset shape refers to a geometric shape that can effectively cover the brightness distribution characteristics around the candidate molecule location, such as a square, rectangle, or circle.
[0102] The preset size refers to the size of the pixel region selected during image processing, which determines the amount of data to be analyzed around the candidate molecule location.
[0103] In some embodiments, a Gaussian point diffusion function is applied to the pixel region to be fitted corresponding to the candidate molecule position to obtain single-molecule coordinates. The pixel region to be fitted is a pixel region with a preset shape and size, centered on the candidate molecule position, cropped from the contrast-enhanced image, including:
[0104] Centered on the candidate molecule position, a pixel region of a preset shape and size is extracted from the contrast-enhanced image as the pixel region to be fitted.
[0105] To fit the pixel intensity distribution in the pixel region, a Gaussian point spread function is used for nonlinear iterative fitting. The parameters of the Gaussian point spread function include at least the center position, peak intensity, and background offset. The fitting process aims to minimize the residual between the predicted intensity of the Gaussian point spread function and the actual pixel intensity until convergence.
[0106] The coordinates of the center position of the Gaussian point diffusion function after fitting convergence are used as the single-molecule coordinates.
[0107] In this process, a Gaussian point spread function is used to perform nonlinear iterative fitting of the pixel intensity distribution in the pixel region to be fitted. The model used is as follows:
[0108] ,in, It is the position of the pixel in the pixel region to be fitted. , ) represents the center position of the Gaussian distribution, A represents the peak intensity (related to the number of photons emitted by the fluorescent molecule), B represents the background shift (estimates the local background noise level), and σ represents the Gaussian standard deviation (which can be fixed as the theoretical PSF width of the optical system or used as a fitting parameter).
[0109] Following step 103, a specific implementation could involve using the ThunderSTORM plugin to analyze the contrast-enhanced image. This plugin is configured with a specific set of parameters that work in conjunction with the preprocessing steps. For example... Figure 1h As shown in the details, the configuration includes: wavelet filtering (B-spline): order 5, scale 2.0; local maximum detection: using 8-neighborhood connectivity; ensemble Gaussian PSF fitting: radius 5 pixels, initial standard deviation 1.6 pixels. Thus, the following can be obtained: Figure 1i The right image shows the precise sub-pixel positioning results and Figure 1j The magnified view has a resolution that breaks through... Figure 1i The diffraction limit of the conventional image in the left figure.
[0110] The ThunderSTORM plugin configuration and the preceding preprocessing workflow achieve synergistic optimization. The specific parameter set used in this step—a 5th-order B-spline wavelet filter, a scale of 2.0, 8-neighbor connectivity, a 5-pixel point spread function (PSF) fitting radius, and an initial standard deviation of 1.6 pixels—is non-obvious, and these parameters are obtained through synergistic optimization of the preprocessed input image.
[0111] The fifth-order B-spline was chosen to match the enhanced single-molecule signal profile after preprocessing, achieving optimal smoothing while avoiding excessive blurring. The 5-pixel PSF fitting radius was carefully selected—preliminary steps have already created a consistent and clear spatial range for the single-molecule spot, thus this radius accurately covers the core region of the spot. Applying this 5-pixel fitting radius directly to the original data (which includes distorted and noisy spots) would significantly degrade the fitting performance.
[0112] It is evident that these specific positioning parameters possess unique effectiveness because they are designed for the output of standardized preprocessing steps, ultimately enabling superior sub-pixel-level positioning accuracy.
[0113] In some embodiments, in order to track the trajectory of a single molecule, before bandpass filtering the original microscope image to obtain a bandpass-filtered image, the method further includes:
[0114] Acquire multiple frames of raw single-molecule localization microscope images acquired in succession;
[0115] After fitting the single-molecule coordinates to the Gaussian point diffusion function of the pixel region corresponding to the candidate molecule position, the process also includes:
[0116] Based on the single-molecule coordinates obtained from each frame of the original single-molecule positioning microscope image, the trajectory of the single molecule is generated.
[0117] This can be understood as follows: after obtaining the individual molecule coordinates corresponding to each of the multiple frames of raw single-molecule localization microscope images, the TrackMate plugin is used for tracking processing, with initial settings as follows. Figure 1k As shown. Select the Gaussian Laplace (LoG) detector and configure it to detect spots (typically 4-6 pixels) with diameters matching the localized molecules. Applying this detector to the pre-location data output by ThunderSTORM generates highly reliable trajectories. Figure 1cThe visual comparison results show that, compared with the original single-molecule localization microscope image, the tracking integrity and reliability of the processed contrast-enhanced image are significantly improved.
[0118] The selection and configuration of the Laplacian of Gaussian (LoG) detector in the TrackMate plugin has a non-obvious correlation with the output of the entire preprocessing and localization process. Through a systematic evaluation of existing detection algorithms (including the Difference of Gaussian (DoG) detector, the Median detector, and the Thresholding detector), the LoG detector was ultimately selected. The core reason is that it has higher accuracy in identifying "small, well-separated light spots"—which are the specific signal characteristics generated after the preliminary steps of this invention.
[0119] Compared to other detectors: the Difference of Gaussian (DoG) detector is faster but less accurate, thresholding is only effective for high-contrast spots, while the LoG detector provides the precise spatial localization required for robust single-molecule tracking. The specific 5-pixel blob diameter was carefully set to perfectly match the spatial distribution of the localization coordinates in step 105 (step 1.4 uses a uniform fitting radius of 5 pixels). This design creates a unified physical model for the entire processing flow: the tracking algorithm's configuration target is to associate with point light sources that have been identified and optimized through specific, consistent spatial contours.
[0120] The output of this step is validated molecular trajectory data in CSV format, containing subpixel-level coordinate information. This result proves that the original microscopic image has been successfully converted into quantifiable and traceable single-molecule data. This precise parameter matching and detector selection is a key step in demonstrating non-obviousness in this integrated workflow, which can avoid the generation of tracking artifacts and ensure the generation of repeatable molecular trajectories under different experimental conditions.
[0121] Understandably, by comparing and analyzing the tracking performance of raw photoactivated localization microscopy (PALM) images with processed PALM images, the technical superiority and non-obvious effects of this integrated workflow are quantitatively verified. The results shown in Table 1 confirm that this invention has practical value and represents a technical improvement over existing technologies.
[0122]
[0123] Table 1 Quantitative Comparison of Single-Molecular Tracking Performance
[0124] The data in Table 1 demonstrate that the continuous workflow claimed in this invention achieves significant synergistic improvements—effects that cannot be achieved by performing individual steps or only on the raw data. The approximately 6-fold improvement in quality score, along with substantial improvements in positional consistency and sub-pixel accuracy, are direct results of this specific integration sequence of steps 101 to 106. Therefore, this invention transforms noisy raw SMLM data into a quantitatively reliable dataset, addressing the unresolved need for standardized optimization processes outlined in the background.
[0125] Those skilled in the art, referring to the accompanying drawings and following the above-described sequence of steps and parameter settings, can reliably reproduce this invention to achieve the aforementioned performance improvement. Modifications and equivalent substitutions made without departing from the inventive concept are considered to fall within the scope of protection of this invention.
[0126] The core innovation of this invention lies in the specific and interdependent execution order of the above stages: the output of each stage provides input for the next stage with significantly improved quality, ultimately achieving a synergistic enhancement of data quality—an enhancement that cannot be achieved by executing each stage individually.
[0127] Compared with existing methods, the beneficial effects of the present invention are as follows:
[0128] 1) Repeatable standardized processes minimize the differences caused by user operations;
[0129] 2) The signal-to-noise ratio is significantly improved, enabling reliable detection of masked single molecules in the original data;
[0130] 3) Better positioning accuracy and positional consistency directly improve the effective resolution of super-resolution image reconstruction.
[0131] To better implement the above methods, this application also provides an apparatus for processing single-molecule localization microscope images. This apparatus can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster consisting of multiple servers.
[0132] For example, in this embodiment, a device for processing single-molecule localization microscope images will be specifically integrated into an electronic device to illustrate the method of this application embodiment in detail.
[0133] For example, such as Figure 2As shown, the highway high slope avalanche artificial control effect evaluation device based on multi-source data may include a filtering unit 201, a homogenization unit 202, an adjustment unit 203, a comparison unit 204, a candidate determination unit 205, and a coordinate determination unit 206, as follows:
[0134] (a) Filtering unit 201.
[0135] The filtering unit 201 is used to perform bandpass filtering on the original single-molecule localization microscope image to obtain a bandpass filtered image.
[0136] In some embodiments, bandpass filtering is performed on the original single-molecule localization microscope image to obtain a bandpass filtered image, including:
[0137] Based on the first and second filtering parameters in the target bandpass filter, the original single-molecule localization microscope image is bandpass filtered to obtain a bandpass filtered image. The first filtering parameter is used to remove low-frequency background structures, and the second filtering parameter is used to suppress high-frequency noise.
[0138] (ii) Homogenization unit 202.
[0139] The homogenization unit 202 is used to perform background homogenization processing on the bandpass filtered image to obtain a background homogenized image.
[0140] In some embodiments, background homogenization processing is performed on the bandpass filtered image to obtain a background homogenized image, including:
[0141] The pixel values of each pixel in the bandpass filtered image are mapped to a preset grayscale range to obtain a mapped image. The preset grayscale range is larger than the original grayscale range of the pixels in the bandpass filtered image.
[0142] The mapped image is then subjected to background homogenization processing to obtain a background homogenized image.
[0143] In some embodiments, mapping the pixel values of each pixel in the bandpass filtered image to a preset grayscale range to obtain a mapped image includes:
[0144] The first pixel difference is obtained by determining the difference between the maximum and minimum gray values in the bandpass filtered image;
[0145] For each pixel in the bandpass filtered image, the difference between the pixel's gray value and the minimum gray value is determined to obtain the second pixel difference;
[0146] Divide the second pixel difference by the first pixel difference to obtain the normalization coefficient;
[0147] Multiply the normalization coefficient by the maximum value of the preset grayscale range to obtain the mapped pixel value;
[0148] The mapped image is obtained based on the mapped pixel values of all pixels.
[0149] In some embodiments, background homogenization processing is performed on the bandpass filtered image to obtain a background homogenized image, including:
[0150] The rolling ball algorithm is used to predict the background of the bandpass filtered image, and the estimated background value corresponding to each pixel in the bandpass filtered image is obtained. The radius of the sphere used in the rolling ball algorithm is selected according to the resolution of the microscope image and the expected background size to be subtracted.
[0151] For each pixel in the bandpass filtered image, the pixel value is subtracted from its corresponding background estimate to obtain a background-normalized pixel, and a background-normalized image is obtained based on each background-normalized pixel.
[0152] In some embodiments, the background homogenization image is subjected to contrast adjustment processing to obtain a contrast-enhanced image, including:
[0153] In response to a contrast adjustment operation on a background homogenization image, an image adjustment page is displayed, which includes a grayscale histogram of the background homogenization image.
[0154] Based on the distribution information of the gray-level histogram, the contrast of the background homogenized image is adjusted to obtain a contrast-enhanced image.
[0155] (III) Adjustment Unit 203.
[0156] The adjustment unit 203 is used to perform contrast adjustment processing on the background homogenization image to obtain a contrast-enhanced image.
[0157] (iv) Comparison Unit 204.
[0158] The comparison unit 204 is used to compare the brightness value of each pixel in the contrast-enhanced image with that of its neighboring pixels. The neighboring pixels are other pixels located within a preset range of the pixel, with the pixel as the center.
[0159] In some embodiments, for each pixel in the contrast-enhanced image, a brightness value comparison process is performed between the pixel and its neighboring pixels, including:
[0160] The brightness distribution of the contrast-enhanced image is matched with a preset feature template to obtain matching information. The preset feature template is similar in spatial shape to the Gaussian distribution light spot presented by a single molecule.
[0161] When the matching information indicates that there is image content in the contrast-enhanced image that does not match the preset feature template, the image content is suppressed to obtain an updated contrast-enhanced image;
[0162] For each pixel in the updated contrast-enhanced image, the brightness value of the pixel is compared with that of its neighboring pixels.
[0163] (v) Candidate determination unit 205.
[0164] The candidate determination unit 205 is used to determine a pixel as a candidate molecule position when the brightness value of a pixel is greater than the brightness value of its adjacent pixels and the brightness value of the pixel is greater than a preset brightness value.
[0165] (vi) Coordinate determination unit 206.
[0166] The coordinate determination unit 206 is used to fit the Gaussian point diffusion function to the pixel region to be fitted corresponding to the candidate molecule position to obtain the single molecule coordinates. The pixel region to be fitted is a pixel region with a preset shape and size that is cropped from the contrast-enhanced image with the candidate molecule position as the center.
[0167] In some embodiments, a Gaussian point diffusion function is applied to the pixel region to be fitted corresponding to the candidate molecule position to obtain single-molecule coordinates. The pixel region to be fitted is a pixel region with a preset shape and size, centered on the candidate molecule position, cropped from the contrast-enhanced image, including:
[0168] Centered on the candidate molecule position, a pixel region of a preset shape and size is extracted from the contrast-enhanced image as the pixel region to be fitted.
[0169] To fit the pixel intensity distribution in the pixel region, a Gaussian point spread function is used for nonlinear iterative fitting. The parameters of the Gaussian point spread function include at least the center position, peak intensity, and background offset. The fitting process aims to minimize the residual between the predicted intensity of the Gaussian point spread function and the actual pixel intensity until convergence.
[0170] The coordinates of the center position of the Gaussian point diffusion function after fitting convergence are used as the single-molecule coordinates.
[0171] In some embodiments, before bandpass filtering the original microscope image to obtain a bandpass filtered image, the method further includes:
[0172] Acquire multiple frames of raw single-molecule localization microscope images acquired in succession;
[0173] After fitting the single-molecule coordinates to the Gaussian point diffusion function of the pixel region corresponding to the candidate molecule position, the process also includes:
[0174] Based on the single-molecule coordinates obtained from each frame of the original single-molecule positioning microscope image, the trajectory of the single molecule is generated.
[0175] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.
[0176] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the methods for processing single-molecule localization microscope images provided in embodiments of this application.
[0177] Since the instructions stored in the storage medium can execute the steps of any of the methods for processing single-molecule localization microscope images provided in the embodiments of this application, the beneficial effects that any of the methods for processing single-molecule localization microscope images provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0178] The above provides a detailed description of a method and apparatus for processing single-molecule localization microscope images provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for processing single-molecule localization microscope images, characterized in that, include: The original single-molecule localization microscope image was subjected to bandpass filtering to obtain a bandpass filtered image; The bandpass filtered image is subjected to background homogenization processing to obtain a background homogenized image; The background homogenization image is subjected to contrast adjustment processing to obtain a contrast-enhanced image; For each pixel in the contrast-enhanced image, the brightness value of the pixel is compared with that of its neighboring pixels, where the neighboring pixels are other pixels located within a preset range of the pixel, with the pixel as the center. When the brightness value of the pixel is greater than the brightness value of the adjacent pixel, and the brightness value of the pixel is greater than a preset brightness value, the pixel is determined to be a candidate molecule position. Gaussian point diffusion function is applied to the pixel region to be fitted corresponding to the candidate molecule position to obtain single molecule coordinates. The pixel region to be fitted is a pixel region with a preset shape and size that is cropped from the contrast-enhanced image with the candidate molecule position as the center.
2. The method as described in claim 1, characterized in that, The step of comparing the brightness values of each pixel in the contrast-enhanced image with those of its neighboring pixels includes: The brightness distribution of the contrast-enhanced image is matched with a preset feature template to obtain matching information. The preset feature template is similar in spatial shape to the Gaussian distribution light spot presented by a single molecule. When the matching information indicates that there is image content in the contrast-enhanced image that does not match the preset feature template, the image content is suppressed to obtain an updated contrast-enhanced image; For each pixel in the updated contrast-enhanced image, the brightness value of the pixel is compared with that of its neighboring pixels.
3. The method as described in claim 1, characterized in that, Before performing bandpass filtering on the original microscope image to obtain the bandpass-filtered image, the method further includes: Acquire multiple frames of raw single-molecule localization microscope images acquired in succession; After performing Gaussian point diffusion function fitting on the pixel region corresponding to the candidate molecule position to obtain the single molecule coordinates, the method further includes: Based on the single-molecule coordinates obtained from each frame of the original single-molecule positioning microscope image, the trajectory of the single molecule is generated.
4. The method as described in claim 1, characterized in that, The process of performing bandpass filtering on the original single-molecule localization microscope image to obtain a bandpass filtered image includes: Based on the first and second filtering parameters in the target bandpass filter, the original single-molecule localization microscope image is bandpass filtered to obtain a bandpass filtered image. The first filtering parameter is used to remove low-frequency background structures, and the second filtering parameter is used to suppress high-frequency noise.
5. The method as described in claim 1, characterized in that, The step of performing background homogenization processing on the bandpass filtered image to obtain a background homogenized image includes: The pixel values of each pixel in the bandpass filtered image are mapped to a preset grayscale range to obtain a mapped image, wherein the preset grayscale range is larger than the original grayscale range of the pixels in the bandpass filtered image. The mapped image is then subjected to background homogenization processing to obtain a background homogenized image.
6. The method as described in claim 5, characterized in that, The step of mapping the pixel values of each pixel in the bandpass filtered image to a preset grayscale range to obtain a mapped image includes: The difference between the maximum and minimum gray values in the bandpass filtered image is determined to obtain the first pixel difference; For each pixel in the bandpass filtered image, the difference between the gray value of the pixel and the minimum gray value is determined to obtain the second pixel difference; Divide the second pixel difference by the first pixel difference to obtain the normalization coefficient; Multiply the normalization coefficient by the maximum value of the preset grayscale range to obtain the mapped pixel value of the pixel; The mapped image is obtained based on the mapped pixel values of all pixels.
7. The method as described in claim 1, characterized in that, The step of performing background homogenization processing on the bandpass filtered image to obtain a background homogenized image includes: The rolling ball algorithm is used to predict the background of the bandpass filtered image to obtain the background estimate value corresponding to each pixel in the bandpass filtered image. The radius of the sphere used in the rolling ball algorithm is selected according to the resolution of the microscope image and the expected background size to be subtracted. For each pixel in the bandpass filtered image, the pixel value of the pixel is subtracted from its corresponding background estimate to obtain a background-normalized pixel, and a background-normalized image is obtained based on each background-normalized pixel.
8. The method as described in claim 1, characterized in that, The step of performing contrast adjustment processing on the background homogenization image to obtain a contrast-enhanced image includes: In response to a contrast adjustment operation on the background homogenization image, an image adjustment page is displayed, the image adjustment page including a grayscale histogram of the background homogenization image; Based on the distribution information of the grayscale histogram, the background homogenization image is subjected to contrast adjustment processing to obtain a contrast-enhanced image.
9. The method as described in claim 1, characterized in that, The Gaussian point diffusion function is applied to the pixel region corresponding to the candidate molecule position to obtain single-molecule coordinates. The pixel region to be fitted is a pixel region with a preset shape and size, centered on the candidate molecule position, cropped from the contrast-enhanced image, including: Centered on the candidate molecule position, a pixel region of a preset shape and preset size is extracted from the contrast-enhanced image as the pixel region to be fitted. For the pixel intensity distribution in the pixel region to be fitted, a Gaussian point spread function is used for nonlinear iterative fitting. The parameters of the Gaussian point spread function include at least the center position, peak intensity, and background offset. The fitting process aims to minimize the residual between the intensity predicted by the Gaussian point spread function and the actual pixel intensity until convergence. The coordinates of the center position of the Gaussian point diffusion function after fitting convergence are used as the single-molecule coordinates.
10. An apparatus for processing single-molecule localization microscope images, characterized in that, include: The filtering unit is used to perform bandpass filtering on the original single-molecule localization microscope image to obtain a bandpass filtered image. A uniformization unit is used to perform background uniformization processing on the bandpass filtered image to obtain a background uniformized image; An adjustment unit is used to perform contrast adjustment processing on the background homogenization image to obtain a contrast-enhanced image. The comparison unit is used to compare the brightness value of each pixel in the contrast-enhanced image with that of its neighboring pixels, wherein the neighboring pixels are other pixels located within a preset range of the pixel, with the pixel as the center. The candidate determination unit is used to determine the pixel as a candidate molecule position when the brightness value of the pixel is greater than the brightness value of the adjacent pixel and the brightness value of the pixel is greater than a preset brightness value. The coordinate determination unit is used to perform Gaussian point diffusion function fitting on the pixel region to be fitted corresponding to the candidate molecule position to obtain single molecule coordinates. The pixel region to be fitted is a pixel region with a preset shape and size that is cropped from the contrast-enhanced image with the candidate molecule position as the center.
