An image registration method for multi-channel single-molecule localization

Abstract

This application discloses an image registration method for multi-channel single-molecule localization. The method includes acquiring several multi-channel images of fluorescent beads with different fields of view and converting them into several dual-channel image pairs, wherein the reference image in each dual-channel image pair is the same; for each dual-channel image pair, locating the fluorescent beads of the image pair to obtain a set of control points; determining the image registration parameters of the dual-channel image pair based on the set of control points; and filtering the set of control points based on the image registration parameters and the baseline registration error to obtain a set of target control points; determining the image registration parameters based on the target set of control points for each dual-channel image pair, and registering the multi-channel images of actual biological samples. This application determines the image registration model through multi-channel images and performs registration operations on actual samples, achieving high-precision and efficient image registration. While being applicable to various multi-channel single-molecule localization systems, it significantly reduces economic costs and technical requirements.

Description

Technical Field

[0001] This application relates to the field of optical microscopy imaging, and in particular to an image registration method for multi-channel single-molecule localization. Background Technology

[0002] Single-molecule localization technology has been widely used in the detection of single molecules and the tracking of single particles in biological samples, and has achieved remarkable results in the field of super-resolution microscopy, with resolutions reaching tens of nanometers, far exceeding the 200-300 nanometer resolution limit of traditional optical microscopes. However, despite its remarkable achievements, this technology also faces some inherent challenges. A particularly significant issue in multi-channel single-molecule localization is the precise matching between multi-channel images. Due to human measurement errors during optical path setup, field curvature and chromatic aberration in multicolor imaging channels, and magnification differences in certain situations, multi-channel images often cannot be strictly matched, thus affecting collaborative localization and analysis.

[0003] Furthermore, to improve registration accuracy, image registration is a crucial step in multi-channel single-molecule localization, which typically requires control point-based methods. These control points can generally be obtained directly through multi-channel imaging and single-molecule localization of a reference material. However, existing techniques and methods for image registration and control point acquisition, especially those employing precision displacement stages and nanogrids, present significant technical challenges and costs, limiting their application in certain scenarios.

[0004] Therefore, how to obtain high-precision image registration parameters in a simplified way without using sophisticated and expensive equipment or requiring the system to have a bright field illumination mode is a technical problem that urgently needs to be solved in current multi-channel single-molecule localization technology.

[0005] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0006] The technical problem to be solved by this application is to provide an image registration method for multi-channel single-molecule localization, addressing the shortcomings of existing technologies.

[0007] To address the aforementioned technical problems, a first aspect of this application provides an image registration method for multi-channel single-molecule localization, the method comprising:

[0008] Acquire multi-channel images of fluorescent beads from several different fields of view;

[0009] For each fluorescent bead multi-channel image, the fluorescent bead multi-channel image is converted into several dual-channel image pairs, wherein the reference image in each dual-channel image pair is the same;

[0010] For each dual-channel image pair, the fluorescent beads of the dual-channel image pair are located to obtain a set of control point pairs. Based on the set of control point pairs, the image registration parameters of the dual-channel image pair are determined. Based on the image registration parameters and the baseline registration error, the set of control point pairs is filtered to obtain a target set of control point pairs.

[0011] Image registration parameters are determined based on the target control point pair set of each dual-channel image pair, and the multi-channel images of actual biological samples are registered using the image registration parameters.

[0012] Based on the above technical means, by using randomly distributed fluorescent bead samples as reference materials and acquiring multi-channel images of them from different fields of view, and by using a method of iteratively screening control point pairs based on the reference registration error during feature matching and transformation model parameter estimation, multi-channel image registration of actual biological samples can be achieved, which enhances the matching accuracy between multi-channel images and reduces technical difficulty and cost.

[0013] In one implementation, the image registration method for multi-channel single-molecule localization, wherein locating the fluorescent beads of the dual-channel image pair to obtain a set of control point pairs specifically includes:

[0014] Gaussian fitting is used to locate the fluorescent beads of the reference image in the dual-channel image to obtain a preset number of reference control points, and fluorescent beads of the image to be registered are used to obtain a preset number of registration control points.

[0015] The control point pair set of the dual-channel image pair is formed based on a preset number of reference control points and a preset number of registration control points.

[0016] Based on the above technical means, the fluorescent beads are located by Gaussian fitting method to obtain high-precision control point position information, which meets the requirements of high-precision image registration. Then, more accurate image registration is achieved by using a set of control point pairs formed by a preset number of reference control points and registration control points. That is, the images in the same set of dual-channel image pairs can be mapped to the same coordinate system, thereby realizing multi-channel single-molecule localization with high precision, stability and reliability.

[0017] In one implementation, the image registration method for multi-channel single-molecule localization, wherein the process of determining the image registration parameters specifically includes:

[0018] The control point pair set is transformed using the local weighted average method to obtain the image registration parameters.

[0019] Based on the above technical means, the optimal estimated parameters are obtained by using control points to perform second-order polynomial fitting based on local weighted average of position information. This method is better suited to situations where there is local nonlinear distortion between different channels and can be applied to complex multi-channel single-molecule positioning systems.

[0020] In one implementation, the image registration method for multi-channel single-molecule localization includes a transformation function in the image registration parameters as follows:

[0021]

[0022]

[0023]

[0024]

[0025] Among them, P i (x,y) represents the fitting polynomial for the i-th control point, and m represents the polynomial P. i The order of (x,y), W i (R) represents the weight factor of the i-th control point, R represents the distance factor, and R n Indicates the distance from the control point (x) within the local control area. i ,y i The furthest distance from other control points, a jk This represents the registration transformation parameters, and N represents the number of control points within the local area.

[0026] Based on the above technical means, flexible and precise control of image registration is achieved through polynomial approximation and multivariate setting. Furthermore, by considering the weight and distance information of control points, the accuracy of image registration and its adaptability to local image characteristics are increased. Moreover, by introducing local control range and registration transformation parameters, the model can be finely adjusted for specific registration tasks.

[0027] In one implementation, the image registration method for multi-channel single-molecule localization, wherein the step of filtering the control point pair set based on the image registration parameters and the baseline registration error to obtain the target control point pair set specifically includes:

[0028] The images to be registered in the dual-channel image pair are transformed based on the image registration parameters to obtain the registered images;

[0029] The registered image is located to obtain registration control points, and the reference registration error between the registration control points and the reference control points in the reference image is calculated respectively.

[0030] Control point pairs with a reference registration error greater than the preset registration error are removed to obtain the filtered set of control point pairs.

[0031] Based on the above technical means, control points are accurately obtained through image transformation and positioning, thereby improving the image registration accuracy. Through calculation and screening processes, only control point pairs with small errors are retained, while inaccurate data is excluded, thereby optimizing the image registration effect and further improving the accuracy and reliability of multi-channel single-molecule localization.

[0032] In one implementation, the image registration method for multi-channel single-molecule localization, after removing control point pairs with reference registration errors greater than a preset registration error to obtain a filtered set of control point pairs, further includes:

[0033] The step of filtering the control point pair set based on the image registration parameters and the preset registration error is repeated until there are no control point pairs with a reference registration error greater than the preset registration error.

[0034] Based on the above technical means, control point pairs are screened through repeated iterations to remove those with large registration errors, thereby continuously optimizing and improving the accuracy of image registration parameters. The above technical means are adaptable to various image registration problems with nonlinear characteristics and have strong robustness in solving registration errors caused by various reasons. In addition, accurate registration of multi-channel single-molecule localization is achieved under relatively simple conditions, improving the accuracy and reliability of multi-channel single-molecule localization.

[0035] In one implementation, the image registration method for multi-channel single-molecule localization, wherein the expression for the reference registration error is:

[0036]

[0037] Among them, (X) i ,Y i (X′) represents the coordinates of the i-th control point in the reference image of the two-channel image pair. i ,Y′ i ) represents (X′ i ,Y′ i The coordinates of the image in the registration image.

[0038] Based on the above technical means, a quantifiable registration error evaluation standard is provided, which helps to evaluate and compare the accuracy and effect of different image registration parameters; by providing accurate registration error evaluation, the accuracy of subsequent image co-localization and analysis is greatly improved.

[0039] A second aspect of this application provides an image registration device for multi-channel single-molecule localization, the device comprising:

[0040] The positioning module is used to acquire multi-channel images of fluorescent beads from several different fields of view;

[0041] The conversion module converts each multi-channel image of a fluorescent bead into several pairs of dual-channel images, wherein the reference images in each pair of dual-channel images are the same.

[0042] The filtering module locates the fluorescent beads of each dual-channel image pair to obtain a set of control point pairs, determines the image registration parameters of the dual-channel image pair based on the set of control point pairs, and filters the set of control point pairs based on the image registration parameters and the baseline registration error to obtain a target set of control point pairs.

[0043] The determination module determines image registration parameters based on the target control point pair set of each dual-channel image pair, and registers the multi-channel images of the actual biological sample using the image registration parameters.

[0044] In one implementation, a third aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the image registration method for multi-channel single-molecule localization as described above.

[0045] A fourth aspect of this application provides a terminal device, which includes: a processor and a memory;

[0046] The memory stores a computer-readable program that can be executed by the processor;

[0047] When the processor executes the computer-readable program, it implements the steps in the image registration method for multi-channel single-molecule localization as described above.

[0048] Beneficial effects:

[0049] By monitoring the reference registration error during feature matching and transformation model parameter estimation, control point pairs are iteratively screened to eliminate inaccurately matched control point pairs caused by inaccurate or poor single-molecule localization. This eliminates the difficulty in obtaining the accuracy and precise matching of control points when using randomly distributed fluorescent bead samples as reference materials, and alleviates the problem of existing methods being heavily dependent on the quality of reference materials and their image acquisition.

[0050] During image registration, fluorescent beads randomly distributed on the coverslip surface are used directly as reference materials, and multi-channel images are acquired from several different fields of view to achieve high-precision image registration. This not only reduces the cost of reference materials and the system, but also makes this technical solution widely applicable to any multi-channel single-molecule localization system.

[0051] Transformation model estimation is performed using a second-order polynomial transformation based on multiple iterations and a local weighted average method. During each iteration of control point pair elimination, the positional information of the remaining control point pairs is used to perform a second-order polynomial fitting based on a local weighted average to obtain the optimal estimation parameters. This better addresses the issue of local nonlinear distortion between different channels, thus improving the accuracy of image registration. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart of the image registration method for multi-channel single-molecule localization provided in this application.

[0054] Figure 2 A schematic diagram of the optical structure of the orthogonal astigmatic single-molecule localization system for the image registration method for multi-channel single-molecule localization provided in this application.

[0055] Figure 3 The image registration method for multi-channel single-molecule localization provided in this application shows the dual-channel image distortion caused by orthogonal astigmatism and the expected registration effect.

[0056] Figure 4 This is a schematic diagram illustrating the principle of the local weighted average method for image registration of multi-channel single-molecule localization provided in this application.

[0057] Figure 5a The left channel reference image is used to illustrate the implementation process of the control point pair elimination step in the image registration method for multi-channel single-molecule localization provided in this application, taking a simulated fluorescent bead image as an example.

[0058] Figure 5b The image to be registered is shown in the right channel of the diagram illustrating the process of control point pair rejection in the method proposed in this invention, using a simulated fluorescent bead image as an example, for the image registration method for multi-channel single-molecule localization provided in this application.

[0059] Figure 6aThe image registration results of the experimental fluorescent bead images provided in this application for multi-channel single-molecule localization are compared with those of the traditional local weighted average method.

[0060] Figure 6b The image comparison diagram shows the registration effect of experimental fluorescent bead images for the image registration method for multi-channel single-molecule localization provided in this application, and the results of the method disclosed in this application.

[0061] Figure 6c for Figure 6a Enlarged view of the blue box in the image.

[0062] Figure 6d for Figure 6b Enlarged view of the blue box in the image.

[0063] Figure 6e for Figure 6a Enlarged view within the red box.

[0064] Figure 6f for Figure 6b Enlarged view within the red box.

[0065] Figure 7 The schematic diagram of the image registration device for multi-channel single-molecule localization provided in this application.

[0066] Figure 8 A schematic diagram of the terminal device provided in this application.

[0067] Specific implementation method

[0068] This application provides an image registration method and related apparatus for multi-channel single-molecule localization. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0069] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0070] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0071] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0072] The inventors discovered that single-molecule localization technology has been widely used in the detection and tracking of single molecules in biological samples, and has achieved remarkable results in the field of super-resolution microscopy, with resolutions reaching tens of nanometers, far exceeding the 200-300 nanometer resolution limit of traditional optical microscopes. However, despite these remarkable achievements, some inherent challenges remain. In multi-channel single-molecule localization, there is a problem of precise matching between multi-channel images. Due to human measurement errors during optical path setup, field curvature and chromatic aberration in multi-color imaging channels, and magnification differences in certain situations, multi-channel images often cannot be strictly matched, thus affecting collaborative localization and analysis.

[0073] Furthermore, to improve registration accuracy, image registration is a crucial step in multi-channel single-molecule localization, typically requiring a control point-based approach. Control points are generally obtained directly through multi-channel imaging and single-molecule localization of a reference object. However, existing techniques and methods for image registration and control point acquisition, especially those employing precision displacement stages and nanogrids, are technically challenging and costly, limiting their application in certain scenarios. Therefore, finding a simplified method to obtain high-precision image registration parameters without using sophisticated and expensive equipment or requiring a bright-field illumination mode is a pressing technical challenge for current multi-channel single-molecule localization technology.

[0074] To address the aforementioned issues, this application, in its embodiments, acquires multi-channel images of fluorescent beads with different fields of view and converts them into several dual-channel image pairs, wherein the reference image in each dual-channel image pair is identical. For each dual-channel image pair, the fluorescent beads of the image pair are located to obtain a set of control point pairs. Based on the control point pair sets, image registration parameters for the dual-channel image pairs are determined, and the control point pair sets are filtered based on the image registration parameters and the baseline registration error to obtain a target set of control point pairs. Based on the target set of control point pairs for each dual-channel image pair, image registration parameters are determined, and multi-channel images of actual biological samples are registered. This application determines an image registration model through multi-channel images, performs registration operations on actual samples, iteratively filters control point pairs by monitoring the baseline registration error, and uses a second-order polynomial transformation based on multiple iterations and a local weighted average method for transformation model estimation, achieving high-precision and efficient image registration. While being applicable to various multi-channel single-molecule localization systems, it significantly reduces economic costs and technical requirements.

[0075] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0076] This embodiment provides an image registration method for multi-channel single-molecule localization, such as... Figure 1 As shown, the method includes:

[0077] S10. Acquire multi-channel images of fluorescent beads with different fields of view.

[0078] Specifically, multi-channel imaging of fluorescent beads refers to multi-channel images obtained by imaging samples containing fluorescent beads using a multi-channel single-molecule localization imaging system. Multi-channel single-molecule localization can compensate for the limitations of traditional optical imaging systems in achieving ultra-high resolution imaging due to diffraction, and can achieve accurate localization of individual fluorescent molecules.

[0079] In the image registration stage of the multi-channel single-molecule localization imaging system, randomly distributed fluorescent beads were selected as reference materials. Because fluorescent beads emit sufficiently bright and stable light, they are ideal reference materials for single-molecule localization. Furthermore, randomly distributed fluorescent beads are more readily available and universally applicable compared to reference materials such as finely crafted nanomesh.

[0080] The fluorescent bead sample was placed on a coverslip, and then the coverslip was moved randomly to change the imaging field of view and acquire multi-channel images of the reference object in different fields of view. After each movement, a multi-channel image of the fluorescent bead in a different field of view was obtained. This ensured that the reference object image could fully cover the entire field of view, so that subsequent control point acquisition and image registration operations could be performed across the entire field of view.

[0081] In the specific implementation of this application, taking the registration of dual-channel images in a complex orthogonal astigmatic single-molecule localization system as an example, fluorescent bead samples randomly distributed on the surface of a coverslip are prepared, placed on the stage of a multi-channel single-molecule localization system, and the samples are randomly moved while adjusting the objective lens focal length to capture multi-channel images of the fluorescent beads from several different fields of view. The excitation and emission wavelengths of the fluorescent beads should match the system, the size should preferably not exceed 200 nanometers in diameter, and the concentration should preferably be no less than 30 beads per field of view. Conventional methods such as planar coating and spin coating can be used for preparation. When taking images, areas with good dispersion and relatively uniform distribution of fluorescent beads in the field of view should be selected as much as possible. Specifically, the fluorescent bead model used is TetraSpeck. TM The T7280 has excitation / emission wavelengths of 360 / 430 nm (blue), 505 / 515 nm (green), 560 / 580 nm (orange), and 660 / 680 nm (deep red). The fluorescent beads are used to match the excitation wavelength (656 nm) and emission wavelength (680 nm) of the orthogonal astigmatism system. When the method of this application is applied to other multichannel single-molecule localization systems, the wavelength of the fluorescent beads used depends on the excitation and emission wavelengths of those other multichannel single-molecule localization systems.

[0082] In the example of orthogonal astigmatism, the concentration of the purchased fluorescent bead sample stock solution was 3.7 × 10⁻⁶. 12 / mL, the specific preparation method is as follows: take out about 2μL of fluorescent bead solution, add 1mL of alcohol to dilute it about 500 times, use a vortex shaker to mix it evenly, then take out 5μL of the diluted solution and drop it onto the surface of a clean coverslip, gently spread it to make the solution spread evenly (i.e., the flat spreading method), let it air dry, then invert it onto a glass slide and seal it to make the number of fluorescent beads in the field of view moderate and the distribution relatively dispersed, minimizing the overlap of fluorescent beads.

[0083] As attached Figure 2The diagram shows the optical path of an orthogonal astigmatic single-molecule localization system. Laser represents the laser, ND represents the attenuator, L represents the lens, M represents the mirror, TS represents the shift stage, DM represents the dichroic mirror, OL represents the objective lens, F represents the filter, A represents the aperture, BS represents the beam splitter, CL represents the cylindrical lens, and EMCCD represents the electron multiplication charge-coupled device. This system splits the fluorescence signal into two beams, which pass through two cylindrical lenses with orthogonal focusing directions, and are then imaged onto different regions of the same detector for dual-channel imaging, enabling high-density three-dimensional single-molecule localization imaging. The system's response is a pair of point spread function images. Therefore, compared to traditional astigmatic three-dimensional single-molecule localization based on a single cylindrical lens, this system's response has lower correlation, significantly improving the accuracy of high-density three-dimensional localization. The greater the defocus, the greater the shape difference between the two astigmatic point spread functions, and the more pronounced this advantage in accurate localization. However, due to the complexity of this dual-channel system, image registration is also more difficult and demanding. (Appendix) Figure 3 The distortion of dual-channel images caused by orthogonal astigmatism and the expected registration effect are presented. It can be seen that, because the focusing directions of the two cylindrical lenses in this system are perpendicular to each other, the average focal planes of the two imaging channels are located on opposite sides of the focal plane before the addition of the cylindrical lenses. This means that while the dual-channel image generates the required orthogonal astigmatism spread function image pair, it inevitably suffers from image distortion caused by the anisotropic amplification of astigmatism in the x and y directions.

[0084] S20. For each fluorescent bead multi-channel image, the fluorescent bead multi-channel image is converted into several dual-channel image pairs, wherein the reference image in each dual-channel image pair is the same.

[0085] Specifically, a dual-channel image pair refers to multiple image pairs decomposed from the multi-channel image of each fluorescent bead. Each image pair includes two channel images: one as a reference image and the other as the image to be registered. Once the reference image is determined, the other images to be registered need to be registered with it. All dual-channel image pairs use the same reference image, ensuring that the registration benchmark for all images to be registered is consistent, thus improving the efficiency and accuracy of the overall image matching and analysis process.

[0086] In the specific implementation of this application, in the dual-channel image registration scenario obtained by the dual-channel single-molecule localization system, the left channel image is the reference image, and the right channel image is the image to be registered. In a multi-channel system, one channel image is used as the reference image, and the remaining channel images are used as the images to be registered, forming several pairs of dual-channel images with the reference image, which are then registered channel by channel.

[0087] S30. For each dual-channel image pair, locate the fluorescent beads of the dual-channel image pair to obtain a control point pair set, determine the image registration parameters of the dual-channel image pair based on the control point pair set, and filter the control point pair set based on the image registration parameters and the baseline registration error to obtain a target control point pair set.

[0088] Specifically, by observing and imaging the target, several dual-channel image pairs are obtained, each containing information from a multi-channel image of the fluorescent beads. A dual-channel image pair refers to two images from different perspectives that share the same reference image and contain the same fluorescent bead object. The control point pair set consists of explicitly located position coordinates within the dual-channel image pair, wherein the position coordinates are sub-pixel level coordinates.

[0089] After determining the control point pair set for the two-channel image pair, image registration parameters for the two-channel image pair are obtained based on the control point pair set. These image registration parameters are used to describe a model of the geometric transformation relationship between corresponding points in the two images, transforming one image to the coordinate space of the other to achieve optimal consistency between them.

[0090] Next, the control point pair set is filtered based on image registration parameters and the baseline registration error. The baseline registration error refers to the error value calculated after image registration by comparing the differences in control point positions between the original image and the registered image. Based on this error value, the control point pair set is iteratively filtered. During the filtering process, control point pairs with poor matching due to inaccurate single-molecule localization are deleted, and only those control point pairs that provide accurate information are retained, thus obtaining the target control point pair set.

[0091] In a specific implementation of this application, the step of locating the fluorescent beads of the dual-channel image pair to obtain the control point pair set specifically includes:

[0092] S31. Use Gaussian fitting to locate the fluorescent beads in the dual-channel image to obtain a preset number of reference control points in the reference image and fluorescent beads in the image to be registered to obtain a preset number of registration control points.

[0093] S32. A set of control point pairs for the dual-channel image pair is formed based on a preset number of reference control points and a preset number of registration control points.

[0094] Specifically, in step S31, Gaussian fitting is a widely used fitting method that provides a clear data center (the highest point of the shape) and the width of the shape, making it suitable for locating the center point of point images or image spots with a similar peak shape. Using Gaussian fitting, a preset number of control points corresponding to the dual-channel image are found from the image of each channel. These control points are used to establish subsequent image registration parameters.

[0095] Secondly, in step S32, the control point pair set contains a large number of reference control points and registration control points, which are used in the subsequent image registration process. These control point pairs are used to determine the transformation relationship of the dual-channel image pairs in geometric space, and complete the registration work from one image pair to another.

[0096] Through the above steps, a complete set of control point pairs is obtained. This set of control point pairs is used to precisely control and adjust the image registration process, ensuring optimal consistency of the registered images. Simultaneously, the Gaussian fitting method achieves high-precision results with relatively low computational complexity, improving the overall system performance.

[0097] In the specific implementation of this application, the fluorescent bead spots in the reference image and the image to be registered are first located separately. The location method employs conventional single-molecule localization methods such as Gaussian fitting, and the localization result (i.e., the coordinates of the spot's center point) serves as the control point coordinates. Specifically, in multi-channel single-molecule localization, the fluorescent bead spots are located using conventional single-molecule localization methods such as Gaussian fitting. Gaussian fitting is a commonly used spot localization method. By fitting the peak value and width of the data, the center of the spot is accurately found, i.e., the precise location of the fluorescent bead. In practice, the image is first smoothed, then the spots are found in the smoothed image, and the brightness of each spot is Gaussian fitted to determine the center point coordinates. The localization result of each fluorescent bead spot (i.e., the center point coordinates of the spot) serves as the control point. The above operation is performed on all fluorescent bead spots in the reference image and the image to be registered to obtain a preset number of reference control points and registration control points. These reference and registration control points will be used in the subsequent image registration process to form a control point pair set.

[0098] In another specific implementation of this application, the process of determining the image registration parameters specifically includes:

[0099] S33. Use the local weighted average method to transform the control point pair set to obtain the image registration parameters.

[0100] Specifically, the local weighted average method is a commonly used local processing algorithm in image processing. It utilizes the local pixel values ​​of an image and their corresponding weights to perform weighted averaging, thereby achieving local image processing. In the specific implementation of this application, it is used to transform the control point pair set to obtain image registration parameters.

[0101] In determining the image registration parameters, the control point pairs are first transformed using a local weighted average method. The control points are obtained by acquiring multi-channel images of randomly distributed fluorescent bead samples as reference objects from several randomly selected fields of view, precisely matching the multi-channel images. Then, the local weighted average method is used to transform these control point pairs, and the image registration parameters are obtained by fitting the transformed point pairs. By directly using randomly distributed fluorescent bead samples as reference objects, and without requiring sophisticated equipment or complex operating procedures, high-precision image registration is achieved, while effectively reducing errors in the acquisition and processing of control points.

[0102] In the specific implementation of this application, the coordinates of the obtained control point pairs are transformed using the local weighted average method to obtain the parameter matrix of the transformation model, wherein the transformation model is selected as a second-order polynomial transformation based on local weighted average.

[0103] Specifically, a second-order polynomial transformation model based on local weighted averaging is chosen to express more complex geometric transformations, including rotation, translation, scaling, and nonlinear deformation. The local weighted averaging model makes the model more robust, able to resist the effects of noise and outliers, and adaptable to transformations with local nonlinearity.

[0104] A transformation is performed to obtain the parameter matrix of the transformation model, which describes the mapping relationship from the original control point pairs to the target positions. This parameter matrix will serve as the basis for further image registration, achieving high-precision image registration and improving the accuracy of multi-channel single-molecule localization.

[0105] In another specific implementation of this application, the transformation function in the image registration parameters is:

[0106]

[0107]

[0108]

[0109]

[0110] Among them, P i (x,y) represents the fitting polynomial for the i-th control point, and m represents the polynomial P. i The order of (x,y), W i (R) represents the weight factor of the i-th control point, R represents the distance factor, and R n Indicates the distance from the control point (x) within the local control area. i ,y i The furthest distance from other control points, a jkThis represents the registration transformation parameters, and N represents the number of control points within the local area.

[0111] In the specific implementation of this application, the image registration parameter estimation uses the position information of control point pairs to perform second-order multi-form fitting based on local weighted average to obtain the optimal estimated parameters. It is suitable for situations where there is local nonlinear distortion between different channels and can be applied to complex multi-channel single-molecule localization systems.

[0112] Specifically, the basic principle of the local weighted average method is as follows: Based on the image registration method using control points, several pairs of control points are selected in the reference image and the image to be registered, and their correspondence is established. The control points are obtained by Gaussian fitting the fluorescent bead image to obtain the coordinates of the positioning points. The correspondence is established by judging whether the coordinates of the control points in their respective channel images are similar. Then, interpolation or fitting is used to determine the transformation function of the remaining points in the image, i.e.:

[0113] X = f(x,y), Y = g(x,y)

[0114] In the formula, (X,Y) and (x,y) represent the coordinates of any corresponding point in the reference image and the image to be registered, respectively. The image registration process is the process of determining the transformation functions f and g, which can be obtained by pre-setting a model and estimating the model parameters using the coordinate information of the control point pairs. Considering the complex situation that there may be nonlinear distortion in the image due to anisotropic amplification in different channels in a multi-channel single-molecule localization system, and that different parts may have different degrees of distortion, the method described in this application uses the local weighted average method when estimating the transformation model. Its basic principle is as follows: Figure 4 As shown in the figure. Black dots represent any point on the image, and red dots represent control points. The size of the local region can be adjusted by setting the number of nearest neighbor points n (n=6 in the figure). For a certain control point (x... i ,y i Using n control points, including the one mentioned above, determine its coordinates X and the corresponding point. i The functional relationship is obtained by using a local weighted average method, which typically employs polynomial fitting.

[0115]

[0116] In the formula, m represents the polynomial P i The order of (x, y) is generally taken as second order. Therefore, by substituting the coordinates of n control point pairs (n≥6) into the above formula, the parameter a can be determined using least squares fitting. jk Thus, the second-order polynomial P corresponding to the i-th control point is obtained. i(x,y). Furthermore, to obtain the transformation function f for the X-coordinate transformation of any point, the local weighted average method introduces the following weighting factor:

[0117]

[0118] In the formula R n For control points (x) within a local area i ,y i The transformation function f for any point (x, y) obtained by weighted averaging the farthest distances from other control points is:

[0119]

[0120] In the formula, N represents the total number of control points. Only when point (x, y) is located at control point (x... i ,y i When the control point is within a local range, the polynomial P corresponding to that control point is... i (x,y) is the only effective coordinate, thus avoiding the adverse effects of control points that are too far apart. The method for determining the transformation function g in the Y direction is similar and will not be elaborated here. Finally, the coefficients of each polynomial in the two direction transformation functions are represented by the parameter matrix T∈R. 6×2×N To express.

[0121] After determining the parameter matrix T, the bilinear interpolation method is used to resample the image to be registered based on the second-order polynomial transformation function corresponding to the parameter matrix, thereby obtaining the registered image.

[0122] In another specific implementation of this application, the step of filtering the control point pair set based on the image registration parameters and the baseline registration error to obtain the target control point pair set specifically includes:

[0123] S341. Based on the image registration parameters, transform the image to be registered in the dual-channel image pair to obtain the registered image;

[0124] S342. The registered image is located to obtain registration control points, and the reference registration error between the registration control points and the reference control points in the reference image is calculated respectively.

[0125] Specifically, in step S341, the image registration parameters along with their parameter matrix are applied to the image to be registered and transformed to obtain the image after registration, so that the image to be registered is aligned with the reference image, thereby enabling the control point positions between the images to be accurately matched in the entire image space.

[0126] Next, the registered image is repositioned using Gaussian fitting, and the positioning results are compared with the positioning results of the reference image one by one to calculate the reference registration error of each control point pair.

[0127] In one specific implementation of this application, the expression for the reference registration error is used as an important parameter for evaluating registration quality, indicating the degree of difference between the reference image and the image to be registered at control points.

[0128] The registration error is defined as:

[0129]

[0130] Among them, (X) i ,Y i (X′) represents the coordinates of the i-th control point in the reference image of the two-channel image pair. i ,Y′ i ) represents (X i ,Y i The coordinates of the image in the registration image.

[0131] S343. Remove control point pairs whose reference registration error is greater than the preset registration error to obtain the filtered control point pair set.

[0132] Specifically, each control point pair is screened based on its reference registration error. When the reference registration error of a control point pair is greater than or equal to a set error threshold, this control point pair will be removed. This process selects a subset of control point pairs with smaller reference registration errors from the control point pair set, forming the target control point pair set. The set error threshold can be designed based on the system's single-molecule positioning accuracy.

[0133] In one specific implementation of this application, after removing control point pairs with reference registration errors greater than a preset registration error to obtain a filtered set of control point pairs, the method further includes:

[0134] The step of filtering the control point pair set based on the image registration parameters and the preset registration error is repeated until there are no control point pairs with a reference registration error greater than the preset registration error.

[0135] Specifically, based on the image registration parameters and the preset registration error, the control point pair set for each dual-channel image pair is evaluated. If the baseline registration error of a control point pair is greater than the preset registration error, the registration effect of this control point pair is considered poor, and it needs to be deleted from the control point pair set to be processed.

[0136] Next, the remaining control point pairs are subjected to the same filtering step based on the image registration parameters and the preset registration error. This step is repeated until the baseline registration error of all remaining control point pairs is less than or equal to the preset registration error, thereby gradually eliminating control point pairs with poor registration results and retaining only those with good registration results.

[0137] Finally, for multi-channel images, the above steps are performed on different channels of each field of view. All control point pairs that meet the conditions are merged to form a large, comprehensive, accurately located, and precisely matched set of control point pairs. Using this set of control point pairs, a better and more comprehensive set of image registration parameters can be obtained to complete the image registration task.

[0138] S40. Determine image registration parameters based on the target control point pair set of each dual-channel image pair, and register the multi-channel image of the actual biological sample using the image registration parameters.

[0139] Specifically, for each dual-channel image pair, based on the filtered and merged target control point pair set, the coordinates of the merged control point pair are transformed using the local weighted average method to obtain the system transformation parameters, thereby determining the final image registration parameters for each dual-channel image pair, which are used for multi-channel image registration of actual biological samples.

[0140] Through the above steps, a high-precision image registration method is achieved. This method can be directly applied to various multi-channel single-molecule localization systems and is particularly suitable for situations where there is nonlinear distortion between different channels. When using this method for image registration, it is not necessary to purchase precise (and often expensive) nanogrids as reference materials, nor is it necessary for the system to have a bright-field illumination mode, nor is it required to equip the system with a nanometer-precision displacement stage. It is only necessary to directly use the fluorescent bead sample, which is usually randomly distributed on the surface of the coverslip and is commonly used in multi-channel single-molecule localization systems, as the reference material, and randomly select several different fields of view to acquire its multi-channel images to achieve high-precision image registration.

[0141] In another specific implementation of this application, as shown in Figure 5, the implementation process of the control point rejection step in the proposed method is illustrated using a set of simulated fluorescent bead image data. Figure 5 (left) and Figure 5 (right) show the simulated left and right channel images of randomly distributed fluorescent bead samples taken using an orthogonal astigmatic single-molecule localization system, respectively, along with their localization results. The white spots represent fluorescent bead images, and the red crosses represent single-molecule localization results. In this simulation experiment, the three-dimensional positions of the fluorescent beads are set to random. The fluorescent bead images in both channels are generated based on the x, y, and z coordinates of the fluorescent beads and the calibration curve of the actual imaging system (i.e., the relationship between the size of the image in the x and y directions and the axial position z). Simultaneously, to simulate the local nonlinear differences between the two orthogonal astigmatic channels in the actual system, a random error of 200 nanometers is introduced into the position of the fluorescent beads in the right channel image. It can be seen that the two channel images are distorted due to different anisotropic magnification (manifested as inconsistent relative distribution patterns of fluorescent bead spots in the two images). Furthermore, because the positions of the fluorescent beads are random, some closely spaced fluorescent bead spots in the field of view result in significantly inaccurate localization (as shown in the red box for positions 2-5). If all localization point pairs (approximately 40) in this field of view are used for local weighted average transformation registration, the average reference registration error obtained after registration (i.e., The error threshold (N being the number of control points) is 29.29 nanometers. However, using the method proposed in this application, with the error threshold set to 30 nanometers, the labels of the control point pairs removed in each iteration and the average reference registration error obtained by performing local weighted average transformation registration using the remaining control points are shown in Table 1. It can be seen that only 3 iterations are needed to accurately find and remove several incorrectly located control point pairs, thereby improving the average reference registration error to 2.89 nanometers. The registration accuracy is 10 times higher than when performing local weighted average transformation registration directly without using this method.

[0142] Furthermore, in another specific implementation of this application, in order to evaluate the actual registration effect of the image registration method proposed in the orthogonal astigmatic single-molecule localization system, the appendix... Figure 6a - Appendix Figure 6fA set of actual captured images of fluorescent beads are presented, where purple represents the left channel image, green represents the registered right channel image, and grayscale represents the image when the two are accurately superimposed. The experiment first prepared 200 nm diameter fluorescent bead samples randomly distributed on a coverslip and placed them on the stage of an orthogonal astigmatic single-molecule localization system for dual-channel imaging. Then, single-molecule localization was performed on the obtained dual-channel images. The localization results were used as control point pair coordinates. Registration parameter matrices were calculated using both the traditional local weighted average transformation method (i.e., without removing mismatched control point pairs) and the method proposed in this application (i.e., removing mismatched control point pairs by monitoring registration errors). These matrices were then used to transform the right channel image, resulting in the registered right channel image. To visually demonstrate the registration effect, the left channel reference image was set as a purple pseudocolor image, and the registered right channel image was set as a green pseudocolor image. The two were then superimposed. The superposition results obtained using the traditional local weighted average method and the method proposed in this invention are shown in the attached figures. Figure 6a and attached Figure 6b As shown, in the two-channel image, the areas where the fluorescent bead images accurately overlap will appear grayish-white; otherwise, they will appear as staggered purple and green. (See attached image.) Figure 6c and attached Figure 6d Each is attached Figure 6a and attached Figure 6b Enlarged image within the blue box (with attached) Figure 6e and attached Figure 6f Each is attached Figure 6a and attached Figure 6b The enlarged view within the red box shows that, in the areas within these two boxes, the fluorescent bead images of the two channels still exhibit significant mismatch after registration using the traditional local weighted average transformation method. However, when the method proposed in this invention is used, the fluorescent bead images match very well, with virtually no purple or green inclusions. Further quantitative calculations indicate that... Figure 6a The corresponding average reference registration error is 54.69 nanometers, while the attached... Figure 6b With a registration accuracy of only 6.08 nanometers, the accuracy is improved by 9 times, which is consistent with the simulation results. This shows that even for dual-channel images such as orthogonal astigmatic single-molecule localization systems with complex and severe nonlinear distortions, the method proposed in this application can still achieve a registration accuracy of about 6 nanometers by directly using randomly distributed fluorescent bead samples.

[0143] In summary, this embodiment provides an image registration method for multi-channel single-molecule localization. This application acquires several multi-channel images of fluorescent beads with different fields of view and converts them into several dual-channel image pairs, where the reference image in each dual-channel image pair is the same. For each dual-channel image pair, the fluorescent beads of the image pair are located to obtain a set of control point pairs. Based on the control point pair sets, image registration parameters for the dual-channel image pairs are determined. The control point pair sets are then filtered based on the image registration parameters and the baseline registration error to obtain a target set of control point pairs. Based on the target control point pair sets of each dual-channel image pair, image registration parameters are determined, and multi-channel images of actual biological samples are registered. This application determines an image registration model through multi-channel images, performs registration operations on actual samples, iteratively filters control point pairs by monitoring the baseline registration error, and uses a second-order polynomial transformation based on multiple iterations and a local weighted average method for transformation model estimation. This achieves high-precision and efficient image registration, significantly reducing economic costs and technical requirements while being applicable to various multi-channel single-molecule localization systems.

[0144] Based on the above-described image registration method for multi-channel single-molecule localization, this embodiment provides an apparatus for image registration of multi-channel single-molecule localization, such as... Figure 2 As shown, the device includes:

[0145] Positioning module 100 is used for image registration methods for multi-channel single-molecule positioning;

[0146] The conversion module 200 converts each multi-channel image of a fluorescent bead into several pairs of dual-channel images, wherein the reference images in each pair of dual-channel images are the same.

[0147] The filtering module 300, for each dual-channel image pair, locates the fluorescent beads of the dual-channel image pair to obtain a set of control point pairs, determines the image registration parameters of the dual-channel image pair based on the set of control point pairs, and filters the set of control point pairs based on the image registration parameters and the baseline registration error to obtain a target set of control point pairs.

[0148] The determination module 400 determines image registration parameters based on the target control point pair set of each dual-channel image pair, and registers the multi-channel images of the actual biological sample using the image registration parameters.

[0149] Based on the above-described image registration method for multi-channel single-molecule localization, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the image registration method for multi-channel single-molecule localization as described in the above embodiment.

[0150] Based on the image registration method for multi-channel single-molecule localization described above, this application also provides a terminal device, as shown in FIG5, which includes at least one processor 20; a display screen 21; and a memory 22, and may further include a communication interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communication interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communication interface 23 can transmit information. The processor 20 can call logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0151] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0152] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0153] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0154] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image registration method for multi-channel single-molecule localization, characterized in that, The method includes: Acquire multi-channel images of fluorescent beads from several different fields of view; For each fluorescent bead multi-channel image, the fluorescent bead multi-channel image is converted into several dual-channel image pairs, wherein the reference image in each dual-channel image pair is the same; For each dual-channel image pair, fluorescent beads in the dual-channel image pair are located to obtain a set of control point pairs. Based on the set of control point pairs, image registration parameters for the dual-channel image pair are determined. Based on the image registration parameters and the baseline registration error, the set of control point pairs is filtered to obtain a target set of control point pairs. Image registration parameters are determined based on the target control point pair set of each dual-channel image pair, and the multi-channel images of the actual biological sample are registered using the image registration parameters. The fluorescence bead multi-channel imaging uses fluorescence bead excitation and emission wavelengths that match the multi-channel single-molecule localization system. The diameter of the fluorescence beads does not exceed 100 nanometers, and the concentration of fluorescence beads is no less than 30 per field of view. The transformation function in the image registration parameters is: ， ， ， ， in, Indicates the first Fitting polynomial for each control point Representing a polynomial The order of Indicates the first Weighting factors for each control point Represents the distance factor. Indicates the distance from the control point within the local control area. The furthest distance from other control points, This represents the registration transformation parameters, and N represents the number of control points within the local area. The optimal estimated parameters for image registration are determined by fitting a second-order polynomial based on local weighted average using the positional information of control point pairs. The process of filtering the control point pair set based on the image registration parameters and the baseline registration error to obtain the target control point pair set specifically includes: The images to be registered in the dual-channel image pair are transformed based on the image registration parameters to obtain the registered images; The fluorescent beads in the registered image are located to obtain registration control points, and the reference registration error between the registration control points and the reference control points in the reference image is calculated respectively. Remove control point pairs whose baseline registration error is greater than the preset registration error to obtain the filtered set of control point pairs; The step of filtering the control point pair set based on the image registration parameters and the preset registration error is repeated until there are no control point pairs with a reference registration error greater than the preset registration error.

2. The image registration method for multi-channel single-molecule localization according to claim 1, characterized in that, The control point pair set for locating the dual-channel image pair specifically includes: Gaussian fitting is used to locate the fluorescent beads of the reference image in the dual-channel image to obtain a preset number of reference control points, and fluorescent beads of the image to be registered are used to obtain a preset number of registration control points. The control point pair set of the dual-channel image pair is formed based on a preset number of reference control points and a preset number of registration control points.

3. The image registration method for multi-channel single-molecule localization according to claim 1, characterized in that, The process of determining the image registration parameters specifically includes: The control point pair set is transformed using the local weighted average method to obtain the image registration parameters.

4. The image registration method for multi-channel single-molecule localization according to claim 1, characterized in that, The expression for the reference registration error is: ， in, Indicates the first The coordinates of each control point in the reference image of the two-channel image pair. express Coordinates in the registered image.

5. An image registration device for multi-channel single-molecule localization, the device comprising: The positioning module is used to acquire multi-channel images of fluorescent beads from several different fields of view; The conversion module converts each multi-channel image of a fluorescent bead into several pairs of dual-channel images, wherein the reference images in each pair of dual-channel images are the same. The filtering module locates the fluorescent beads of each dual-channel image pair to obtain a set of control point pairs, determines the image registration parameters of the dual-channel image pair based on the set of control point pairs, and filters the set of control point pairs based on the image registration parameters and the baseline registration error to obtain a target set of control point pairs. The determination module determines image registration parameters based on the target control point pair set of each dual-channel image pair, and registers the multi-channel images of the actual biological sample using the image registration parameters. The fluorescence bead multi-channel imaging uses fluorescence bead excitation and emission wavelengths that match the multi-channel single-molecule localization system. The diameter of the fluorescence beads does not exceed 100 nanometers, and the concentration of fluorescence beads is no less than 30 per field of view. The transformation function in the image registration parameters is: ， ， ， ， in, Indicates the first Fitting polynomial for each control point Representing a polynomial The order of Indicates the first Weighting factors for each control point Represents the distance factor. Indicates the distance from the control point within the local control area. The furthest distance from other control points, This represents the registration transformation parameters, and N represents the number of control points within the local area. The optimal estimated parameters for image registration are determined by fitting a second-order polynomial based on local weighted average using the positional information of control point pairs. The process of filtering the control point pair set based on the image registration parameters and the baseline registration error to obtain the target control point pair set specifically includes: The images to be registered in the dual-channel image pair are transformed based on the image registration parameters to obtain the registered images; The fluorescent beads in the registered image are located to obtain registration control points, and the reference registration error between the registration control points and the reference control points in the reference image is calculated respectively. Remove control point pairs whose baseline registration error is greater than the preset registration error to obtain the filtered set of control point pairs; The step of filtering the control point pair set based on the image registration parameters and the preset registration error is repeated until there are no control point pairs with a reference registration error greater than the preset registration error.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that can be executed by one or more processors to implement the steps in the image registration method for multi-channel single-molecule localization as described in any one of claims 1-4.

7. A terminal device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the image registration method for multi-channel single-molecule localization as described in any one of claims 1-4.

Images