Cell image registration method and device and computer equipment
By combining cell flow field maps to generate cell instance point sets and performing feature matching, the problems of unstable feature matching and insufficient generalization ability in cell image registration are solved, achieving high-precision and stable cell image registration that is applicable to various imaging conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LEAD HEALTHCARE TECHNOLOGY (GUANGZHOU) CO LTD
- Filing Date
- 2025-12-25
- Publication Date
- 2026-05-08
AI Technical Summary
Existing cell image registration techniques are unstable in feature matching and difficult to achieve accurate alignment when faced with changes in cell life activities such as cell division, movement, and apoptosis, as well as imaging conditions. They are particularly prone to getting stuck in local optima in areas with weak texture or uniformity. Furthermore, deep learning-based methods have limited generalization ability and are difficult to adapt to different imaging conditions.
By combining the cell flow field map, a set of cell instance points is generated. Through feature matching and affine transformation, the cell flow field map is used to characterize the cell geometry and orientation information. A registration constraint based on orientation similarity is established to avoid the adverse effects of lighting or texture changes and achieve high-precision registration.
It improves the accuracy and stability of cell image registration, expands the generalization ability of the registration method, is applicable to various types of cell image registration, and can maintain efficient registration under different imaging conditions.
Smart Images
Figure CN121999019A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the fields of biomedical technology and image processing technology, and in particular relates to a cell image registration method, apparatus and computer equipment. Background Technology
[0002] Precise registration of cell images is a key technique in biomedical image analysis, aiming to spatially align two or more cell images acquired at different times, from different perspectives, or under different conditions. This technique has crucial value in fields such as cell dynamics research, drug response tracking, disease diagnosis, and super-resolution image reconstruction.
[0003] Existing cell image registration techniques mainly fall into two categories: feature-based registration methods and image grayscale-based registration methods. In cell images, due to cell division, movement, apoptosis, and changes in staining and imaging conditions, the image content exhibits significant non-rigid deformation and appearance variations. Traditional feature-based registration methods often struggle to extract stable and repeatable feature points in such environments, leading to high mismatch rates and registration failures. Furthermore, because regions such as the cytoplasm typically lack rich texture information, exhibiting weak texture or uniform characteristics, image grayscale-based registration methods are prone to getting trapped in local optima in these areas, causing the optimization process to stall and preventing accurate alignment, thus affecting the final registration result.
[0004] In recent years, with the development of deep learning technology, deep learning-based registration methods have become a research hotspot. These methods can use convolutional neural networks (CNNs) to directly learn the mapping from image pairs to spatial transformations, which helps improve registration efficiency. However, the network models of deep learning-based registration methods are trained on specific datasets, and their registration performance significantly degrades for new data with different imaging patterns, cell types, and fluorescent markers. The model's generalization ability strongly depends on the diversity and representativeness of the training data, severely limiting the practical application of this method in a wide range of biomedical research. Summary of the Invention
[0005] In view of this, embodiments of this application provide a cell image registration method, apparatus, and computer device, which can register cell images in conjunction with cell flow field maps, helping to improve the accuracy of image registration, expand the generalization ability of the registration method, and be applicable to the registration of various types of cell images.
[0006] A first aspect of this application provides a cell image registration method, including: Obtain feature maps corresponding to the two cell images to be registered, wherein the feature maps include a foreground probability map and a cell flow field map; Based on the foreground probability map and the cell flow field map, two sets of cell instance point sets are generated; Feature matching is performed on the two sets of cell instance points to obtain the target matching combination; Based on the target matching combination, a first affine transformation of the two cell images is determined; Based on the cell flow field map and the first affine transformation, the two cell images are registered.
[0007] Optionally, the step of generating two sets of cell instance point sets based on the foreground probability map and the cell flow field map includes: Initialize a coordinate map, which is used to represent the coordinate position of the flow direction of each coordinate point corresponding to the input image, and the values in the coordinate map represent the coordinate values of each coordinate point; The coordinate map is updated by overlaying the cell flow field map onto the coordinate map to obtain a target coordinate map, in which any cell point in the target coordinate map flows to the center point of its respective cell; The multiple cell points in the target coordinate map are processed according to the foreground probability map to obtain two sets of cell instance points that correspond one-to-one with the two cell images.
[0008] Optionally, the step of processing multiple cell points in the target coordinate map according to the foreground probability map to obtain two sets of cell instance point sets that correspond one-to-one with the two cell images includes: For any of the target coordinate maps, remove the cell center points in the target coordinate map that correspond to each center point whose probability value in the foreground probability map is lower than the first threshold. Based on the remaining cell center points in the target coordinate map, the multiple cell points are merged into multiple point sets; Multiple point sets containing fewer than the second threshold cell points are removed to obtain a cell instance point set.
[0009] Optionally, the step of performing feature matching on the two sets of cell instance points to obtain a target matching combination includes: The shape features of the two sets of cell instance points are determined respectively, and feature matching is performed on the shape features to obtain multiple matching combinations, and any matching combination has a corresponding similarity. Based on the similarity, a target matching combination is determined from multiple sets of matching combinations.
[0010] Optionally, determining the shape features of the two sets of cell instance point sets respectively includes: Calculate the HU invariant moments for the two sets of cell instance points respectively to obtain the shape features corresponding to the two sets of cell instance points.
[0011] Optionally, determining the first affine transformation of the two cell images based on the target matching combination includes: Calculate the displacement of each corresponding cell center point in the target matching combination; A first affine transformation of the two cell images is determined based on the displacement of the multiple cell center points.
[0012] Optionally, registering the two cell images based on the cell flow field map and the first affine transformation includes: Align the cell flow field maps corresponding to the two cell images according to the first affine transformation; Calculate the second affine transformation, which satisfies the direction-based flow field consistency constraint. The two cell images are registered according to the second affine transformation.
[0013] Optionally, the calculation of the second affine transformation includes: Construct a direction-based flow field consistency loss function; Calculate the optimal second affine transformation that minimizes the loss value of the direction-based flow field consistency loss function.
[0014] A second aspect of this application provides a cell image registration apparatus, comprising: The acquisition module is used to acquire feature maps corresponding to two cell images to be registered, the feature maps including a foreground probability map and a cell flow field map; The generation module is used to generate two sets of cell instance point sets based on the foreground probability map and the cell flow field map; The matching module is used to perform feature matching on the two sets of cell instance points to obtain a target matching combination; A transformation module is used to determine a first affine transformation of the two cell images based on the target matching combination; The registration module is used to register the two cell images based on the cell flow field map and the first affine transformation.
[0015] A third aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer device performs the method as described in any of the first aspects above.
[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a computer, implements the method described in any of the first aspects above.
[0017] A fifth aspect of this application provides a computer program product, including a computer program that, when the computer program is run, causes the method described in any of the first aspects above to be executed.
[0018] Compared with the prior art, the embodiments of this application have the following beneficial effects: In this embodiment, by acquiring the feature maps corresponding to the two cell images to be registered, namely the foreground probability map and the cell flow field map, two sets of cell instance point sets can be generated based on the foreground probability map and the cell flow field map. By performing feature matching on the two sets of cell instance point sets, a target matching combination can be obtained. The computer device can determine the first affine transformation of the two cell images based on the target matching combination, thus completing the coarse registration of the cell images. On this basis, by combining the cell flow field map and the first affine transformation, the two cell images can be registered more accurately, improving the accuracy of cell image registration.
[0019] This application employs a cell center point detection algorithm based on cell flow field. Through vector integration, it locates convergence points with consistent flow direction as cell centers, thereby avoiding feature drift caused by illumination or texture changes and solving the problem of unstable feature matching in existing registration methods. Furthermore, by constructing cell shape features based on physical consistency, this application avoids the adverse effects of illumination or texture changes on the registration process, further improving the stability of feature matching. Secondly, addressing the difficulty of registering weak textures and uniform regions, this application establishes a registration constraint term based on directional similarity by statistically analyzing local directional consistency, achieving reliable alignment of weak texture regions. Thirdly, addressing the issues of generalization dependence on data distribution and performance degradation under different imaging conditions in existing deep learning registration methods, this application establishes registration constraints based on cell flow field maps. It utilizes flow field maps to characterize cell geometry and orientation information, without relying on grayscale or texture features, and possesses good invariance to imaging modes and labeling methods. This allows the registration method provided by this application to be widely applied to cell image registration in various scenarios and of various types. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a schematic diagram of a cell image registration method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a possible implementation of S102 in a cell image registration method provided in this application embodiment; Figure 3 This is an algorithm flowchart of a cell image registration method provided in an embodiment of this application; Figure 4 This is a schematic diagram of a cell image registration system provided in an embodiment of this application; Figure 5 This is a schematic diagram of a cell image registration device provided in an embodiment of this application; Figure 6 This is a schematic diagram of a computer device provided in an embodiment of this application. Detailed Implementation
[0022] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0023] Before introducing the cell image registration method provided in the embodiments of this application, we will first introduce the existing cell image registration technology.
[0024] As described in the background section, existing image registration techniques can be mainly divided into two categories: feature-based registration methods and image grayscale-based registration methods.
[0025] Feature-based registration methods first extract salient feature points (such as corners and edge intersections), lines (such as the cytoskeleton), or regions (such as cell nuclei) from the images. Then, the features are described using feature descriptors (such as Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), Oriented FAST and Rotated BRIEF (ORB), etc.), and a correspondence between the features of the two images is established. Finally, registration is completed by solving a spatial transformation model (such as rigid body transformation, affine transformation, or perspective transformation). In cell images, structures such as the cell nucleus and specific organelles are often used as feature sources. These methods are relatively computationally efficient and insensitive to changes in illumination and grayscale.
[0026] Image grayscale-based registration methods: These methods do not rely on explicit feature extraction, but directly utilize all grayscale information of the images to measure the similarity between two images. By defining a similarity measure (such as mutual information, normalized cross-correlation, etc.) and a spatial transformation model, optimization algorithms (such as gradient descent, quasi-Newton methods, etc.) are used to continuously adjust the transformation parameters to maximize the similarity measure. These methods can utilize all the information of the images for registration, and theoretically can achieve higher registration accuracy, especially suitable for situations where features are not obvious.
[0027] In recent years, with the development of deep learning technology, deep learning-based registration methods have become a research hotspot. These methods typically use convolutional neural networks (CNNs) to directly learn the mapping from image pairs to spatial transformations. A mainstream approach is to use a U-Net network with an encoder-decoder structure, taking a floating image and a reference image as input, to directly predict a dense displacement field, i.e., the movement vector of each pixel. This method, through end-to-end learning, avoids the complex iterative optimization process in traditional methods, significantly improving the registration speed.
[0028] However, although the registration methods described above in the prior art are widely used in cell image registration, they still face the following challenges and shortcomings when applied to complex and varied cell images: 1. Vulnerability of Feature Matching: In cell images, due to cell division, movement, apoptosis, and other life activities, as well as changes in staining and imaging conditions, the image content will exhibit significant non-rigid deformation and appearance changes. Traditional feature-based registration methods (such as SIFT) often struggle to extract stable and repeatable feature points in such environments, and the discriminative power of feature descriptors is insufficient, leading to a high mismatch rate and a high likelihood of registration failure.
[0029] 2. Insufficient ability to handle weak textures and uniform regions: Regions such as cytoplasm often lack rich texture information, exhibiting weak textures or uniform characteristics. Image grayscale-based registration methods are prone to getting stuck in local optima in these regions because the similarity measure lacks effective gradient guidance, causing the optimization process to stagnate and failing to achieve accurate alignment.
[0030] 3. Limited generalization ability to specific scenarios and markers: Many deep learning-based registration methods, after being trained on specific datasets, show a significant decrease in registration performance when faced with new data with different imaging patterns, cell types, and fluorescent markers. The model's generalization ability strongly depends on the diversity and representativeness of the training data, which limits its practical application in a wide range of biomedical research.
[0031] The aforementioned shortcomings of existing registration methods stem from the fact that cell images are essentially projections of dynamic, heterogeneous biological systems into two- or three-dimensional space. Existing registration methods, especially traditional ones, suffer from mathematical models and assumptions (such as feature stability and grayscale invariance) that fail to fully encompass such complex biological variations. While deep learning-based registration methods possess powerful representational capabilities, their "black box" nature makes it difficult to embed hard constraints on the physical consistency of deformation, and their performance is highly data-dependent, performing poorly when data is insufficient or distributionally offset.
[0032] In developing this application, the applicant discovered that cell segmentation algorithms can generate flow field maps representing cell structure information. These flow field maps reflect the direction vector of each pixel pointing towards the cell center, thus allowing them to describe the cell's morphology and the consistency of its local space. However, such flow fields are typically only used for cell instance segmentation and are not fully utilized in image registration. When cell images from different times or different viewpoints exhibit non-rigid deformation, registration methods based directly on grayscale or edges often struggle to achieve accurate cell-level alignment.
[0033] Therefore, this application provides a novel cell image registration scheme based on cell flow field that can balance high precision, strong robustness, and good generalization ability. It can register cell images by combining cell flow field maps, which helps to improve the accuracy of image registration, expand the generalization ability of the registration method, and is applicable to the registration of various types of cell images.
[0034] The technical solution of this application will be described below through specific embodiments.
[0035] Reference Figure 1 The diagram illustrates a cell image registration method provided in an embodiment of this application, which may specifically include the following steps: S101. Obtain feature maps corresponding to the two cell images to be registered, wherein the feature maps include a foreground probability map and a cell flow field map.
[0036] It should be noted that the embodiments of this application can be applied to computer devices, that is, the executing subject of this method is a computer device. This computer device can achieve accurate registration of the cell image to be registered by executing the various steps of the method provided in the embodiments of this application. The aforementioned computer device can be an electronic device with image processing capabilities that can execute the various steps of this method, such as a desktop computer or a cloud server. The embodiments of this application do not limit the specific type of computer device.
[0037] In this embodiment, the number of cell images to be registered can be two or more. These cell images can be obtained at different times, from different perspectives, or under different conditions. For example, before stitching the cell images, a microscope can be used to photograph the sample on a slide region by region to obtain multiple cell images with a certain overlap between adjacent areas. For the two cell images with the overlapping area, registration processing is required. In this embodiment, the cell images to be registered can be two adjacent cell images in the aforementioned scenario. Of course, this embodiment can also be used to register cell images that need registration in other scenarios, and can also obtain highly accurate registration results. The source of the cell images to be registered is not limited in this embodiment.
[0038] In this embodiment, the feature map corresponding to the cell image may include a foreground probability map and a cell flow field map. The foreground probability map is a probability matrix of the same size as the input image, where the value of each pixel (typically between 0 and 1) represents the probability that the pixel belongs to a cell region (foreground). The foreground probability map can provide rich confidence information. The cell flow field map is a vector graphic that visualizes the movement patterns and directions of a cell population. It can overlay the displacement or velocity information of cells in an image sequence (such as live cell microscopy video) onto the image using arrows, streamlines, or color codes, visually displaying the collective movement trends, speeds, and directions of cells in space.
[0039] In one possible implementation of this application, the computer device may be configured with a network, model, or module capable of processing cell images and generating corresponding foreground probability maps and cell flow maps. When the computer device receives a cell image to be processed, it can invoke the corresponding network or model to process the cell image and obtain the corresponding foreground probability map and cell flow map. For example, Cellpose is a model capable of outputting cell flow maps; by configuring it in the computer device, the model can output the flow map of the cell image. Of course, other models capable of generating cell flow maps exist in the prior art, and this application does not limit how the computer device generates the foreground probability map and cell flow map of the cell image.
[0040] As an example of an embodiment of this application, the cell structure feature generation module in the computer device can integrate the relevant processing functions of the foreground probability map and the cell flow field map. After receiving any cell image, the computer device can use the cell structure feature generation module to process the image to obtain the foreground probability map and the cell flow field map corresponding to the cell image.
[0041] In this embodiment of the application, the two cell images to be input can be represented as images. I 0 and image I 1. For the two images above, corresponding foreground probability maps can be generated respectively. p and cell flow field diagram flow Among them, images I The foreground probability plot of 0 can be further distinguished into p 0, Image I The foreground probability map of 1 can be further divided into p 1; correspondingly, the image I The cell flow field map of 0 can be used to flow 0, Image I The cell flow field map of 1 can be used to... flow 1. It should be noted that the above cell flow field diagram... flow 0 or flow 1. It can be further divided into two diagrams: one in the x-direction (horizontal direction) and the other in the y-direction (vertical direction).
[0042] The above prospect probability diagram p and cell flow field diagram flow The size is the same as the size of the input cell image, i.e., the cell image and the corresponding foreground probability map. p and cell flow field diagram flow All dimensions are height × width.
[0043] S102. Based on the foreground probability map and the cell flow field map, generate two sets of cell instance point sets.
[0044] A cell instance is a specific example of a cell. In an image, each cell instance is represented as a set of points. A computer device can generate the corresponding set of cell instance points based on a foreground probability map and a cell flow field map.
[0045] Specifically, for cell images I 0, its corresponding foreground probability map is p 0, the cell flow field diagram is flow 0. Computer equipment can target foreground probability maps. p 0 and cell flow field diagram flow 0 is processed to obtain a set of cell instance points. For cell images...I 1, its corresponding foreground probability map is p 1. Cell flow field diagram is flow 1. Computer equipment can analyze foreground probability maps. p 1 and cell flow field diagram flow 1. Processing is performed to obtain another set of cell instance points. Thus, by processing the foreground probability map and cell flow field map of the two cell images to be registered, two sets of cell instance points can be obtained.
[0046] In one possible implementation of this application embodiment, a cell instance acquisition module may be configured in the computer device. The cell instance acquisition module can be used to process the foreground probability map and the cell flow field map to obtain the corresponding cell instance point set.
[0047] In one possible implementation of the embodiments of this application, see [link to relevant documentation]. Figure 2 This is a schematic diagram of a possible implementation of S102 in the cell image registration method provided in this application embodiment. S102 generates two sets of cell instance point sets based on the foreground probability map and the cell flow field map, specifically including the following steps S1021-S1023: S1021. Initialize the coordinate map, which is used to represent the coordinate position of each coordinate point corresponding to the input image. The values in the coordinate map represent the coordinate values of each coordinate point.
[0048] In this embodiment of the application, a coordinate map can be initialized first. This coordinate map The dimensions are the same as those of the cell image, foreground probability map, and cell flow field map, all being height × width. (The above coordinate map...) It can be used to characterize the coordinate position of the flow direction for each coordinate point corresponding to the input image, the initial coordinate map. The value in the table represents the coordinates of each point. That is:
[0049]
[0050] The above and Two graphs can be represented, one in the x-direction and the other in the y-direction. .
[0051] S1022. The coordinate map is updated by overlaying the cell flow field map onto the coordinate map to obtain a target coordinate map, in which any cell point in the target coordinate map flows to the center point of its respective cell.
[0052] Initialize coordinate map Then, the computer equipment can update it, overlaying the already obtained cell flow field map. Update the coordinate map. The process can be represented as:
[0053] In the above expression, i Indicates the number of updates. This is achieved by updating the coordinate map. This allows each coordinate point in the diagram to point to a new coordinate along the flow field diagram.
[0054] In this embodiment, step S1022, updating the coordinate map, can be repeated. For example, the coordinate map can be updated N times, and the value of N can be determined according to actual needs. For instance, N can be 200, meaning the update stops after updating the coordinate map 200 times; alternatively, the specific number of updates can be unlimited, and the update stops when the coordinate map no longer changes, thus obtaining the updated coordinate map, i.e., the target coordinate map. This application does not limit the number of times the coordinate map is updated.
[0055] Coordinate map to be completed The updated map yields the target coordinates. After that, it can be assumed that all points on all cells in the diagram eventually flow to the center point of their respective cells.
[0056] S1023. Process the multiple cell points in the target coordinate map according to the foreground probability map to obtain two sets of cell instance points that correspond one-to-one with the two cell images.
[0057] Then, the computer device can combine the foreground probability map with the target coordinate map. Each cell point on the image is further processed to obtain the corresponding cell instance point set. Thus, for a cell image... I 0, can obtain a set of cell instance points. For cell images I 1. Another set of cell instance points can be obtained.
[0058] In one possible implementation of this application embodiment, when processing multiple cell points in a target coordinate map based on a foreground probability map, the processing can be applied to any target coordinate map, for example, based on a cell image. I Foreground probability map corresponding to 0 p 0 and cell flow field diagram flow The target coordinate map obtained from 0 can be compared with the foreground probability map. pCell center points corresponding to center points with a probability value below a first threshold are removed. For example, the first threshold can be set to 0.4, which will remove the center points corresponding to points with a probability value below 0.4 in the foreground probability map from the target coordinate map.
[0059] Then, based on the remaining cell center points in the target coordinate map, multiple cell points can be merged into multiple point sets, and point sets containing fewer than a second threshold can be discarded. For example, the second threshold can be set to 50, which will filter out point sets containing fewer than 50 points after merging, resulting in a set of cell instance point sets.
[0060] For cell-based images I Foreground probability map corresponding to 1 p 1 and cell flow field diagram flow The target coordinate map obtained in step 1 can also be processed in the same way as described above to obtain another set of cell instance points.
[0061] S103. Perform feature matching on the two sets of cell instance points to obtain the target matching combination.
[0062] In this embodiment of the application, the computer device can perform feature matching on two sets of cell instance points that have been obtained, thereby filtering out multiple matching pairs and determining the final matching combination.
[0063] For example, a computer device can determine the shape features of two sets of cell instance points separately. For instance, in one example, the HU-invariant moments of the two sets of cell instance points can be calculated separately to obtain the shape features corresponding to the two sets of cell instance points. That is: and .
[0064] Hu invariant moments, also known as Hu moments, are a set of shape descriptors based on image moments that are invariant to translation, scaling, and rotation of images. Proposed by mathematician Ming-Kuei Hu in his 1962 paper "Visual Pattern Recognition by Moment Invariants," they are a classic feature extraction method in computer vision and image processing.
[0065] Alternatively, Fourier descriptors can be used to extract the shape features of the two sets of cell instance points, but this application does not limit this approach.
[0066] After extracting the shape features of the two sets of cell instance points, the computer device can perform feature matching on the shape features to obtain multiple matching combinations. The computer device can then calculate the corresponding similarity for any matching combination. Based on the similarity, a target matching combination can be determined from the multiple matching combinations. For example, the matching combination with the highest similarity can be selected as the target matching combination.
[0067] S104. Based on the target matching combination, determine the first affine transformation of the two cell images.
[0068] In this embodiment, the computer device can determine the cell image by calculating the displacement of each corresponding cell center point in the final determined target matching combination. I 0 to I 1. First Affine Transformation T 0, the first affine transformation T 0 refers to the initial affine transformation during image registration.
[0069] Specifically, for target matching combinations, the computer device can calculate the displacement between each corresponding cell center point, and then determine the first affine transformation of the two cell images based on the displacement of multiple cell center points, for example, by counting all calculated displacements and taking the median.
[0070] S105. Register the two cell images according to the cell flow field map and the first affine transformation.
[0071] In this embodiment, the first affine transformation can be viewed as a coarse registration process. Based on the coarse registration, the computer device can combine the cell flow field diagram and the preliminary first affine transformation to further obtain the precise affine transformation, i.e., the second affine transformation. T 1. Use the second affine transformation to register the two cell images to obtain an accurate final registration result.
[0072] In this embodiment of the application, when calculating the second affine transformation T Before 1, we can first apply the first affine transformation. T 0. Align the cell flow field maps corresponding to the two cell images. The above process can be represented as:
[0073] in, express The inverse transform of can make and Coarse alignment.
[0074] The above processing procedure involves processing cell images. I 0 andI Cellular flow field diagram 1 flow 0 and flow 1. Alignment can be achieved by moving. flow 1. Obtain flow 1, to make it and flow Zero alignment. Due to T 0 is movement I 0 or flow 0 makes it with I 1 or flow For alignment transformations, the above processing is changed to moving. flow 1. Therefore, adopt T The inverse transform of 0 is... To achieve this.
[0075] Then, the second affine transformation can be calculated. T 1. The second affine transformation mentioned above T 1. The direction-based flow field consistency constraint is satisfied. Thus, it can be determined according to the second affine transformation. T 1. Register the two cell images.
[0076] In this embodiment of the application, the second affine transformation is calculated. T When the value is 1, a direction-based flow field consistency loss function can be constructed, which can be expressed as follows:
[0077] Thus, by calculating the loss value of the direction-based flow field consistency loss function, the optimal second affine transformation can be obtained. T 1. That is:
[0078] At this point, we can obtain the final result that can be used to analyze the two cell images. I 0 and I 1. Final affine transformation for accurate registration T=T 1 (T 0 ) Computer equipment can use this transformation T to register cell images and obtain accurate registration results.
[0079] In this embodiment, a cell center point detection algorithm based on cell flow field is employed. Through vector integration, convergence points with consistent flow directions are located as cell centers, thereby avoiding feature drift caused by illumination or texture changes and solving the problem of unstable feature matching in existing registration methods. Furthermore, by constructing cell shape features based on physical consistency, this embodiment avoids the adverse effects of illumination or texture changes on the registration process, further improving the stability of feature matching. Secondly, addressing the difficulty of registering weak textures and uniform regions, this embodiment establishes a registration constraint term based on directional similarity by statistically analyzing local directional consistency, achieving reliable alignment of weak texture regions. Thirdly, addressing the issues of generalization dependence on data distribution and performance degradation under different imaging conditions in existing deep learning registration methods, this embodiment establishes registration constraints based on cell flow field maps. It utilizes flow field maps to characterize cell geometry and orientation information, without relying on grayscale or texture features, and possesses good invariance to imaging modes and labeling methods. This allows the registration method provided by this embodiment to be widely applied to cell image registration in various scenarios and of various types.
[0080] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0081] To facilitate understanding, a complete example will be used to introduce the cell image registration method provided in the embodiments of this application.
[0082] like Figure 3 The diagram shown is an algorithm flowchart of a cell image registration method provided in an embodiment of this application. Figure 3 The flowchart shown fully illustrates the complete process of registering cell images by combining cell flow field maps. This process can be implemented by various modules of a cell image registration system. For example... Figure 4 The diagram shown is a schematic diagram of a cell image registration system provided in an embodiment of this application; Figure 4 The registration system shown includes a cell structure feature generation module 401, a cell instance acquisition module 402, a feature extraction module 403, a feature matching module 404, and a flow field consistency constraint module 405. These modules can cooperate with each other to execute... Figure 3 The algorithm flow shown achieves accurate registration of cell images. Below, we will combine... Figure 3 and Figure 4 The cell image registration method of the embodiments of this application will be described in detail.
[0083] The cell structure feature generation module 401 is mainly used to process the feature structure of the cell image to be registered, generating corresponding foreground probability maps and cell flow field maps. In this embodiment, the cell image to be registered includes... I 0 and I 1.
[0084] See Figure 3 For cell images to be registered I 0 and I 1. A flow field map generation model can be used to generate a foreground probability map and a cell flow field map for two images. Each cell image has the same dimensions as its corresponding foreground probability map and cell flow field map, which are both height × width.
[0085] The cell instance acquisition module 402 is used to acquire cell instances based on a foreground probability map and a cell flow field map of a cell image. For example, the foreground probability map and the cell flow field map can be input into the cell instance acquisition module to obtain two sets of instance point sets. The steps for acquiring the cell instance point sets include: Step 1: Initialize the coordinate map map Its dimensions are height × width, used to represent the coordinate position of the flow direction of each coordinate point corresponding to the input image. The values in the initial coordinate map represent the coordinate values of each coordinate point, that is:
[0086]
[0087] The above and Two graphs can be represented, one in the x-direction and the other in the y-direction. .
[0088] Step 2: Update the coordinate map map , superimposed cell flow field map flow ,Right now:
[0089] In the above expression, i This indicates the number of updates. At this point, the coordinate point points to the new coordinates along the flow field diagram.
[0090] Step 3: Repeat Step 2, updating N times. For example, N can be 200, or the update can stop when the map stops changing, obtaining the final coordinate map, i.e., the target coordinate map. .
[0091] It can be assumed that at this point, all points on the cells eventually flow to the center point of their respective cells.
[0092] The feature extraction module 403 is mainly used to extract features from two sets of cell instance point sets. In this embodiment, the extracted features can be HU moments. HU moments are 7-dimensional features based on image geometric moments, and can be represented as:
[0093] Accordingly, the features obtained based on the two sets of cell instance point sets can include H 1 and H 2.
[0094] The feature matching module 404 is mainly used to match the extracted features to obtain the target matching combination with the highest similarity.
[0095] Through calculation H 1 and H A similarity score of 2 allows selecting the combination with the highest similarity and removing combinations with similarity scores below a threshold. Calculation... H 1 and H The similarity between two can be expressed by the following formula:
[0096] The flow field consistency constraint module 405 is mainly used to obtain the first affine transformation by coarse registration based on target matching combination. T 0, and based on this, combined with the flow field uniformity constraint term, a more accurate second affine transformation is calculated. T 1. This yields the final affine transformation that can be used for cell image matching. T=T 1 (T 0 ) . Specifically: First, align the cell flow field diagram:
[0097] in, express The inverse transform of can make and Coarse alignment.
[0098] Then, the direction-based flow field consistency loss function is calculated:
[0099] in, yes Inverse transform, and Fine alignment.
[0100] By solving for the optimal T1, the above flow field consistency loss function is minimized:
[0101] Thus, the final registration result transformation is obtained. T=T 1 (T 0 ) .
[0102] Reference Figure 5 The diagram illustrates a cell image registration device according to an embodiment of this application, which may specifically include an acquisition module 501, a generation module 502, a matching module 503, a transformation module 504, and a registration module 505, wherein: The acquisition module 501 is used to acquire feature maps corresponding to the two cell images to be registered, the feature maps including a foreground probability map and a cell flow field map; The generation module 502 is used to generate two sets of cell instance point sets based on the foreground probability map and the cell flow field map; Matching module 503 is used to perform feature matching on the two sets of cell instance point sets to obtain target matching combination; Transformation module 504 is used to determine a first affine transformation of the two cell images based on the target matching combination; The registration module 505 is used to register the two cell images according to the cell flow field map and the first affine transformation.
[0103] In one possible implementation of this application embodiment, the generation module 502 may specifically be used for: Initialize a coordinate map, which is used to represent the coordinate position of the flow direction of each coordinate point corresponding to the input image, and the values in the coordinate map represent the coordinate values of each coordinate point; The coordinate map is updated by overlaying the cell flow field map onto the coordinate map to obtain a target coordinate map, in which any cell point in the target coordinate map flows to the center point of its respective cell; The multiple cell points in the target coordinate map are processed according to the foreground probability map to obtain two sets of cell instance points that correspond one-to-one with the two cell images.
[0104] In this embodiment of the application, the generation module 502 can also be used for: For any of the target coordinate maps, remove the cell center points in the target coordinate map that correspond to each center point whose probability value in the foreground probability map is lower than the first threshold. Based on the remaining cell center points in the target coordinate map, the multiple cell points are merged into multiple point sets; Multiple point sets containing fewer than the second threshold cell points are removed to obtain a cell instance point set.
[0105] In one possible implementation of this application embodiment, the matching module 503 may specifically be used for: The shape features of the two sets of cell instance points are determined respectively, and feature matching is performed on the shape features to obtain multiple matching combinations, and any matching combination has a corresponding similarity. Based on the similarity, a target matching combination is determined from multiple sets of matching combinations.
[0106] In this embodiment of the application, the matching module 503 can also be used for: Calculate the HU invariant moments for the two sets of cell instance points respectively to obtain the shape features corresponding to the two sets of cell instance points.
[0107] In one possible implementation of this application embodiment, the transformation module 504 may specifically be used for: Calculate the displacement of each corresponding cell center point in the target matching combination; A first affine transformation of the two cell images is determined based on the displacement of the multiple cell center points.
[0108] In one possible implementation of this application embodiment, the registration module 505 may specifically be used for: Align the cell flow field maps corresponding to the two cell images according to the first affine transformation; Calculate the second affine transformation, which satisfies the direction-based flow field consistency constraint. The two cell images are registered according to the second affine transformation.
[0109] In this embodiment of the application, the registration module 505 can also be used for: Construct a direction-based flow field consistency loss function; Calculate the optimal second affine transformation that minimizes the loss value of the direction-based flow field consistency loss function.
[0110] This application provides a cell image registration device, which can be a unit with corresponding functions in the computer equipment, cell image registration device, or related equipment described in the foregoing embodiments. Using this device, the steps in the foregoing method embodiments can be implemented.
[0111] As the apparatus embodiments are basically similar to the method embodiments, they are described in a relatively simple manner. For relevant details, please refer to the description in the method embodiment section.
[0112] Reference Figure 6 The diagram illustrates a computer device provided in an embodiment of this application. Figure 6As shown, the computer device 600 in this embodiment includes: a processor 610, a memory 620, and a computer program 621 stored in the memory 620 and executable on the processor 610. When the processor 610 executes the computer program 621, it implements the steps in the various embodiments of the cell image registration method described above, for example... Figure 1 Steps S101 to S105 are shown. Alternatively, when the processor 610 executes the computer program 621, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5 The functions of modules 501 to 505 are shown.
[0113] For example, the computer program 621 can be divided into one or more modules / units, which are stored in the memory 620 and executed by the processor 610 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which can be used to describe the execution process of the computer program 621 in the computer device 600. For example, the computer program 621 can be divided into an acquisition module, a generation module, a matching module, a transformation module, and a registration module, with the specific functions of each module as follows: The acquisition module is used to acquire feature maps corresponding to two cell images to be registered, the feature maps including a foreground probability map and a cell flow field map; The generation module is used to generate two sets of cell instance point sets based on the foreground probability map and the cell flow field map; The matching module is used to perform feature matching on the two sets of cell instance points to obtain a target matching combination; A transformation module is used to determine a first affine transformation of the two cell images based on the target matching combination; The registration module is used to register the two cell images based on the cell flow field map and the first affine transformation.
[0114] The computer device 600 may be a device capable of implementing the steps in the foregoing method embodiments, and may be a desktop computer, cloud server, or other similar device. The computer device 600 may include, but is not limited to, a processor 610 and a memory 620. Those skilled in the art will understand that... Figure 6 This is merely one example of computer device 600 and does not constitute a limitation on computer device 600. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device 600 may also include input / output devices, network access devices, buses, etc.
[0115] The processor 610 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0116] The memory 620 can be an internal storage unit of the computer device 600, such as a hard disk or RAM of the computer device 600. The memory 620 can also be an external storage device of the computer device 600, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the computer device 600. Furthermore, the memory 620 can include both internal and external storage units of the computer device 600. The memory 620 is used to store the computer program 621 and other programs and data required by the computer device 600. The memory 620 can also be used to temporarily store data that has been output or will be output.
[0117] This application also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the methods described in the foregoing embodiments.
[0118] This application also discloses a computer-readable storage medium storing a computer program that, when executed by a computer, implements the methods described in the foregoing embodiments.
[0119] This application also discloses a computer program product, including a computer program that, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.
[0120] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A cell image registration method, characterized in that, include: Obtain feature maps corresponding to the two cell images to be registered, wherein the feature maps include a foreground probability map and a cell flow field map; Based on the foreground probability map and the cell flow field map, two sets of cell instance point sets are generated; Feature matching is performed on the two sets of cell instance points to obtain the target matching combination; Based on the target matching combination, a first affine transformation of the two cell images is determined; Based on the cell flow field map and the first affine transformation, the two cell images are registered.
2. The method according to claim 1, characterized in that, The process of generating two sets of cell instance point sets based on the foreground probability map and the cell flow field map includes: Initialize a coordinate map, which is used to represent the coordinate position of the flow direction of each coordinate point corresponding to the input image, and the values in the coordinate map represent the coordinate values of each coordinate point; The coordinate map is updated by overlaying the cell flow field map onto the coordinate map to obtain a target coordinate map, in which any cell point in the target coordinate map flows to the center point of its respective cell; The multiple cell points in the target coordinate map are processed according to the foreground probability map to obtain two sets of cell instance points that correspond one-to-one with the two cell images.
3. The method according to claim 2, characterized in that, The step of processing multiple cell points in the target coordinate map based on the foreground probability map to obtain two sets of cell instance point sets that correspond one-to-one with the two cell images includes: For any of the target coordinate maps, remove the cell center points in the target coordinate map that correspond to each center point whose probability value in the foreground probability map is lower than the first threshold. Based on the remaining cell center points in the target coordinate map, the multiple cell points are merged into multiple point sets; Multiple point sets containing fewer than the second threshold cell points are removed to obtain a cell instance point set.
4. The method according to any one of claims 1 to 3, characterized in that, The step of performing feature matching on the two sets of cell instance points to obtain the target matching combination includes: The shape features of the two sets of cell instance points are determined respectively, and feature matching is performed on the shape features to obtain multiple matching combinations, and any matching combination has a corresponding similarity. Based on the similarity, a target matching combination is determined from multiple sets of matching combinations.
5. The method according to claim 4, characterized in that, The determination of the shape features of the two sets of cell instance points includes: Calculate the HU invariant moments for the two sets of cell instance points respectively to obtain the shape features corresponding to the two sets of cell instance points.
6. The method according to any one of claims 1 to 3 or 5, characterized in that, The step of determining the first affine transformation of the two cell images based on the target matching combination includes: Calculate the displacement of each corresponding cell center point in the target matching combination; A first affine transformation of the two cell images is determined based on the displacement of the multiple cell center points.
7. The method according to claim 6, characterized in that, The registration of the two cell images based on the cell flow field map and the first affine transformation includes: Align the cell flow field maps corresponding to the two cell images according to the first affine transformation; Calculate the second affine transformation, which satisfies the direction-based flow field consistency constraint. The two cell images are registered according to the second affine transformation.
8. The method according to claim 7, characterized in that, The calculation of the second affine transformation includes: Construct a direction-based flow field consistency loss function; Calculate the optimal second affine transformation that minimizes the loss value of the direction-based flow field consistency loss function.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the computer device to implement the method as described in any one of claims 1 to 8.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is run, the method as described in any one of claims 1 to 8 is performed.