A multi-level segmentation method for cell electron microscopy images and a quantification method for organelle interactions

Multi-level segmentation and interaction quantification of cell electron microscopy images were performed through the Mask2Former model, which solved the problem of organelle segmentation and interaction analysis in large-scale electron microscopy images, and achieved efficient panoramic segmentation and quantitative analysis, improving segmentation effect and efficiency.

CN120014641BActive Publication Date: 2025-08-05BEIJING UNIV OF POSTS & TELECOMM
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510190268.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2025-01-23
Filing Date
2025-02-20
Publication Date
2025-08-05
Estimated Expiration
2045-02-20

AI Technical Summary

Technical Problem

The prior art is difficult to perform effective organelles segmentation and interaction analysis in large-scale electron microscope images, especially for organelles that cannot be instantiated. The existing methods have large calculations and scarce labeling, resulting in insufficient analysis.

Method used

The Mask2Former model is used to perform multi-level segmentation of cell electron microscope images, combining pixel-level annotation and mask attention mechanisms to achieve panoramic segmentation of two-dimensional and three-dimensional electron microscope images, and quantification of the interaction of the cell organelles is calculated through distance transformation and contact maps.

Benefits of technology

Quantitative analysis of organelle interactions at single-cell level and tissue-wide level in large-scale 2D/3D electron microscopy images is achieved, which improves efficiency, reduces artificial intervention, and has better segmentation effect than existing methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120014641B_ABST
    Figure CN120014641B_ABST
Patent Text Reader

Abstract

The present invention discloses a multi-level segmentation method for cell electron microscopy images and a method for quantifying organelle interactions, including: S1, collecting electron microscopy images of organelles and performing preprocessing; S2, training the Mask2Former model using the preprocessed electron microscopy images; S3, performing multi-level segmentation of organelle electron microscopy images according to the trained Mask2Former model and cell boundary segmentation; S4, performing quantitative analysis of organelle interactions according to the multi-level segmentation results of organelles. Based on the multi-level segmentation results, the present application performs multi-level automatic organelle interaction analysis on large-scale two-dimensional / three-dimensional electron microscopy images, improving efficiency and reducing human intervention.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the fields of image processing and deep learning, and particularly to a method for multi-level segmentation of cell electron microscopy images and quantification of organelle interactions. Background Art

[0002] In two-dimensional electron microscopy images, organelles can be divided into two types according to whether they can be instantiated. One type is instantiable organelles, such as mitochondria, which have regular shapes and obvious instance boundaries and can be distinguished as individual instances. The other type is non-instantiable organelles, such as the endoplasmic reticulum, which do not have regular shapes and cannot be distinguished as individual instances and can only be regarded as a whole.

[0003] Existing organelle segmentation methods mainly use two deep learning models:

[0004] Semantic segmentation models, such as UNet. Semantic segmentation models can divide pixels belonging to the same category in an image, but cannot distinguish individual instances. For example, semantic segmentation can distinguish organelle type A and organelle type B, but within organelle type A, it is impossible to further distinguish instances A1, A2, A3,....

[0005] Instance segmentation models, such as MaskRCNN. Instance segmentation models can identify instantiable organelles and can distinguish individual instance individuals. However, instance segmentation models cannot identify non-instantiable organelles.

[0006] Combining an instance segmentation model and a semantic segmentation model, such as Deepcontact, this method combines the advantages of both, using a semantic segmentation model to identify non-instantiable organelles, while instantiable organelles are handled by the instance segmentation model. However, this method requires training two different models for the two types of organelles, increasing the difficulty of parameter adjustment, and there will inevitably be overlap in the segmentation results of the two models for the same image, increasing the difficulty of post-processing.

[0007] Currently, in the field of cell electron microscopy image processing, there is still a lack of segmentation and organelle interaction analysis methods for large-scale electron microscopy images (i.e., with a resolution of over 100 million pixels).

[0008] Three-dimensional volume electron microscopy image segmentation mainly uses three-dimensional semantic segmentation methods based on 3D CNN, such as 3DUNet. This method can utilize the depth information in space and has a high segmentation accuracy when there is sufficient annotation. However, 3D CNN has a high computational cost, resulting in a low inference speed of the model; on the other hand, complete three-dimensional annotations in existing electron microscopy images are scarce, which to some extent restricts the performance of 3D CNN models. Summary of the Invention

[0009] In view of the above deficiencies in the prior art, the present invention provides a multi-level segmentation method for cell electron microscopy images and a method for quantifying organelle interactions, which solves the problem that existing algorithms can only analyze electron microscopy images of a small local field of view. By manually screening local regions of interest in biological tissues for small-scale local analysis, it is easy to introduce personal biases, resulting in incomplete analysis.

[0010] To achieve the above invention objective, the technical solution adopted by the present invention is as follows: A multi-level segmentation method for cell electron microscopy images and a method for quantifying organelle interactions, including:

[0011] S1. Collect electron microscopy images of organelles and perform preprocessing;

[0012] S2. Use the preprocessed electron microscopy images to train the Mask2Former model;

[0013] S3. Perform multi-level segmentation of organelle electron microscopy images based on the trained Mask2Former model and cell boundary segmentation;

[0014] S4. Perform quantitative analysis of organelle interactions based on the multi-level segmentation results of organelles.

[0015] Further: S101 includes:

[0016] S11. Perform pixel-level annotation on organelles and cell boundaries in the electron microscopy images;

[0017] S12. Convert the annotation information into the COCO format.

[0018] Further: In S12, the Mask2Former model includes: an image encoder, a pixel decoder, and an object decoder;

[0019] The image encoder is used to extract features of the input image in stages;

[0020] The pixel decoder is used to receive the features of the input image in stages, generate a feature pyramid to extract features of pixel points, and obtain a high-resolution feature map;

[0021] The object decoder is used to receive the high-resolution feature map and perform learnable embedding, and replace the standard cross-attention mechanism through a mask attention mechanism to obtain a segmentation mask and classification categories.

[0022] Further: In the pixel decoder, each layer includes a deformable attention module and a feed-forward network. The deformable attention extracted by the deformable attention module is expressed as:

[0023]

[0024] Among them, MSAttn(.) represents deformable attention, z q To query features, represents a two-dimensional reference point, l=1,...,L is the pixel decoder layer index, L is the number of pixel decoder layers, m represents the attention head, k represents the sampled point, K represents the total number of sampled points, Δp mlqk and A mlqk They represent the offset and attention weight of the kth sampling point of the mth attention head in the lth level feature map, Φ l (.) means rescaling the input resolution to the size of the l-th level feature map, f l represents the l-th level feature map of the input feature map, W m and W m ' are all weight matrices.

[0025] Furthermore: In the pixel decoder, the calculation formula of the mask attention mechanism is:

[0026]

[0027] Among them, X l is the learnable embedding of the input of the lth layer of the object decoder, softmax(.) represents the activation function, K l and V l are all image features from the pixel decoder after linear transformation. The superscript T represents the matrix transpose. l is the learnable embedding X of the input to the l-1 layer of the object decoder l-1 The query embedding obtained after linear transformation is M l-1 is the attention mask.

[0028] Furthermore: For 2D organelle electron microscopy images, S3 includes:

[0029] S31, extracting the region of interest from the two-dimensional electron microscope image of the organelle and encapsulating it as an instance object;

[0030] Instance objects include cell bounding boxes, segmentation masks, and cell types;

[0031] S32, inputting the two-dimensional organelle electron microscope image into the trained Mask2Former model to perform two-dimensional organelle panoramic segmentation;

[0032] S33, cropping the two-dimensional organelle panorama segmentation result according to the cell bounding box in the instance object to obtain the bounding box range of the two-dimensional organelle;

[0033] S34. Perform bitwise multiplication on the segmentation mask in the instance object and the bounding box range of the organelle to obtain a two-dimensional organelle multi-level segmentation result.

[0034] Furthermore: For two-dimensional organelle electron microscopy images, S4 includes:

[0035] S401. Select organelle A and organelle B from the multi-level segmentation results of organelles, and obtain the segmentation masks of organelle A and organelle B;

[0036] S402. Extract the edge information of organelle A and organelle B respectively according to the segmentation masks of organelle A and organelle B;

[0037] S403. According to the edge information of organelle A and organelle B, use distance transformation or pixel-by-pixel traversal to calculate the shortest distance between organelle A and organelle B, and generate a distance map and a contact map between organelle A and organelle B;

[0038] Among them, the distance map is used to record the distance from each pixel point in organelle A to the nearest organelle B, forming a global distance distribution;

[0039] The contact map is used to mark the contact area between organelle A and organelle B, and quantify the distribution characteristics between contact points;

[0040] S404. According to the distance map and contact map between organelle A and organelle B, perform distance characteristic calculation, contact characteristic calculation and organelle geometric attribute calculation;

[0041] S405. Structure the data formats of the distance characteristic calculation results, contact characteristic calculation results and organelle geometric attribute calculation results as the interaction quantification analysis results of organelle A and organelle B.

[0042] Furthermore: For three-dimensional organelle electron microscopy images, S3 includes:

[0043] S311. Layer by layer extract the regions of interest in the profiles of three-dimensional organelle electron microscopy images;

[0044] S312. Use a tracking algorithm to perform z-axis matching on the regions of interest in each layer of the profile, and establish a three-dimensional instance object;

[0045] The three-dimensional instance object includes a three-dimensional cell bounding box, a three-dimensional segmentation mask and a cell type;

[0046] S312. Input the profiles of three-dimensional organelle electron microscopy images layer by layer into the trained Mask2Former model for three-dimensional organelle panoramic segmentation;

[0047] S313. Use the three-dimensional cell bounding box and three-dimensional segmentation mask in the three-dimensional instance object to crop the regions of interest from the three-dimensional organelle panoramic segmentation results, and obtain the multi-level segmentation results of three-dimensional organelle electron microscopy images.

[0048] Furthermore: For three-dimensional organelle electron microscopy images, S4 includes:

[0049] S411. Perform block processing on the multi-level segmentation results of three-dimensional organelle electron microscopy images;

[0050] S412. In each block, extract the surface voxels of each type of organelle;

[0051] S413. Determine the contact area and contact points between the surface voxels of the organelles;

[0052] S414. Summarize the contact points, contact areas, and organelle areas of all blocks, and calculate the contact ratio to complete the quantitative analysis of organelle interactions in three-dimensional organelle electron microscopy images.

[0053] The beneficial effects of the present invention are as follows:

[0054] 1. This solution can perform quantitative analysis of organelle interactions at the single-cell level / whole-tissue level in large-scale two-dimensional / three-dimensional electron microscopy images simultaneously;

[0055] 2. Based on the multi-level segmentation results, automatically screen out all regions of interest in the image, improving efficiency and reducing human intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flowchart of the multi-level segmentation of cell electron microscopy images and the quantitative method of organelle interactions.

[0057] Figure 2 is a schematic diagram of the multi-level segmentation and interaction quantification process of large-scale two-dimensional electron microscopy images.

[0058] Figure 3 is a schematic diagram of the multi-level segmentation and interaction quantification process of three-dimensional electron microscopy images. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] The following describes the specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions made using the concept of the present invention are within the scope of protection.

[0060] As Figure 1 shown, in an embodiment of the present invention, a method for multi-level segmentation of cell electron microscopy images and quantification of organelle interactions is provided, including:

[0061] S1. Collect electron microscopy images of organelles and perform preprocessing;

[0062] S2. Train the Mask2Former model using the preprocessed electron microscopy images;

[0063] S3. Perform multi-level segmentation of the organelle electron microscopy images based on the trained Mask2Former model and cell boundary segmentation;

[0064] S4. Conduct quantitative analysis of organelle interactions based on the multi-level segmentation results of organelles.

[0065] Specifically, S1 includes:

[0066] S11. Perform pixel-level annotation on the organelles and cell boundaries in the electron microscopy images;

[0067] S12. Convert the annotation information into the COCO format.

[0068] In S12, the Mask2Former model is a typical "encoder-decoder" structure, including: an image encoder, a pixel decoder, and an object decoder;

[0069] The image encoder is used to extract the features of the input image in stages;

[0070] The pixel decoder is used to receive the features of the input image in stages and generate a feature pyramid to extract features for pixel points, obtaining a high-resolution feature map;

[0071] The object decoder is used to receive the high-resolution feature map and perform learnable embedding, replacing the standard cross-attention mechanism through the mask attention mechanism to obtain the segmentation mask and classification categories.

[0072] Specifically: For an image with a size of H0×W0, it is intended to use an image encoder, such as ResNet and swin-transformer, etc., to extract the features of the image.

[0073] The feature maps generated by the last L stages of the image encoder are respectively input into the pixel decoder to generate a feature pyramid. Each layer of the pixel decoder consists of a deformable attention module and a feed-forward network. The advantage of deformable attention is that it can focus the attention on the sampling points around specific pixels, with lower computational cost and easier convergence compared to the standard attention mechanism.

[0074] For a given input feature map f, query feature z q and two-dimensional reference points The deformable attention is calculated by the following formula:

[0075]

[0076] Among them, MSAttn(.) represents deformable attention, l = 1, ..., L is the index of the pixel decoder layer number, L is the number of pixel decoder layers, m represents the attention head, k represents the sampled points, K represents the total number of sampled points, and Δp mlqk and A mlqk respectively represent the offset and attention weight of the k-th sampled point in the l-th level feature map of the m-th attention head, and Φ l (.) represents rescaling the input resolution to the size of the l-th level feature map, and f l represents the l-th level feature map of the input feature map, and W m and W′ m are both weight matrices, and W′ m is used to perform a linear transformation on the input feature map, and W m is used to perform a linear transformation on the output features of the deformable attention.

[0077] Input the obtained high-resolution feature map into the object decoder to train the learnable embedding, and obtain the segmentation mask and classification category. In the object decoder, the mask attention mechanism is used to replace the standard cross-attention mechanism, so that the attention is concentrated on the predicted mask area, and the model convergence speed is accelerated. The calculation formula of the mask attention mechanism is as follows:

[0078]

[0079] Among them, X l is the learnable embedding input to the l-th layer of the object decoder, softmax(.) represents the activation function, K l and V l are both image features from the pixel decoder after linear transformation, the superscript T represents the matrix transpose, and Q l is the query embedding obtained after linear transformation of the learnable embedding X l-1 input to the (l - 1)-th layer of the object decoder, and M l-1 is the attention mask, and the value corresponding to its position (x, y) on the feature map is calculated as follows:

[0080]

[0081] Among them, F l-1 (x, y) = 1 is the binarized output of the mask prediction in the l-th layer of the object decoder.

[0082] As Figure 2 shown, for two-dimensional organelle electron microscopy images, S3 includes:

[0083] S31. Extract the region of interest in the two-dimensional organelle electron microscopy image and encapsulate it as an instance object;

[0084] For the segmentation of the region of interest (i.e., cell boundary segmentation), since it is an instance segmentation task (i.e., only segmenting instantiable cell individuals), through the open-source sahi framework, in the form of a sliding window, the cell regions in the image are identified and extracted window by window, and after post-processing with nms (non-maximum suppression algorithm), redundant instances are removed. Then, for instances across windows, the IoU threshold is used for merging. If the IoU of adjacent instances is greater than the threshold of 0.4, the two are merged; otherwise, they are not merged. All the extracted regions of interest are saved in a set of encapsulated instance objects, and each individual object in this set contains the bounding box, segmentation mask, confidence, and cell type of a single region of interest. By setting the confidence and size threshold of the bounding box, regions with low confidence and too small area are filtered out to improve the efficiency of subsequent processing.

[0085] Specifically, the instance object includes a cell bounding box, a segmentation mask, and a cell type;

[0086] S32. Input the two-dimensional electron microscopy image of the organelle into the trained Mask2Former model for two-dimensional organelle panoptic segmentation;

[0087] S33. Crop the two-dimensional organelle panoptic segmentation result according to the cell bounding box in the instance object to obtain the bounding box range of the two-dimensional organelle;

[0088] S34. Multiply the segmentation mask in the instance object and the bounding box range of the organelle bit by bit to obtain the two-dimensional organelle multi-level segmentation result.

[0089] For the two-dimensional electron microscopy image of the organelle, S4 includes:

[0090] S401. Select organelle A and organelle B from the two-dimensional organelle multi-level segmentation result and obtain the segmentation masks of organelle A and organelle B;

[0091] S402. Extract the edge information of organelle A and organelle B according to the segmentation masks of organelle A and organelle B respectively;

[0092] S403. According to the edge information of organelle A and organelle B, using distance transform or pixel-by-pixel traversal, calculate the shortest distance between organelle A and organelle B, and generate a distance map and a contact map between organelle A and organelle B;

[0093] Among them, the distance map is used to record the distance from each pixel point in organelle A to the nearest organelle B, forming a global distance distribution;

[0094] The contact map is used to mark the contact area between organelle A and organelle B and quantify the distribution characteristics between contact points;

[0095] S404. Calculate distance characteristics, contact characteristics, and organelle geometric properties based on the distance map and contact map between organelle A and organelle B;

[0096] S405. Structure the data formats of the calculation results of distance characteristics, contact characteristics, and organelle geometric properties as the quantitative analysis results of the interaction between organelle A and organelle B.

[0097] Taking the instantiable organelle mitochondria and the non-instantiable organelle endoplasmic reticulum as examples, the distance map (Distance Map): First, traverse each pixel point in the endoplasmic reticulum region and check whether its neighborhood within the specified range contains mitochondria pixels. If there are mitochondria pixels in the neighborhood, calculate the Euclidean distance and record the minimum distance as the nearest distance value for that point. The minimum distance values of all points are aggregated to form a distance map, which is used to quantify the spatial relationship between the endoplasmic reticulum and mitochondria.

[0098] Contact map (Contact Map): First, traverse each pixel point in the mitochondria region and check whether there are endoplasmic reticulum pixels in its neighborhood. If an endoplasmic reticulum pixel is found and the distance is less than or equal to the threshold, mark that mitochondria pixel as a contact point and record its distance value in the contact map. If the mitochondria pixel and the endoplasmic reticulum pixel completely overlap (distance is 0), it is directly marked as a contact point.

[0099] As Figure 3 shown, for three-dimensional organelle electron microscopy images, S3 includes:

[0100] S311. Extract the regions of interest in the cross-sections of three-dimensional organelle electron microscopy images layer by layer;

[0101] S312. Use a tracking algorithm to perform z-axis matching on the regions of interest in each cross-section and establish three-dimensional instance objects;

[0102] To solve the problem of three-dimensional ROI extraction, this application relies on an intuitive idea: in adjacent cross-sections, if the 2D instance in the current cross-section does not match the instance in the previous cross-section, it is considered a new 3D instance. This problem can be represented as a weighted bipartite graph matching problem. Consider the instances in different cross-sections as nodes in the bipartite graph, and the intersection over union (IoU) between instance masks as the weights of the edges. Select the Hungarian algorithm as the bipartite graph matching algorithm. First, calculate the IoU between instances and construct an IoU matrix as the cost matrix for the Hungarian matching algorithm. If the IoU between a pair of instances is below the set threshold, force it to be set to 0. Then, apply the Hungarian matching algorithm to select the optimal matching scheme. Since the goal is to select the solution with the maximum cost or the highest IoU, which is opposite to the original goal of the algorithm, we calculate the negative of the IoU matrix.

[0103] The three-dimensional instance object includes a three-dimensional cell bounding box, a three-dimensional segmentation mask, and cell types;

[0104] S312. Input the cross-sections of the three-dimensional organelle electron microscopy images layer by layer into the trained Mask2Former model for three-dimensional organelle panoptic segmentation;

[0105] S313. Use the three-dimensional cell bounding box and three-dimensional segmentation mask in the three-dimensional instance object to crop the region of interest from the three-dimensional organelle panoptic segmentation result, obtaining the multi-level segmentation result of the three-dimensional organelle electron microscopy images.

[0106] For the three-dimensional organelle electron microscopy images, S4 includes:

[0107] S411. Perform block processing on the multi-level segmentation result of the three-dimensional organelle electron microscopy images;

[0108] S412. In each block, extract the surface voxels of each organelle;

[0109] The specific method is: perform an erosion operation (binary_erosion) on the voxels of each instance, and calculate the difference before and after erosion to extract the surface voxels;

[0110] S413. Determine the contact area and contact points between the surface voxels of the organelles;

[0111] The specific method is: use cKDTree in the python scientific computing library scipy to construct a KD tree, and calculate the distance between the surface voxels of two organelles. For each distance value, determine whether the contact condition is met (i.e., whether it is less than or equal to the given maximum distance max_distance), and record the contact points;

[0112] S414. Aggregate the contact points, contact areas, and organelle areas of all blocks, and calculate the contact ratio to complete the quantitative analysis of the interactions of the three-dimensional organelle electron microscopy images.

[0113] In an embodiment of the present invention, the quantitative result comparison of the method of the present application and UNet in organelle segmentation (as shown in Table 1) and the quantitative result comparison of the present application and MaskRCNN in the segmentation of the region of interest (as shown in Table 2) are provided;

[0114] Table 1 Quantitative result comparison of the contrast experiment between the present application and UNet in organelle segmentation

[0115]

[0116] Table 2 Quantitative result comparison of the contrast experiment between the present application and MaskRCNN in the segmentation of the region of interest

[0117]

[0118] Through the above comparison, it shows that the segmentation effect of the organelles by itself is excellent, leading in multiple metrics on the private dataset, and capable of simultaneously performing quantitative analysis of organelle interactions at the single-cell level / whole-tissue level in large-scale two-dimensional / three-dimensional electron microscopy images.

[0119] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for multi-level segmentation of cell electron microscopy images and quantification of organelle interactions, characterized in that: include: S1. Collect electron microscope images of organelles and perform preprocessing; S2, use the preprocessed electron microscope images to train the Mask2Former model; S3. Perform multi-level segmentation of organelle electron microscopy images based on the trained Mask2Former model and cell boundaries. S4. Based on the results of organelle multi-level segmentation, quantitative analysis of organelle interactions was performed; In S2, the Mask2Former model includes: image encoder, pixel decoder and object decoder; The image encoder is used to extract features of the input image in stages; The pixel decoder is used to receive the features of the input image in stages and generate a feature pyramid to extract features from the pixels to obtain a high-resolution feature map. The object decoder is used to receive high-resolution feature maps and perform learnable embedding, replacing the standard cross attention mechanism with a mask attention mechanism to obtain segmentation masks and classification categories; For 2D organelle electron microscopy images, S3 includes: S31, extracting the region of interest from the two-dimensional electron microscope image of the organelle and encapsulating it as an instance object; Instance objects include cell bounding boxes, segmentation masks, and cell types; S32, inputting the two-dimensional organelle electron microscope image into the trained Mask2Former model to perform two-dimensional organelle panoramic segmentation; S33, cropping the two-dimensional organelle panorama segmentation result according to the cell bounding box in the instance object to obtain the bounding box range of the two-dimensional organelle; S34, performing bitwise multiplication on the segmentation mask in the instance object and the bounding box range of the organelle to obtain a two-dimensional organelle multi-level segmentation result; For 3D organelle electron microscopy images, S3 includes: S311, extracting the region of interest from the electron microscope image section of the three-dimensional organelle layer by layer; S312, using a tracking algorithm to perform z-axis matching on the region of interest in each cross-section to create a three-dimensional instance object; The 3D instance object includes 3D cell bounding box, 3D segmentation mask and cell type; S312, inputting the cross-section of the 3D organelle electron microscope image layer by layer into the trained Mask2Former model to perform 3D organelle panoramic segmentation; S313. Use the 3D cell bounding box and 3D segmentation mask in the 3D instance object to crop a region of interest from the 3D organelle panoramic segmentation result to obtain a multi-level segmentation result of the 3D organelle electron microscope image.

2. The method for multi-level segmentation of cell electron microscopy images and quantification of organelle interactions according to claim 1, characterized in that: S1 includes: S11. Pixel-level annotation of organelles and cell boundaries in electron microscope images; S12. Convert the annotation information into COCO format.

3. The method for multi-level segmentation of cell electron microscopy images and quantification of organelle interactions according to claim 1, characterized in that: In the pixel decoder, each layer includes a deformable attention module and a feedforward network. The deformable attention extracted by the deformable attention module is represented as: in, represents deformable attention, To query features, represents a two-dimensional reference point, is the pixel decoder layer index, L is the number of pixel decoder layers, m represents the attention head, k represents the sampling point, K represents the total number of sampling points, and Respectively represent m The attention head l The first level feature map k The offset and attention weight of the sampling points, Rescales the input to the resolution of l The size of the level feature map, Represents the first l Level feature map, and are all weight matrices.

4. The method for multi-level segmentation of cell electron microscopy images and quantification of organelle interactions according to claim 1, characterized in that: In the object decoder, the calculation formula of the mask attention mechanism is: in, For object decoder l learnable embeddings of layer inputs, represents the activation function, and are all image features from the pixel decoder after linear transformation, and the superscript T represents matrix transposition. For object decoder l -1 layer of learnable embeddings for input The query embedding obtained after linear transformation is is the attention mask.

5. The method for multi-level segmentation of cell electron microscopy images and quantification of organelle interactions according to claim 1, characterized in that: For 2D organelle electron microscopy images, S4 includes: S401. Select organelles from organelle multi-level segmentation results A and organelles B , and obtain organelles A and organelles B Segmentation mask of S402, according to the organelles A and organelles B The segmentation mask is used to extract the organelles A and organelles B Edge information; S403, according to organelles A and organelles B The edge information of the cell is calculated by using distance transformation or pixel traversal. A and organelles B The shortest distance between them and the generation of organelles A and organelles B distance map and contact map between them; Among them, the distance map is used to record the organelles A From each pixel to the nearest organelle B distance, forming a global distance distribution; Contact maps are used to label organelles A and organelles B The contact area and quantify the distribution characteristics between contact points; S404, according to organelles A and organelles B The distance map and contact map between them are used to calculate distance characteristics, contact characteristics and organelle geometric properties; S405, the distance characteristic calculation results, the contact characteristic calculation results and the organelle geometric property calculation results are structured into a data format as the organelle A and organelles B The results of quantitative interaction analysis.

6. The method for multi-level segmentation of cell electron microscopy images and quantification of organelle interactions according to claim 1, characterized in that: For 3D organelle electron microscopy images, S4 includes: S411, performing block processing on the multi-level segmentation results of the three-dimensional organelle electron microscope image; S412. In each block, extract the surface voxels of each organelle; S413, determining the contact area and contact points between surface voxels of the organelle; S414. Summarize the contact points, contact areas, and organelle areas of all blocks, and calculate the contact ratio to complete the quantitative analysis of the interaction between three-dimensional organelle electron microscopy images.

Citation Information

Patent Citations

  • Training method of cell electron microscope image segmentation model and organelle interaction analysis method

    CN114612738A

  • Electron microscope image segmentation model training and interaction analysis method, device and equipment

    CN115775255A