Electron microscope image segmentation model training and interaction analysis method, device and equipment

Abstract

The present disclosure provides an electron microscope image segmentation model training and interaction analysis method, device and equipment, which belongs to the technical fields of biomedicine, image processing and the like. The training method includes: based on the spatial similarity, local heterogeneity and small, narrow and nearly circular geometric characteristics of the segmentation target of the electron microscope image slices, a two-stage micro-supervised training of the segmentation model is performed using a very small number of labeled slices and an appropriate amount of unlabeled slices. The micro-supervised training method provided by the present disclosure greatly reduces the dependence of the training process on segmentation annotations while ensuring the performance of the model. The interaction analysis method includes: based on the segmentation results of three-dimensional organelles, locating the areas where various organelles interact and the interaction information such as the area and number of the interaction areas, to realize interaction analysis in three-dimensional space. The organelle interaction analysis method provided by the present disclosure can realize the analysis of the interaction status of multiple organelles in real space, and improve the accuracy of the interaction information obtained between organelles.

Claims

1. A method for training an electron microscope image segmentation model, characterized in that: The method comprises: Acquire a plurality of labeled image slices and corresponding segmentation labels and a plurality of unlabeled image slices, wherein a ratio of a total number of the labeled image slices divided by a total number of the plurality of image slices is not greater than a first threshold, the plurality of image slices including the plurality of labeled image slices and the plurality of unlabeled image slices, and the plurality of image slices are obtained by scanning a sample object to be photographed along a preset coordinate axis using an electron microscope; Based on the slice similarity of the electron microscope image slices, a plurality of target image slices similar to the labeled image slices are determined from the plurality of unlabeled image slices; based on the small target nature of the target to be segmented in the electron microscope image slices and the local heterogeneity of the electron microscope image, a target cropping size is determined from a plurality of candidate cropping sizes; Determining a training set for training an electron microscope image segmentation model based on the multiple annotated image slices and corresponding segmentation labels, the multiple target image slices, and the target crop size; Based on the training set, performing mutual supervision training based on consistency regularization on the first network model and the second network model in the electron microscope image segmentation model to obtain an intermediate segmentation model; Inputting the plurality of image slices into the intermediate segmentation model respectively to obtain segmentation pseudo labels for the plurality of image slices; Based on the multiple image slices and the segmentation pseudo labels of the multiple image slices, the intermediate segmentation model is self-trained to obtain an electron microscope image segmentation model.

2. The method according to claim 1, characterized in that The method of performing mutual supervision training based on consistency regularization on the first network model and the second network model based on the training set to obtain an intermediate segmentation model includes: Determining a first loss function based on the narrowness and / or near-circularity of the object to be segmented in the electron microscope image slice; Based on the training set and the first loss function, the first network model and the second network model are subjected to mutual supervision training based on consistency regularization to obtain an intermediate segmentation model, where the intermediate segmentation model is the network model with the largest average intersection-over-union ratio between the first network model and the second network model after the mutual supervision training is completed.

3. The method according to claim 1, characterized in that The self-training of the intermediate segmentation model based on the multiple image slices and the segmentation pseudo labels of the multiple image slices to obtain the electron microscope image segmentation model includes: Determining a second loss function based on the narrowness and / or near-circularity of the object to be segmented in the electron microscope image slice; Based on the multiple image slices, the segmentation pseudo labels of the multiple image slices and the second loss function, the intermediate segmentation model is self-trained to obtain the electron microscope image semantic segmentation model, wherein the electron microscope image segmentation model is a semantic segmentation model.

4. The method according to any one of claims 1 to 3, characterized in that The determining of a plurality of target image slices similar to the labeled image slices from the plurality of unlabeled image slices based on the slice similarity of the electron microscope image slices includes: For each annotated image slice, based on the slice similarity of the electron microscope image slice, determine the similarity between the annotated image slice and each unannotated image slice respectively; among the multiple unannotated image slices, the unannotated image slices whose similarity with the annotated image slice is greater than a second threshold are determined as multiple target image slices similar to the annotated image slice.

5. The method according to any one of claims 1 to 3, characterized in that Determining a training set for training an electron microscope image segmentation model based on the multiple annotated image slices and corresponding segmentation labels, the multiple target image slices and / or the target cropping size includes: Based on the target cropping size, cropping each target image slice to obtain a plurality of first sample images corresponding to each target image slice; Based on the target cropping size, cropping each labeled image slice to obtain a plurality of second sample images corresponding to each labeled image slice; Based on the target cropping size, the segmentation label corresponding to each labeled image slice is cut to obtain a segmentation label corresponding to each second sample image; A training set for training an electron microscope image segmentation model is determined, wherein the training set includes a plurality of first sample images corresponding to each target image slice and a plurality of second sample images and corresponding segmentation labels corresponding to each labeled image slice.

6. The method according to any one of claims 1 to 3, characterized in that The method of determining a target cropping size from a plurality of candidate cropping sizes based on the smallness of the target to be segmented in the electron microscope image slice and / or the local heterogeneity of the electron microscope image comprises: Based on the smallness of the target to be segmented in the electron microscope image slice and the local heterogeneity of the electron microscope image, the remaining cropping sizes of the multiple candidate cropping sizes except the maximum cropping size and the minimum cropping size are determined as multiple target cropping sizes; For each target cropping size, each labeled image slice is cut based on the target cropping size to obtain a plurality of third sample images corresponding to each labeled image slice, and the segmentation label of each labeled image slice is cut to obtain a segmentation label corresponding to each third sample image; based on the plurality of third sample images corresponding to each labeled image and the segmentation label of each third sample image, a preset model is fully supervised trained to obtain a trained segmentation model corresponding to the target cropping size; The target cropping size corresponding to the segmentation model with the largest average intersection-over-union ratio among the trained segmentation models corresponding to the multiple target cropping sizes is determined as the target cropping size.

7. A device for training an electron microscope image segmentation model, characterized in that: The device comprises: a first acquisition module configured to acquire a plurality of annotated image slices and corresponding segmentation labels and a plurality of unannotated image slices, wherein a ratio of a total number of the annotated image slices divided by a total number of the plurality of image slices is not greater than a first threshold, the plurality of image slices including the plurality of annotated image slices and the plurality of unannotated image slices, and the plurality of image slices are obtained by scanning a sample object to be photographed along a preset coordinate axis using an electron microscope; a first determining module configured to determine, from the plurality of unlabeled image slices, a plurality of target image slices similar to the labeled image slices based on slice similarity of the electron microscope image slices, and / or to determine, from a plurality of candidate cropping sizes, a target cropping size based on the smallness of the target to be segmented in the electron microscope image slices and / or local heterogeneity of the electron microscope image; a training set determination module configured to determine a training set for training an electron microscope image segmentation model based on the plurality of annotated image slices and corresponding segmentation labels, the plurality of target image slices and / or the target cropping size; a mutual supervision training module configured to perform mutual supervision training based on consistency regularization on the first network model and the second network model in the electron microscope image segmentation model based on the training set to obtain an intermediate segmentation model; a pseudo-label generation module configured to input the plurality of image slices into the intermediate segmentation model respectively to obtain segmentation pseudo-labels for the plurality of image slices; The self-training module is configured to self-train the intermediate segmentation model based on the multiple image slices and the segmentation pseudo labels of the multiple image slices to obtain an electron microscope image segmentation model.

8. A method for analyzing organelle interactions, characterized in that: The method comprises: Obtaining electron microscope image slices of cells, wherein the electron microscope image slices are two-dimensional images obtained by scanning a cell sample along a preset coordinate axis, wherein the cells include various organelles to be analyzed; Inputting the electron microscope image slice into an electron microscope image segmentation model to obtain an organelle segmentation result, wherein the electron microscope image segmentation model is pre-trained using the electron microscope image segmentation model training method according to any one of claims 1 to 6; Performing three-dimensional reconstruction on the organelle segmentation result along the preset coordinate axis to obtain a three-dimensional organelle segmentation result, wherein the three-dimensional organelle segmentation result includes segmentation results of various organelles to be analyzed; Based on the three-dimensional organelle segmentation results, an interaction pairing information set is determined.

9. The method according to claim 8, characterized in that The interaction pairing information set includes interaction pairing information between any first type of organelle and any second type of organelle in the various types of organelles; as well as Determining the interaction pairing information set based on the organelle three-dimensional segmentation result includes: Determining a first interaction region set of the first type of organelles based on the segmentation result of the first type of organelles in the three-dimensional organelle segmentation result, where the first interaction region set includes a plurality of first interaction regions; Determining a second interaction region set of the second type of organelles based on the segmentation result of the second type of organelles in the three-dimensional organelle segmentation result, where the second interaction region set includes a plurality of second interaction regions; calculating the Euclidean distance between each first surface point in each first interaction region and each second surface point in each second interaction region; When a minimum Euclidean distance among all the calculated Euclidean distances is less than the interaction distance threshold, determining a first target surface point and a second target surface point corresponding to the minimum Euclidean distance, where the distance between the first target surface point and the second target surface point is equal to the minimum Euclidean distance; Determining a first target interaction region where the first target surface point is located, a second target interaction region where the second target surface point is located, a first area of ​​the first target interaction region, and a second area of ​​the second target interaction region, where the first target interaction region is a first interaction region among the multiple first interaction regions, and the second target interaction region is a second interaction region among the multiple second interaction regions; The minimum Euclidean distance, the first target surface point, the second target surface point, the first target interaction region, the second target interaction region, the first area, and the second area are determined as interaction pairing information between the first type of organelle and the second type of organelle.

10. The method according to claim 9, characterized in that The determining, based on the segmentation result of the first type of organelles in the three-dimensional organelle segmentation result, a first interaction region set of the first type of organelles, comprises: Determine a first surface contour of the first type of organelle based on the segmentation result of the first type of organelle, where the first surface contour consists of a plurality of first surface points; calculating the Euclidean distance between each first surface point and a second surface point of the second type of organelle; Among all the calculated Euclidean distances, if the Euclidean distance between a first surface point and a second surface point is less than the interaction distance threshold, the first surface point is added to the first interaction region set of the first type of organelle.

11. A device for analyzing organelle interactions, characterized in that: The device comprises: a second acquisition module configured to acquire electron microscope image slices of cells, wherein the electron microscope image slices are two-dimensional images obtained by scanning a cell sample along a preset coordinate axis, wherein the cells include various organelles to be analyzed; an image segmentation module configured to input the electron microscope image slice into an electron microscope image segmentation model to obtain an organelle segmentation result, wherein the electron microscope image segmentation model is pre-obtained using the electron microscope image segmentation model training method according to any one of claims 1 to 6; a three-dimensional reconstruction module configured to perform three-dimensional reconstruction on the organelle segmentation result along the preset coordinate axis to obtain a three-dimensional organelle segmentation result, wherein the three-dimensional organelle segmentation result includes segmentation results of various organelles to be analyzed; The interaction analysis module is configured to determine an interaction pairing information set based on the three-dimensional organelle segmentation result.

12. An electronic device comprising: at least one processor; and at least one memory communicatively connected to the at least one processor, wherein the at least one memory stores instructions that, when executed by the at least one processor, cause the at least one processor to execute the electron microscope image segmentation model training method as described in any one of claims 1 to 6 and / or the organelle interaction analysis method as described in any one of claims 8 to 10.

13. A non-transitory computer-readable storage medium storing instructions, wherein: When the instructions are executed by at least one processor of a computer, the computer executes the electron microscope image segmentation model training method as described in any one of claims 1 to 6 and / or the organelle interaction analysis method as described in any one of claims 8 to 10.

14. A computer program product, comprising a computer program, which, when executed by a processor, implements the electron microscope image segmentation model training method according to any one of claims 1 to 6 and / or the organelle interaction analysis method according to any one of claims 8 to 10.

Images