A vesicle segmentation method and system based on cryo-em tomogram

Abstract

The application provides a vesicle segmentation method and system based on a cryo-EM tomography image, and is applied to the technical field of intelligent processing and analysis of biomedical images.The application carries out preprocessing on the cryo-EM tomography image through deconvolution and missing wedge recovery, combines parallel inference of a double-path pre-training segmentation model, and then carries out multi-stage optimization processing such as fusion, ellipsoid fitting and morphological filtering, so that the accuracy and morphological expression ability of vesicle segmentation are effectively improved, and the technical problems of insufficient vesicle segmentation precision and missing morphological parameters of the cryo-EM tomography image are solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent processing and analysis technology of biomedical images, and in particular to a vesicle segmentation method and system based on cryo-electron microscopy tomography images. Background Technology

[0002] Cryo-electron tomography (cryo-ET), as an advanced structural biology imaging technique, enables direct observation of the three-dimensional morphology of intracellular macromolecules and subcellular structures under near-physiological conditions. It is particularly suitable for studying the in-situ spatial distribution and morphological characteristics of key neurotransmission elements such as synaptic vesicles. Through cryo-ET, researchers can reveal the dynamic behavior of vesicles within neurons and their interactions with other cellular components, thus advancing the fields of neuroscience and cell biology.

[0003] However, the inherent low signal-to-noise ratio and missing wedge artifacts in cryo-ET data significantly increase the difficulty of vesicle segmentation in the images. Furthermore, the high heterogeneity of vesicle morphology, including the diversity in size, shape, and spatial arrangement, also presents technical challenges for high-precision 3D segmentation.

[0004] Therefore, how to accurately segment vesicles from cryo-electron microscopy tomography images and obtain the morphological parameters of the corresponding vesicles has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, the present invention provides a vesicle segmentation method and system based on cryo-electron microscopy tomography images to overcome or at least partially solve the above problems. The technical solution is as follows:

[0006] A vesicle segmentation method based on cryo-electron microscopy tomography images includes:

[0007] Obtain cryo-electron tomographic images containing vesicle structures;

[0008] The cryo-electron microscopy tomographic images are preprocessed to generate deconvolution images and missing wedge recovery images;

[0009] The deconvolution image and the missing wedge recovery image are respectively input into the corresponding pre-trained segmentation model for parallel inference to obtain the first vesicle segmentation result and the second vesicle segmentation result, respectively.

[0010] The first vesicle segmentation result and the second vesicle segmentation result are fused to obtain the first fused vesicle segmentation result;

[0011] Ellipsoid fitting and morphological filtering are performed on the first fusion vesicle segmentation result to obtain the second fusion vesicle segmentation result, which includes multiple segmented vesicle instances and their morphological parameters.

[0012] A vesicle segmentation system based on cryo-electron microscopy tomography images includes: a cryo-electron microscopy tomography image acquisition module, a preprocessing module, a dual-path segmentation module, a fusion module, and a morphology processing module;

[0013] The cryo-electron microscopy tomography image acquisition module is used to acquire cryo-electron microscopy tomography images containing vesicle structures;

[0014] The preprocessing module is used to preprocess the cryo-electron microscopy tomographic images to generate deconvolution images and missing wedge recovery images;

[0015] The dual-path segmentation module is used to input the deconvolution image and the missing wedge recovery image into the corresponding pre-trained segmentation model for parallel inference, and obtain the first vesicle segmentation result and the second vesicle segmentation result respectively.

[0016] The fusion module is used to fuse the first vesicle segmentation result and the second vesicle segmentation result to obtain a first fused vesicle segmentation result.

[0017] The morphology processing module is used to perform ellipsoid fitting and morphology filtering on the first fusion vesicle segmentation result to obtain a second fusion vesicle segmentation result, which includes multiple segmented vesicle instances and their morphological parameters.

[0018] By employing the above technical solution, this invention provides a vesicle segmentation method and system based on cryo-electron microscopy tomography images. The method obtains cryo-electron microscopy tomography images containing vesicle structures; preprocesses the cryo-electron microscopy tomography images to generate deconvolution images and missing wedge restoration images; inputs the deconvolution images and missing wedge restoration images into corresponding pre-trained segmentation models for parallel inference, respectively obtaining a first vesicle segmentation result and a second vesicle segmentation result; fuses the first and second vesicle segmentation results to obtain a first fused vesicle segmentation result; and performs ellipsoid fitting and morphological filtering on the first fused vesicle segmentation result to obtain a second fused vesicle segmentation result, which includes multiple segmented vesicle instances and their morphological parameters. This invention, through preprocessing cryo-electron microscopy tomography images with deconvolution and missing wedge restoration, combined with parallel inference using a dual-path pre-trained segmentation model, and further optimized through multi-stage processes such as fusion, ellipsoid fitting, and morphological filtering, effectively improves the accuracy and morphological representation ability of vesicle segmentation, thereby solving the technical problems of insufficient accuracy and missing morphological parameters in cryo-electron microscopy tomography image vesicle segmentation.

[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0021] Figure 1 A schematic diagram illustrating the overall workflow of vesicle segmentation provided in an embodiment of the present invention is shown;

[0022] Figure 2 A schematic diagram of the functional layout of the graphical user interface provided in an embodiment of the present invention is shown;

[0023] Figure 3 The diagram illustrates a flowchart of one embodiment of the vesicle segmentation method based on cryo-electron microscopy tomography provided by this invention.

[0024] Figure 4 A schematic diagram of the deconvolution preview interface in the image user interface provided in an embodiment of the present invention is shown;

[0025] Figure 5 The diagram shows a specific implementation of step S340 in the vesicle segmentation method based on cryo-electron microscopy tomography provided in this invention.

[0026] Figure 6 A schematic diagram of the fitting process for three-dimensional ellipsoid fitting provided in an embodiment of the present invention is shown;

[0027] Figure 7 The flowchart illustrates another embodiment of the vesicle segmentation method based on cryo-electron microscopy tomography provided in this invention.

[0028] Figure 8 This diagram illustrates a masking effect in the third fusion vesicle segmentation result provided by an embodiment of the present invention.

[0029] Figure 9 A schematic diagram illustrating the effect of interactive correction provided in an embodiment of the present invention is shown. Detailed Implementation

[0030] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0031] Currently, vesicle segmentation techniques mainly include the following categories:

[0032] 1. Manual annotation and semi-automatic segmentation methods: Segmentation is achieved through manual frame-by-frame annotation or manual adjustment based on thresholds. These methods are not only time-consuming and have low throughput, but the results are also easily affected by the operator's experience and subjective judgment, making it difficult to meet the needs of large-scale, high-throughput data analysis;

[0033] 2. Traditional image processing algorithms, such as template matching and edge detection, are usually extremely sensitive to noise and have difficulty adapting to the diversity of vesicle morphology and complex backgrounds in cryo-ET images, resulting in large segmentation errors.

[0034] 3. Deep Learning-Based Segmentation Models: In recent years, deep learning technology has been widely used in biological image segmentation and has achieved certain results. However, most existing models have the following limitations: a) Most models are trained based on two-dimensional slices, ignoring the continuity and overall structural features of vesicles in three-dimensional space, which limits the segmentation accuracy; b) Many methods assume that vesicles are ideal spheres, which makes it difficult to accurately reflect the actual ellipsoidal shape of vesicles, leading to morphological analysis distortion; c) In images with CTF blur and missing wedge artifacts, the model segmentation performance drops significantly, and accuracy is difficult to guarantee; d) The models have a high barrier to entry and generally lack ease of use, making it difficult for ordinary researchers without programming and deep learning backgrounds to apply them directly.

[0035] In summary, existing cryo-ET vesicle 3D segmentation and morphological analysis techniques still face problems such as insufficient accuracy, distorted morphological description, and high operational complexity.

[0036] Based on this, this invention provides a vesicle segmentation method based on cryo-electron microscopy tomography images. First, by combining specific image preprocessing techniques with a dual-path 3D deep learning segmentation model, the method effectively overcomes the effects of low signal-to-noise ratio and missing wedge artifacts, improving segmentation robustness and accuracy. Second, an ellipsoidal morphology-aware post-processing algorithm is introduced to accurately preserve the true 3D morphology of vesicles and their ellipsoidal structural features, enhancing the reliability of morphological analysis. Simultaneously, a user-friendly integrated platform is provided, requiring no programming or deep learning expertise, achieving integrated image preprocessing, deep learning segmentation, morphological post-processing, and interactive 3D visualization analysis, greatly improving user experience and application breadth. Therefore, based on the above technical solutions, this invention can achieve high-precision, automated, and high-throughput processing of cryo-ET vesicle 3D segmentation and morphological analysis, providing strong technical support and tool guarantees for vesicle-related research in neuroscience, cell biology, and structural biology.

[0037] For a complete workflow of vesicle segmentation provided in this embodiment of the invention, please refer to [link / reference]. Figure 1 , Figure 1 In this context, "1" represents the preprocessing process, "2" represents the dual-path 3D segmentation and decision fusion process, "3" represents the ellipsoid fitting and morphological postprocessing process, and "4" represents the interactive verification and correction process.

[0038] Preprocessing: First, input the original cryo-ET tomographic map (e.g., Figure 1 In the middle "a"), CTF deconvolution processing is performed sequentially to obtain a tomographic map with higher signal-to-noise ratio and enhanced contrast (see Figure 1 The missing wedge recovery algorithm was then applied to further compensate for the loss of structural information, resulting in a more complete three-dimensional tomographic map of the structure (see [reference]). Figure 1 (c in the middle).

[0039] Dual-path 3D segmentation and decision fusion process: Two types of preprocessed tomographic images are input into the dual-path 3D segmentation model. 3D vesicle segmentation is performed independently on the CTF deconvolution image and the missing wedge recovery image. Then, the segmentation masks output from the two paths are fused to obtain the comprehensive segmentation result (see...). Figure 1 The "d" in the middle is used to improve the robustness and recall of segmentation.

[0040] Ellipsoid fitting and morphological post-processing: The fused segmentation results undergo ellipsoid fitting and morphological post-processing, including ellipsoid parameter fitting and outlier filtering, to obtain an optimized segmentation map (see...). Figure 1 The “e” in the middle provides high-fidelity three-dimensional morphological data for subsequent statistical analysis.

[0041] Interactive verification and correction process: Users interactively verify and correct the automatic segmentation results in the graphical user interface (GUI), which updates and outputs the final corrected segmentation map in real time (see...). Figure 1 The "f" in the middle ensures the accuracy of each vesicle instance.

[0042] Optionally, the functional layout of the graphical user interface provided in the embodiments of the present invention is as follows: Figure 2 As shown, it may include the following functional areas: basic layer operation area (see...) Figure 2 The box containing "1" in the middle), the main tool menu area (see...) Figure 2 The box containing "2" in the middle), the three orthogonal view display area (see Figure 2 The box containing "3" in the middle), the status and progress display area (see...) Figure 2 The box containing "4" and the batch list area (see [link]). Figure 2 (The box containing "5").

[0043] like Figure 3 The diagram shows a flowchart of one embodiment of the vesicle segmentation method based on cryo-electron microscopy tomography provided by this invention. The method may include:

[0044] S300: Obtain cryo-electron tomography images containing vesicle structures.

[0045] Among them, vesicle structures refer to spherical or ellipsoidal small bodies enclosed by membranes within cells, which are used to store and transport substances.

[0046] Cryo-electron tomography images refer to three-dimensional data of cellular ultrastructure obtained using cryo-electron tomography (cryo-ET) technology, including cellular structures such as vesicles.

[0047] Specifically, embodiments of the present invention can import cryo-electron microscopy tomographic images containing vesicle structures into a unified data interface system, and ensure that the spatial scale and format of the images are consistent, providing a basis for subsequent processing.

[0048] As examples, embodiments of the present invention can load user-specified cryo-electron computed tomography (cryo-CT) raw data files via a graphical user interface or command-line interface. These files can be 3D data in MRC or REC format, recording 3D tomographic images of biological samples (such as neural synapses) after imaging and reconstruction at different tilt angles. By automatically reading and parsing the file header information, the 3D tomographic image data containing information on subcellular structures such as vesicles is loaded into memory as the initial input for the entire vesicle segmentation process.

[0049] To achieve batch processing and data management, all cryo-electron microscopy tomographic images to be analyzed can be organized according to a unified file structure. For example, a main folder (main_folder) can contain multiple subfolders (such as p545, p565, etc.), each containing the corresponding tomographic image file, such as p545_wbp.rec, p545_wbp_corrected.mrc, etc. In addition, the main folder should also contain a task index file named segVesicle.batch, which lists all tomographic image files to be processed, serving as a unified management basis for segmentation tasks. Users can automatically generate this index file using the "Create segVesicle.batch" button on the graphical user interface. The interface can display all task files, and users can mark the processing status of tasks using checkboxes and set task priorities using the "heart" function, facilitating task progress management and priority adjustment.

[0050] During the data loading process, after the user clicks the "Open Original Tomo" button, the system will automatically perform voxel resampling on the selected tomographic images, adjusting all images to a standard voxel size of 17.14 Å. This ensures that image data from different devices or resolutions are consistent in spatial scale, thereby guaranteeing the accuracy and comparability of subsequent segmentation and analysis.

[0051] S310. Preprocess the cryo-electron microscopy tomography images to generate deconvolution images and missing wedge restoration images.

[0052] Among them, the deconvolution image refers to the processed image obtained by using the CTF deconvolution algorithm to suppress noise and enhance contrast in the original tomographic image.

[0053] Among them, missing wedge restoration image refers to using deep learning algorithms to compensate for the information lost in the tilted perspective of the tomographic image, and generate a more complete three-dimensional image.

[0054] Specifically, embodiments of the present invention can perform parallel dual-modal preprocessing on cryo-electron microscopy tomographic images: on the one hand, image deconvolution is performed by calculating the contrast transfer function to suppress noise and enhance structural contrast; on the other hand, missing wedge artifacts are restored based on a deep learning model to compensate for the loss of structural information caused by data acquisition limitations, thereby generating two optimized three-dimensional images that highlight different aspects of structural features and provide differentiated high-quality input for subsequent dual-path segmentation.

[0055] Furthermore, embodiments of the present invention can address the problems of low signal-to-noise ratio and severe missing wedge artifacts in cryo-electron microscopy tomography images by applying deconvolution algorithms optimized for different imaging conditions to suppress noise and enhance contrast in the images. At the same time, a deep learning model trained with a large amount of vesicle tomography data is used to recover the missing wedge information, generating two processing results: a deconvolution image and a missing wedge restored image.

[0056] S320. Input the deconvolution image and the missing wedge recovery image into the corresponding pre-trained segmentation model for parallel inference to obtain the first vesicle segmentation result and the second vesicle segmentation result respectively.

[0057] Among them, the pre-trained segmentation model refers to a three-dimensional deep learning network trained on a large amount of vesicle labeled data, which is used to automatically segment vesicle structures.

[0058] The first vesicle segmentation result refers to the vesicle segmentation mask obtained by inferring the deconvolutioned image through the corresponding pre-trained model.

[0059] The second vesicle segmentation result refers to the vesicle segmentation mask obtained by inferring the missing wedge recovery image through the corresponding pre-trained model.

[0060] Specifically, in this embodiment of the invention, the preprocessed deconvolutioned image and the missing wedge restored image can be input into a 3D segmentation model optimized and trained for their respective image features for parallel inference. One model is specifically adapted to the optimized image contrast features, while the other model is specifically adapted to the structural integrity restoration features, forming a dual-path parallel vesicle segmentation. This simultaneously obtains preliminary segmentation results based on different preprocessing modalities, providing a segmentation information basis for subsequent fusion decisions.

[0061] Furthermore, in this embodiment of the invention, the deconvolution image and the missing wedge recovery image can be input into the corresponding pre-trained 3D deep learning segmentation model for parallel inference, automatically identify the vesicle position and morphology, and obtain the first vesicle segmentation result and the second vesicle segmentation result respectively, ensuring that reliable segmentation can be obtained under different image quality conditions.

[0062] S330. The first vesicle segmentation result and the second vesicle segmentation result are fused to obtain the first fused vesicle segmentation result.

[0063] The first fusion vesicle segmentation result refers to the segmentation mask obtained by logically fusion of the first vesicle segmentation result and the second vesicle segmentation result.

[0064] Specifically, embodiments of the present invention may employ a decision fusion strategy to logically combine the first vesicle segmentation result and the second vesicle segmentation result to generate a first fused vesicle segmentation result, thereby improving the comprehensiveness and robustness of vesicle segmentation and reducing missed detections.

[0065] As examples, embodiments of the present invention can compare the three-dimensional binary masks output by two segmentation models on a voxel-by-voxel basis. If either model predicts a voxel as a vesicle, then the voxel is ultimately determined to belong to a vesicle. This decision fusion strategy aims to combine the advantages of the two paths to improve segmentation robustness and recall under different image quality conditions, generating a more comprehensive and reliable preliminary fused segmentation result, namely the first fused vesicle segmentation result.

[0066] S340. Ellipsoid fitting and morphological filtering are performed on the first fusion vesicle segmentation result to obtain the second fusion vesicle segmentation result, which includes multiple segmented vesicle instances and their morphological parameters.

[0067] The second fusion vesicle segmentation result refers to the segmentation result obtained after performing ellipsoid fitting and morphological filtering on the first fusion segmentation result to remove abnormal objects.

[0068] Specifically, in this embodiment of the invention, the first fusion vesicle segmentation result can be subjected to ellipsoid fitting processing. The fitted vesicle segmentation result includes multiple segmented vesicle instances and their morphological parameters.

[0069] The second fusion vesicle segmentation result refers to the vesicle instance and its morphological parameters output after three-dimensional ellipsoid fitting of each vesicle based on the first fusion result.

[0070] Among them, a vesicle instance refers to a single vesicle object obtained by segmentation, and each instance corresponds to a specific spatial location and shape.

[0071] Among them, morphological parameters refer to the geometric feature information of vesicle objects extracted through ellipsoid fitting.

[0072] Specifically, in this embodiment of the invention, for each vesicle instance in the first fusion vesicle segmentation result, a standardized three-dimensional ellipsoidal model fitting process is performed to quantitatively calculate its core morphological characterization parameters, calculate the precise morphological parameter set of the vesicle instance, and generate a second fusion vesicle segmentation result with standardized geometric characterization.

[0073] Furthermore, in this embodiment of the invention, a three-dimensional ellipsoid fitting is performed on each vesicle instance in the first fusion vesicle segmentation result. Using the least squares method combined with an adaptive regularization term, the center coordinates, the lengths of the three principal axes, and the direction vector are accurately estimated to extract the vesicle's morphological parameters, thus forming the vesicle segmentation result. The vesicle segmentation result after ellipsoid fitting contains multiple segmented vesicle instances and their detailed morphological information, meeting the needs of subsequent statistical analysis and visualization.

[0074] Specifically, embodiments of the present invention can calculate and screen morphological parameters for each vesicle instance after ellipsoidal fitting based on known prior knowledge of vesicle biophysical morphology. By using a preset, quantifiable threshold for morphological rationality, segmentation artifacts or noise instances that do not conform to the laws of biological structure are systematically identified and filtered out, thereby obtaining a second fused vesicle segmentation result that is more consistent and rational in morphology.

[0075] This invention provides a vesicle segmentation method based on cryo-electron microscopy (cryo-EM) tomographic images. The method includes: obtaining a cryo-EM tomographic image containing vesicle structures; preprocessing the cryo-EM tomographic image to generate a deconvolutioned image and a missing wedge restoration image; inputting the deconvolutioned image and the missing wedge restoration image into corresponding pre-trained segmentation models for parallel inference to obtain a first vesicle segmentation result and a second vesicle segmentation result, respectively; fusing the first and second vesicle segmentation results to obtain a first fused vesicle segmentation result; and performing ellipsoid fitting and morphological filtering on the first fused vesicle segmentation result to obtain a second fused vesicle segmentation result, which includes multiple segmented vesicle instances and their morphological parameters. This invention, by preprocessing the cryo-EM tomographic image with deconvolution and missing wedge restoration, combined with parallel inference using a dual-path pre-trained segmentation model, and further optimized through multi-stage processes such as fusion, ellipsoid fitting, and morphological filtering, effectively improves the accuracy and morphological representation ability of vesicle segmentation, thereby solving the technical problem of insufficient accuracy in vesicle segmentation of cryo-EM tomographic images.

[0076] Optionally, in the above Figure 3 Based on one or more corresponding embodiments, in another optional embodiment provided by the present invention, step S310 may specifically include:

[0077] The contrast transfer function algorithm is used to deconvolve cryo-electron microscopy tomography images to obtain deconvolutioned images.

[0078] Specifically, embodiments of the present invention can utilize the Contrast Transfer Function (CTF) algorithm to deconvolve cryo-electron microscopy tomographic images. First, the CTF parameters are automatically or manually adjusted according to different imaging conditions (such as whether a phase plate (VPP) is used) to accurately describe the frequency response characteristics during image imaging. For selected regions, a high-performance deconvolution algorithm is applied to perform deconvolution operations on the image, effectively suppressing noise and enhancing low-frequency signal characteristics, thereby improving the overall signal-to-noise ratio and image contrast, resulting in a deconvolutioned image.

[0079] As examples, users can select two points on an image through the image user interface to define a preview area and preview the correction effect of different signal-to-noise ratio (SNR) parameters in real time: low SNR parameters preserve edge details but retain more noise, while high SNR parameters have a good denoising effect but may be over-smoothed. After interactively adjusting the parameters based on the preview effect, users can click the "Apply" button to start performing CTF deconvolution operations on the entire image. This algorithm interactively optimizes the correction parameters according to different imaging conditions, effectively suppressing noise and enhancing structural information, and finally generating a deconvolutioned image as dedicated input to the "deconvolution path segmentation model (Dec-model)" in dual-path segmentation.

[0080] The deconvolution preview interface in the image user interface provided in this embodiment of the invention can be found here. Figure 4 , Figure 4 The image preview is displayed under different signal-to-noise ratio (SNR) parameters, allowing users to evaluate the effect and select parameters before applying deconvolution. The parameter panel on the right includes adjustable parameters such as acceleration voltage (Voltage kV), spherical aberration coefficient Cs, defocus value, pixel size, SNR falloff, deconv strength, and highpass weight. Users can quickly preview the parameter grid using the "Preview Deconv" button and apply the optimal parameters to the entire image by clicking "Apply Deconv".

[0081] A deep learning-based missing wedge restoration model was used to restore missing wedge artifacts in cryo-electron microscopy tomographic images, resulting in restored missing wedge images.

[0082] Specifically, the deep learning-based missing wedge restoration model in this embodiment addresses the missing wedge artifact problem caused by the tilted viewing angle in cryo-electron microscopy tomographic images. It utilizes the IsoNet algorithm, trained with a large amount of vesicle structure data, for compensation and repair. The missing wedge restoration model automatically infers and reconstructs missing structural information by learning the spatial structural features of vesicles and the missing patterns of image tomography, thereby restoring the integrity and continuity of the three-dimensional structure in the image. The generated missing wedge restored image not only reduces the impact of artifacts but also enhances the recognizability of microstructures such as vesicles, improving the structural integrity of subsequent segmentation.

[0083] As examples, to compensate for missing wedge artifacts caused by the limitations of the tilted acquisition angle, a deep learning model (based on the IsoNet algorithm architecture) specifically trained on a large amount of vesicle tomographic data can be used for structural restoration. This deep learning model is optimized for the membranous and spherical structural characteristics of vesicles and can automatically input the uniformly sampled original tomographic image into the pre-trained missing wedge restoration model. The deep learning network predicts and fills in the structural information loss caused by the missing wedge, reconstructing a more complete and isotropic 3D density map. The generated missing wedge restored image serves as the dedicated input for the other independent segmentation path in the dual-path segmentation, the "Missing Wedge Restoration Path Segmentation Model (MWR-model)," and is processed in parallel with the deconvolutioned image to jointly improve the robustness and structural integrity of subsequent segmentation.

[0084] Optionally, in the above Figure 3 Based on one or more corresponding embodiments, in another optional embodiment provided by the present invention, step S320 may specifically include:

[0085] The deconvolutioned image is input into a pre-trained deconvolutional path segmentation model to obtain the first vesicle segmentation result output by the deconvolutional path segmentation model.

[0086] Specifically, in this embodiment of the invention, a deconvolutioned image can be input into a pre-trained deconvolutioned path segmentation model, and a deep learning model (Dec-model) based on the 3D Res-UNet architecture can be loaded. This model has been trained on a large number of vesicle tomographic images that have undergone CTF deconvolution processing. The model performs convolution operations and feature extraction on the three-dimensional voxel data, automatically identifies vesicle regions, and finally generates the corresponding three-dimensional segmentation mask, which is the first vesicle segmentation result. This ensures efficient and accurate vesicle segmentation under the unique contrast enhancement and noise suppression conditions of deconvolutioned images.

[0087] As examples, when a user triggers a segmentation command, the deconvolutioned image data is automatically fed into the GPU-accelerated Dec-model. The model processes the entire 3D volume data via forward propagation, predicts vesicle probabilities on a voxel-by-voxel basis, and generates a binary segmentation mask through thresholding, which is the first vesicle segmentation result, thus capturing the continuous edge structure of the vesicle membrane under optimized signal-to-noise ratio conditions.

[0088] The missing wedge recovery image is input into a pre-trained missing wedge recovery path segmentation model to obtain the second vesicle segmentation result output by the missing wedge recovery path segmentation model.

[0089] Specifically, in this embodiment of the invention, when the missing wedge restoration image is input into a pre-trained missing wedge restoration path segmentation model, another model (MWR-model) also based on 3D Res-UNet is loaded. This model is specifically trained for image data whose tilted view missing structures have been restored using the IsoNet algorithm. After the input image is preprocessed, the model also performs 3D convolutional inference, identifies vesicle voxels, and outputs the corresponding 3D segmentation mask, i.e., the second vesicle segmentation result. This effectively utilizes the completed structural information in the missing wedge restoration image, improving the completeness and robustness of vesicle segmentation.

[0090] As examples, when a user triggers a segmentation command, the MWR-model is loaded, and the image restored from the missing wedge is fed into the model for inference. The model outputs the predicted probability that each voxel belongs to a vesicle. After post-processing, a 3D binary segmentation mask is generated, which is the second vesicle segmentation result. This achieves complete 3D vesicle structural information restored from the missing wedge, improving vesicle segmentation accuracy and recall rate in cases of severe missing wedges.

[0091] Optional, based on Figure 3 The method shown is as follows: Figure 5 The diagram shows a specific implementation of step S340 in the vesicle segmentation method based on cryo-electron microscopy tomography provided in this invention. Step S340 may specifically include:

[0092] S500. Perform three-dimensional ellipsoid fitting on each vesicle instance in the first fusion vesicle segmentation result to obtain vesicle fitting results. The vesicle fitting results include the calculated mask and morphological parameters of each vesicle instance. The morphological parameters include the center coordinates, principal axis length and direction vector of the vesicle instance.

[0093] Specifically, in this embodiment of the invention, an initial ellipsoid model can be constructed for each vesicle instance in the first fusion segmentation result, and an improved least squares fitting algorithm based on an adaptive regularization term of a sigmoid function can be used to iteratively optimize the ellipsoid model parameters so that the ellipsoid model and the three-dimensional point cloud data of the vesicle instance achieve the best match. This effectively suppresses the local noise interference introduced by the high-density protein complex on the membrane surface, thereby robustly and accurately calculating the core morphological parameters such as the center coordinates, principal axis length, and direction vector of each vesicle instance.

[0094] Furthermore, in this embodiment of the invention, after completing the 3D ellipsoidal fitting of all vesicle instances and calculating their respective morphological parameters, these geometric parameters are associated and integrated with the original vesicle instance labels. Each vesicle instance is no longer simply represented by a set of voxels, but is given a precise mathematical geometric description (ellipsoidal model). Based on these fitted ellipsoidal parameters, a higher-fidelity, geometrically model-based 3D representation can be generated. This result is the vesicle fitting result, which contains a set of vesicles that has undergone morphological correction and parameterization.

[0095] The embodiments of the present invention not only improve the accuracy and biological rationality of vesicle segmentation through ellipsoid fitting, but also provide a solid data foundation for subsequent statistical analysis, morphological studies and visualization, significantly enhancing its practical value and application effect in the analysis of complex biological structures.

[0096] S510. Based on the preset vesicle morphology rationality threshold and the morphological parameters of each vesicle instance, remove abnormal vesicle instances from the vesicle fitting results to obtain the second fused vesicle segmentation result.

[0097] The preset vesicle morphology rationality threshold refers to a set of limiting parameters set for the geometric morphology of vesicles based on prior biological knowledge and statistical laws during the automatic segmentation and analysis of vesicle 3D structures. The preset vesicle morphology rationality threshold can include the diameter range of vesicles (e.g., 25–70 nanometers), aspect ratio (e.g., less than 3), etc. The preset vesicle morphology rationality threshold is used to filter and remove objects with abnormal size, abnormal morphology, or that do not conform to biological characteristics from the segmentation results, ensuring that only biologically rational vesicle instances are retained in subsequent analyses, thereby improving the accuracy and reliability of the segmentation results.

[0098] Specifically, in this embodiment of the invention, based on the geometric feature parameters extracted from each vesicle instance during ellipsoid fitting, including indicators such as diameter (measured by the equivalent sphere diameter) and aspect ratio (the ratio of the maximum to the minimum length of the principal axis), when removing abnormal instances from the first fused vesicle segmentation result according to a preset vesicle morphology rationality threshold, three-dimensional connected component analysis is first performed on each segmented vesicle instance. Then, all segmented instances are automatically screened according to a pre-set biological rationality threshold range, and abnormal objects that do not meet the conditions are judged as noise, artifacts, or incorrect segmentation. This process can use a batch automated filtering algorithm without manual intervention, significantly improving the accuracy and biological rationality of the segmentation results. After this cleaning step, the generated second fused vesicle segmentation result has eliminated invalid objects that are too small, too large, or have abnormal shapes, providing high-quality input data for subsequent accurate ellipsoid fitting and morphological analysis.

[0099] As examples, the equivalent diameter is calculated by converting the volume of an instance to the volume of a sphere. This automatically removes aberrant segmentation instances with an equivalent diameter less than 25 nanometers or greater than 70 nanometers, or with an aspect ratio greater than 3. These removed instances typically correspond to noisy, non-vesicle organelle, or severely distorted erroneous segmentations. After filtering, the set of all remaining vesicle instances that meet the preset vesicle morphology rationality threshold constitutes the second fusion vesicle segmentation result.

[0100] Optional, see Figure 6 The 3D ellipsoid fitting process provided in this embodiment of the invention includes two fitting strategies: automatic fitting and manual fitting. Automatic fitting automatically generates a prediction mask based on the dual-path segmentation results and performs ellipsoid fitting on each vesicle region to obtain the center, major axis length, and direction. Manual fitting involves the user manually marking vesicle regions or adjusting mask boundaries in the interface, and the system automatically calculates the fitting parameters based on the annotation information. Both strategies output the mask and its morphological parameters, achieving accurate morphological modeling.

[0101] Optionally, in the above Figure 5 Based on one or more corresponding embodiments, in another optional embodiment provided by the present invention, step S500 may specifically include:

[0102] An initial ellipsoidal model is constructed for each vesicle instance in the first fusion vesicle segmentation result. A loss function is constructed, which includes the sum of squared projected distances from the sampling points to the ellipsoidal surface and a regularization term based on a Sigmoid function. The Sigmoid function is configured to reduce the weight when the projected distance is less than a preset membrane thickness threshold and increase the weight when the projected distance is greater than the preset membrane thickness threshold, in order to suppress the interference of abnormal density points corresponding to high-density protein complexes on the membrane surface on the fitting. The loss function is minimized using a memory-constrained quasi-Newton optimization algorithm to optimize the ellipsoidal fitting parameters of the initial ellipsoidal model until a preset convergence condition is met. The quality of the optimized fitting results is evaluated, and fitting results with a coefficient of variation exceeding a first preset threshold or a cross-correlation coefficient lower than a second preset threshold are discarded. The retained fitting parameters are used as morphological parameters.

[0103] Specifically, in this embodiment of the invention, for each vesicle instance in the first fusion segmentation result, the boundary point cloud is first extracted from the three-dimensional voxel set of the vesicle instance to construct the corresponding initial ellipsoid model. This initial ellipsoid model is typically based on the centroid and variance estimates of the segmented voxels, providing a rough estimate of the ellipsoid center position, initial length and direction of the principal axis, laying the foundation for subsequent fine fitting.

[0104] As examples, embodiments of the present invention can extract the three-dimensional coordinates of all voxel points of a vesicle instance to construct the three-dimensional point cloud data of that instance. Then, the covariance matrix of the point cloud data is calculated, and principal component analysis is used to obtain the three principal directions of the point cloud and their corresponding eigenvalues. Based on this, the centroid coordinates of the point cloud are initialized to the center of the ellipsoidal model, the three principal directions are set as the initial direction vectors of the ellipsoid, and the length of each principal axis is initialized to a certain proportion of the point cloud distribution range in the corresponding direction (e.g., twice the standard deviation of that direction), thereby constructing an initial ellipsoidal model for the vesicle instance that is approximately aligned with its actual spatial distribution.

[0105] The least squares fitting algorithm was used to optimize the ellipsoid fitting parameters of the initial ellipsoid model until the ellipsoid fitting parameters of the initial ellipsoid model matched the three-dimensional point cloud data of the corresponding vesicle instance. The optimization process was based on an adaptive regularization term similar to a sigmoid function to reduce the interference of high-density protein complexes on the membrane surface.

[0106] Specifically, this embodiment of the invention employs a restricted-memory quasi-Newton optimization algorithm (L-BFGS-B) to iteratively optimize the parameters of the initial ellipsoid model. By continuously adjusting the ellipsoid center coordinates, the lengths of the three principal axes, and the direction vectors, it minimizes the loss function, which includes a sum of squared distance terms and an adaptive regularization term, to achieve the best match between the fitted ellipsoid surface and the 3D point cloud data of the vesicle instance. An adaptive regularization term based on a sigmoid-like function is introduced during the optimization process to ensure the stability and biological rationality of the fitted parameters. Iterative optimization stops when the relative change of the objective function falls below a preset convergence threshold or the maximum number of iterations is reached. Furthermore, quality control is performed on the fitting results, discarding fitting results with a coefficient of variation exceeding a first preset threshold or a cross-correlation coefficient falling below a second preset threshold. Finally, high-precision ellipsoid fitting parameters are output, comprehensively reflecting the 3D morphological characteristics of the vesicles.

[0107] As examples, after constructing the initial ellipsoid model, an iterative least squares fitting algorithm is used to optimize it, with the objective function being:

[0108] ;

[0109] in, Indicates the center coordinates of the ellipsoid model; This represents the length of the principal axis of the ellipsoidal model; This represents the direction vector of the ellipsoidal model; For the first point cloud The projected distance of each voxel to the surface of the current ellipsoidal model. The optimization process can continuously adjust these parameters using algorithms such as gradient descent or Levenberg-Marquardt to minimize the sum of the squared distances of all voxels, i.e. This allows the ellipsoidal model to fit the point cloud data as closely as possible. It is the key adaptive regularization term, in which It is a sigmoid-like function, which is effective for distance. A nonlinear weight is applied. When a voxel point is far from the ellipsoidal surface (potentially corresponding to extracellular noise or attached macromolecular complexes), this function increases the point's contribution to the total loss, thus imposing a stronger penalty on such anomalous density points. Conversely, for voxel points that are closer and more likely to represent the true membrane structure, their contribution to the regularization loss is minimized. Iterative optimization continues until the fitting parameters converge or a preset number of iterations is reached, ultimately obtaining the optimal ellipsoidal model, after regularization correction, that highly matches the 3D point cloud of the vesicle instance.

[0110] Optional, based on Figure 1 The method shown is as follows: Figure 7 As shown, this is a flowchart illustrating another embodiment of the vesicle segmentation method based on cryo-electron microscopy tomography provided by this invention. After step S340, the method further includes:

[0111] S700. Perform interactive manual verification on the second fusion vesicle segmentation result to obtain the third fusion vesicle segmentation result.

[0112] The third-generation fusion vesicle segmentation results, after manual interactive verification, can be directly used for subsequent statistical analysis (such as calculating volume, aspect ratio distribution, spatial distribution, etc.) and high-precision 3D visualization. Users can render and view vesicle instances based on these ellipsoidal parameters. It also supports exporting morphological parameter tables of all vesicles in JSON or Excel format, as well as segmentation map files containing instance labels and fitted geometric information, providing a structured data foundation for in-depth biological research.

[0113] Optionally, the masking effect in the third fusion vesicle segmentation result can be as follows: Figure 8 As shown, Figure 8 This image shows a 3D reconstruction of vesicles generated based on cryo-electron microscopy tomographic images containing vesicle structures. Different colors represent different segmentation instances, demonstrating the spatial distribution and morphological diversity of vesicles. A scale bar of 100 nm is used to represent the spatial scale. It is evident that this embodiment of the invention can accurately identify and reconstruct the true 3D morphology of vesicles, exhibiting high segmentation accuracy and morphological fidelity.

[0114] Optionally, in the above Figure 7 Based on one or more corresponding embodiments, in another optional embodiment provided by the present invention, step S700 may specifically include:

[0115] The tri-orthogonal view of the second fusion vesicle segmentation result is displayed synchronously through a graphical user interface.

[0116] In 3D image processing and visualization, a tri-orthogonal view refers to the simultaneous display of three mutually perpendicular 2D cross-sectional views, corresponding to the three directions of the spatial coordinate axes: the XY plane (lateral cross-section), the XZ plane (vertical cross-section), and the YZ plane (lateral cross-section). Through synchronized scrolling and interactive operations, users can observe and locate different cross-sectional positions of 3D structures in real time within these three views, facilitating precise analysis, identification, and editing of 3D objects such as vesicles.

[0117] Specifically, embodiments of the present invention can automatically generate three orthogonal views—namely, three two-dimensional slice views (XY, XZ, and YZ)—after loading the vesicle 3D segmentation mask data generated in the third fusion stage, based on a graphical user interface built on Napari. These three views are displayed in tandem through a synchronous scrolling and cursor linkage mechanism. When the user selects or scrolls a slice in any view, the corresponding slice positions in the other views are automatically updated, ensuring that the user can observe the spatial distribution and morphological characteristics of vesicle instances in real time and comprehensively from different spatial dimensions, thereby improving the interactive experience and positioning accuracy.

[0118] As examples, once the user loads the results, three orthogonal view windows are automatically created and simultaneously displayed in the main area of ​​the graphical user interface. Each view window renders the current tomographic image (original or preprocessed) as a background in real time, and overlays the third fused vesicle segmentation result—the ellipsoidal fitting model and its label for each vesicle instance—in a semi-transparent or outlined form. As the user scrolls through any view along an axis perpendicular to the plane using the mouse wheel or slider, the slice positions of the three views are updated in tandem, ensuring simultaneous observation of the segmentation and fitting at the same spatial location from three spatial dimensions. Vesicle instances are typically distinguished by different colors, and the ellipsoidal model is visualized using its mathematical representation (center, principal axis), allowing the user to intuitively assess the segmentation integrity.

[0119] Receives interactive calibration commands input by the user through the graphical user interface.

[0120] Interactive correction commands refer to a series of operation instructions actively input by the user through a graphical user interface, used to correct, adjust, or improve errors or inaccuracies in the automatic segmentation and morphological fitting results. For example, if the vesicle examples fitted in the segmentation results of the second fusion vesicle do not meet the requirements, the user can delete the corresponding vesicle examples and then add vesicle examples that meet the requirements. Interactive correction commands can include operations such as adding and deleting to correct the three-dimensional vesicle model and ensure the accuracy and biological rationality of the analysis results.

[0121] Specifically, embodiments of the present invention can receive interactive correction commands input by users through a graphical user interface. For example, users can use mouse clicks or keyboard shortcuts to annotate and correct the displayed vesicle model. Embodiments of the present invention monitor and parse the user's input commands in real time, converting them into specific modification requests for the data structure, such as adding vesicle instances or deleting misjudged objects.

[0122] As examples, the graphical user interface provides various interactive tools to receive user correction instructions for segmentation results. The interface includes dedicated toolbars or operation panels. Users trigger these operations via mouse clicks or keyboard shortcuts. For instance, in "Delete" mode, users can directly click on one or more displayed vesicle instances in any orthographic view, capture the coordinates of the click location, map them to 3D space, and delete the selected vesicle instance label. For the "Add" operation, users can define the position and size of new vesicles by clicking multiple points in different views, automatically performing ellipsoidal fitting to generate a new instance mask and its corresponding morphological parameters. All user interaction events are captured by Napari's event system and passed to the background processing logic.

[0123] In response to the interactive correction command, the mask and morphological parameters of one or more vesicle instances in the second fusion vesicle segmentation result are modified to obtain the third fusion vesicle segmentation result.

[0124] Specifically, for the add operation, new ellipsoid parameters are calculated based on the spatial points or annotation information input by the user and inserted into the segmentation result; for the delete operation, the corresponding instance is directly removed from the segmentation result. The updated parameters and segmentation mask are fed back to the tri-orthogonal view in real time, enabling instant visualization and verification of the results.

[0125] As examples, for the "delete" command, the specified vesicle entry can be directly removed from the vesicle instance list and the associated ellipsoid parameter table. For add commands (such as "six-point fitting add"), a new ellipsoid model is directly calculated using the least squares method based on six spatial coordinate points provided by the user, obtaining its center, principal axis length, and direction vector. Then, the instance label of this new vesicle and the calculated ellipsoid parameters are added to the segmentation mask and parameter table, respectively.

[0126] Optionally, the interactive correction commands include at least one of the following commands: three-dimensional fitting addition command, two-dimensional fitting addition command, six-point fitting addition command, and deletion operation command.

[0127] The 3D fitting add command refers to the operation command whereby users directly specify a location in the 3D space view through the graphical user interface to add a new vesicle ellipsoid model instance. The 3D fitting add command allows users to accurately locate vesicles based on 3D data and visual information, and automatically generate corresponding ellipsoid fitting parameters based on the input points, achieving high-precision 3D morphological supplementation and correction.

[0128] The 2D fitting addition command refers to the interactive operation command of the user to draw or annotate the vesicle contour region on a 2D slice view (such as the XY plane). Embodiments of this invention can add vesicle segmentation results to the 2D slice based on the 2D annotation information.

[0129] The six-point fitting addition command allows users to select six key control points in a two-dimensional slice view (such as the XY plane) and use a geometric fitting algorithm to generate an accurate ellipsoidal model that conforms to the constraints of these points. This command allows users to finely define the vesicle morphology using six points, improving the accuracy of the fitting and is suitable for areas with complex morphology or where automatic segmentation results are poor.

[0130] The deletion command is issued by the user through the graphical user interface by selecting one or more erroneous vesicle instances and clicking the delete button or using a shortcut key. Upon receiving this command, this embodiment of the invention immediately removes the corresponding instance from the segmentation results and ellipsoid parameter list, enabling rapid removal of incorrect segmentation objects and ensuring data quality and analytical accuracy.

[0131] The embodiments of the present invention can manually calibrate each vesicle instance in the second fusion vesicle segmentation result, including three-dimensional fitting addition instructions, two-dimensional fitting addition instructions, six-point fitting addition instructions, and deletion operation instructions, to obtain an accurate third fusion vesicle segmentation result.

[0132] See Figure 9 The interactive correction effect diagram provided in this embodiment of the invention shows that the 3D fitting addition command can directly add vesicle fittings in 3D space. The 2D fitting addition command can add vesicle regions on 2D slices. The six-point fitting addition mass can generate an accurate ellipsoid through six-point fitting. The deletion command can delete erroneously segmented or redundant vesicles. Figure 9 Different colors represent different segmentation instances, and the 50nm scale indicates the spatial scale.

[0133] Optionally, in response to an interactive correction command, after modifying the mask and morphological parameters of one or more vesicle instances in the second fusion vesicle segmentation result to obtain the third fusion vesicle segmentation result, the mask of each vesicle instance in the third fusion vesicle segmentation result is used as a sample to retrain the pre-trained segmentation model.

[0134] Specifically, embodiments of the present invention can extract the original image region (i.e., the 3D voxel data of the vesicle in the deconvolution image and the missing wedge recovery image) corresponding to each vesicle instance that has been confirmed or modified by the user from the currently processed 3D tomographic image. Simultaneously, a high-quality 3D binary segmentation mask, i.e., sample labeling, is generated corresponding to the ellipsoid fitting parameters (center coordinates, principal axis length, direction vector) finally confirmed by the user or generated by the correction operation. Finally, the original image patch (as input) and its corresponding high-quality segmentation mask (as target label) are saved as a new training set and stored in a designated sample library to prepare for subsequent model fine-tuning.

[0135] Specifically, embodiments of the present invention can retrain the pre-trained segmentation model using the new samples collected in the above steps through a "one-click model fine-tuning" function. When the user clicks the "Fine-tune" button, a multi-threaded training task is started in the background. This task first loads the weights of the pre-trained dual-path 3D segmentation models (Dec-model and MWR-model) integrated in the system as initial weights. Then, the newly collected samples form the fine-tuning dataset. The optimizer Adam performs several rounds of iterative training on the fine-tuning dataset to minimize the loss function (Dice loss) between predicted segmentation and sample annotation. After training is completed, the updated model weight file is automatically saved, and the original pre-trained model can be optionally replaced. This process allows the model to quickly adapt to the characteristics of the new dataset (such as different imaging conditions or vesicle morphology), significantly improving segmentation performance on new data without having to perform large-scale annotation and training from scratch.

[0136] Although the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous.

[0137] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0138] Corresponding to the above method embodiments, this invention also provides a vesicle segmentation system based on cryo-electron microscopy tomography images, which may include: a cryo-electron microscopy tomography image acquisition module, a preprocessing module, a dual-path segmentation module, a fusion module, and a morphology processing module.

[0139] The cryo-electron microscopy tomography module is used to obtain cryo-electron microscopy tomographic images containing vesicle structures.

[0140] The preprocessing module is used to preprocess cryo-electron microscopy tomographic images to generate deconvolution images and missing wedge restoration images.

[0141] The dual-path segmentation module is used to input the deconvolution image and the missing wedge recovery image into the corresponding pre-trained segmentation model for parallel inference, and obtain the first vesicle segmentation result and the second vesicle segmentation result respectively.

[0142] The fusion module is used to fuse the first vesicle segmentation result and the second vesicle segmentation result to obtain the first fused vesicle segmentation result.

[0143] The morphology processing module is used to perform ellipsoid fitting and morphology filtering on the first fusion vesicle segmentation result to obtain the second fusion vesicle segmentation result, which includes multiple segmented vesicle instances and their morphological parameters.

[0144] Optionally, the preprocessing module can be used to deconvolve the cryo-electron microscopy tomographic images using the contrast transfer function algorithm to obtain deconvolutioned images; and to restore the missing wedge artifacts in the cryo-electron microscopy tomographic images using a deep learning-based missing wedge restoration model to obtain missing wedge restored images.

[0145] Optionally, the dual-path segmentation module can be used to input the deconvolutioned image into a pre-trained deconvolutional path segmentation model to obtain the first vesicle segmentation result output by the deconvolutional path segmentation model; and to input the missing wedge restoration image into a pre-trained missing wedge restoration path segmentation model to obtain the second vesicle segmentation result output by the missing wedge restoration path segmentation model.

[0146] Optionally, the morphology processing module may include an ellipsoid fitting subunit and a morphology filtering subunit.

[0147] The ellipsoid fitting subunit is used to perform three-dimensional ellipsoid fitting on each vesicle instance in the first fusion vesicle segmentation result to obtain the vesicle fitting result. The vesicle fitting result includes the calculated mask and morphological parameters of each vesicle instance. The morphological parameters include the center coordinates, principal axis length and direction vector of the vesicle instance.

[0148] The morphological filtering subunit is used to remove abnormal vesicle instances from the vesicle fitting results based on a preset vesicle morphological rationality threshold and the morphological parameters of each vesicle instance, so as to obtain the second fused vesicle segmentation result.

[0149] Optionally, an ellipsoid fitting subunit can be used to construct an initial ellipsoid model for each vesicle instance in the first fusion vesicle segmentation result; construct a loss function, which includes the sum of squared projected distances from the sampling points to the ellipsoid surface and a regularization term based on a Sigmoid function. The Sigmoid function is configured to reduce the weight when the projected distance is less than a preset membrane thickness threshold and increase the weight when the projected distance is greater than the preset membrane thickness threshold, so as to suppress the interference of abnormal density points corresponding to high-density protein complexes on the membrane surface on the fitting; minimize the loss function using a limited-memory quasi-Newton optimization algorithm to optimize the ellipsoid fitting parameters of the initial ellipsoid model until a preset convergence condition is met; evaluate the quality of the optimized fitting results, discard fitting results with a coefficient of variation exceeding a first preset threshold or a cross-correlation coefficient lower than a second preset threshold, and use the retained fitting parameters as morphological parameters.

[0150] Optionally, the vesicle segmentation system based on cryo-electron microscopy tomography images may also include a calibration module.

[0151] The proofreading module is used by the morphology processing module to perform ellipsoid fitting and morphology filtering on the first fused vesicle segmentation result to obtain the second fused vesicle segmentation result. Then, the second fused vesicle segmentation result is manually proofread to obtain the third fused vesicle segmentation result.

[0152] Optionally, the calibration module is specifically used to synchronously display the tri-orthogonal view of the second fusion vesicle segmentation result through a graphical user interface; receive interactive calibration instructions input by the user through the graphical user interface; and, in response to the interactive calibration instructions, modify the mask and morphological parameters of one or more vesicle instances in the second fusion vesicle segmentation result to obtain the third fusion vesicle segmentation result.

[0153] Optionally, the interactive correction commands include at least one of the following commands: three-dimensional fitting addition command, two-dimensional fitting addition command, six-point fitting addition command, and deletion operation command.

[0154] Optionally, the vesicle segmentation system based on cryo-electron microscopy tomography images may also include: a fine-tuning module.

[0155] The fine-tuning module is used to interactively and manually calibrate the second fusion vesicle segmentation result by the calibration module. After obtaining the third fusion vesicle segmentation result, the mask of each vesicle instance in the third fusion vesicle segmentation result is used as a sample to retrain the pre-trained segmentation model.

[0156] This invention provides a vesicle segmentation system based on cryo-electron microscopy (cryo-EM) tomographic images. The system is used to: obtain cryo-EM tomographic images containing vesicle structures; preprocess the cryo-EM tomographic images to generate deconvolutioned images and missing wedge restoration images; input the deconvolutioned images and missing wedge restoration images into corresponding pre-trained segmentation models for parallel inference to obtain a first vesicle segmentation result and a second vesicle segmentation result, respectively; fuse the first and second vesicle segmentation results to obtain a first fused vesicle segmentation result; and perform ellipsoid fitting and morphological filtering on the first fused vesicle segmentation result to obtain a second fused vesicle segmentation result, which includes multiple segmented vesicle instances and their morphological parameters. This invention, through preprocessing cryo-EM tomographic images with deconvolution and missing wedge restoration, combined with parallel inference using a dual-path pre-trained segmentation model, and further optimized through multi-stage processes such as fusion, ellipsoid fitting, and morphological filtering, effectively improves the accuracy and morphological representation ability of vesicle segmentation, thereby solving the technical problem of insufficient accuracy in vesicle segmentation of cryo-EM tomographic images.

[0157] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0158] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the vesicle segmentation method based on cryo-electron microscopy tomography images.

[0159] This invention provides a processor for running a program, wherein the program executes the vesicle segmentation method based on cryo-electron microscopy tomography images.

[0160] This invention provides an electronic device, which includes at least one processor, at least one memory, a bus, and a graphics processor connected to the processor. The processor and memory communicate with each other via the bus. The processor is used to call program instructions from the memory to execute the aforementioned vesicle segmentation method based on cryo-electron microscopy tomography images. The graphics processor is used to call a deep learning model for computation. The electronic device described herein can be a server or a PC, etc.

[0161] The present invention also provides a computer program product that, when executed on an electronic device, is adapted to execute a program having steps for initializing a vesicle segmentation method based on cryo-electron microscopy tomography images.

[0162] This invention is described with reference to flowchart illustrations and / or block diagrams of methods and systems according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0163] In a typical configuration, an electronic device includes one or more processors (CPUs), memory, and a bus. The electronic device may also include input / output interfaces, etc.

[0164] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM, and memory includes at least one memory chip. Memory is an example of computer-readable media.

[0165] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0166] In the description of this invention, it should be understood that if the terms "upper", "lower", "front", "rear", "left" and "right" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the position or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0167] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0168] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0169] The above are merely embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the present invention.

Claims

1. A vesicle segmentation method based on cryo-electron microscopy tomography images, characterized in that, include: Obtain cryo-electron tomographic images containing vesicle structures; The cryo-electron microscopy tomographic images are preprocessed to generate deconvolution images and missing wedge recovery images; The deconvolution image and the missing wedge recovery image are respectively input into the corresponding pre-trained segmentation model for parallel inference to obtain the first vesicle segmentation result and the second vesicle segmentation result, respectively. The first vesicle segmentation result and the second vesicle segmentation result are fused to obtain the first fused vesicle segmentation result; Ellipsoid fitting and morphological filtering are performed on the first fusion vesicle segmentation result to obtain the second fusion vesicle segmentation result, which includes multiple segmented vesicle instances and their morphological parameters.

2. The method according to claim 1, characterized in that, The preprocessing of the cryo-electron microscopy tomographic images to generate deconvolution images and missing wedge restoration images includes: The cryo-electron microscopy tomographic image was deconvolved using the contrast transfer function algorithm to obtain a deconvolved image. The cryo-electron microscopy tomographic image was processed to restore the missing wedge artifact using a deep learning-based missing wedge restoration model, resulting in a missing wedge restored image.

3. The method according to claim 1, characterized in that, The step of inputting the deconvolutioned image and the missing wedge recovery image into the corresponding pre-trained segmentation models for parallel inference to obtain the first vesicle segmentation result and the second vesicle segmentation result, respectively, includes: The deconvolutioned image is input into a pre-trained deconvolutional path segmentation model to obtain the first vesicle segmentation result output by the deconvolutional path segmentation model. The missing wedge recovery image is input into a pre-trained missing wedge recovery path segmentation model to obtain the second vesicle segmentation result output by the missing wedge recovery path segmentation model.

4. The method according to claim 1, characterized in that, The step of performing ellipsoidal fitting and morphological filtering on the first fusion vesicle segmentation result to obtain the second fusion vesicle segmentation result includes: Three-dimensional ellipsoid fitting is performed on each vesicle instance in the first fusion vesicle segmentation result to obtain vesicle fitting results. The vesicle fitting results include the calculated mask and morphological parameters of each vesicle instance. The morphological parameters include the center coordinates, principal axis length and direction vector of the vesicle instance. Based on a preset threshold for vesicle morphology rationality and the morphological parameters of each vesicle instance, abnormal vesicle instances in the vesicle fitting results are removed to obtain a second fused vesicle segmentation result.

5. The method according to claim 4, characterized in that, The step of performing three-dimensional ellipsoid fitting on each vesicle instance in the first fused vesicle segmentation result to obtain vesicle fitting results includes: For each vesicle instance in the first fusion vesicle segmentation result, an initial ellipsoid model is constructed; A loss function is constructed, which includes a term for the sum of squared projected distances from the sampling points to the ellipsoidal surface and a regularization term based on a Sigmoid function. The Sigmoid function is configured to reduce the weight when the projected distance is less than a preset membrane thickness threshold and increase the weight when the projected distance is greater than the preset membrane thickness threshold, so as to suppress the interference of abnormal density points corresponding to high-density protein complexes on the membrane surface on the fitting. The loss function is minimized using a memory-constrained quasi-Newton optimization algorithm to optimize the ellipsoid fitting parameters of the initial ellipsoid model until a preset convergence condition is met. The optimized fitting results are evaluated for quality. Fitting results with a coefficient of variation exceeding a first preset threshold or a cross-correlation coefficient lower than a second preset threshold are removed, and the remaining fitting parameters are used as the morphological parameters.

6. The method according to claim 1, characterized in that, After performing ellipsoid fitting and morphological filtering on the first fusion vesicle segmentation result to obtain the second fusion vesicle segmentation result, the method further includes: The second fusion vesicle segmentation result is manually verified interactively to obtain the third fusion vesicle segmentation result.

7. The method according to claim 6, characterized in that, The interactive manual verification of the second fusion vesicle segmentation result to obtain the third fusion vesicle segmentation result includes: The second fusion vesicle segmentation result is simultaneously displayed in a tri-orthogonal view via a graphical user interface; Receive interactive correction commands input by the user through the graphical user interface; In response to the interactive correction command, the mask and morphological parameters of one or more vesicle instances in the second fusion vesicle segmentation result are modified to obtain the third fusion vesicle segmentation result.

8. The method according to claim 7, characterized in that, The interactive correction instructions include at least one of the following: a three-dimensional fitting addition instruction, a two-dimensional fitting addition instruction, a six-point fitting addition instruction, and a deletion operation instruction.

9. The method according to claim 7, characterized in that, After modifying the mask and morphological parameters of one or more vesicle instances in the second fusion vesicle segmentation result in response to the interactive correction command to obtain the third fusion vesicle segmentation result, the method further includes: The pre-trained segmentation model is retrained by using the masks of each vesicle instance in the third fusion vesicle segmentation result as samples.

10. A vesicle segmentation system based on cryo-electron microscopy tomography images, characterized in that, include: The system includes a cryo-electron microscopy tomography image acquisition module, a preprocessing module, a dual-path segmentation module, a fusion module, and a morphological processing module. The cryo-electron microscopy tomography image acquisition module is used to acquire cryo-electron microscopy tomography images containing vesicle structures; The preprocessing module is used to preprocess the cryo-electron microscopy tomographic images to generate deconvolution images and missing wedge recovery images; The dual-path segmentation module is used to input the deconvolution image and the missing wedge recovery image into the corresponding pre-trained segmentation model for parallel inference, and obtain the first vesicle segmentation result and the second vesicle segmentation result respectively. The fusion module is used to fuse the first vesicle segmentation result and the second vesicle segmentation result to obtain a first fused vesicle segmentation result. The morphology processing module is used to perform ellipsoid fitting and morphology filtering on the first fusion vesicle segmentation result to obtain a second fusion vesicle segmentation result, which includes multiple segmented vesicle instances and their morphological parameters.

Images