Method, system, device, chip and medium for generating cryo-electron microscopy images guided by physical information

Abstract

A method, system, device, chip and medium for generating cryo-EM images guided by physical information, the method comprising: S1, preparing a virtual sample containing a large number of target molecular structures; S2, performing a physics-based simulation on the virtual sample to simulate the cryo-EM imaging process to obtain a two-dimensional image; S3, based on slice-by-slice contrast learning, converting simple Gaussian noise in the two-dimensional imaging into real cryo-EM noise; S4, using contrast learning of mask-guided sampling to effectively distinguish particle features from background features. The present invention combines physics-based simulation with contrast noise generation to improve the reconstruction results of cryo-EM images and generate a highly realistic synthetic cryo-EM dataset with annotations.

Description

Technical Field

[0001] The present invention relates to computer graphics processing technology applied to structural biology, and specifically to a method, system, device, chip and medium for generating cryo-electron microscope images guided by physical information. Background Art

[0002] Cryo-electron microscopy (cryo-EM) uses extreme low temperatures to rapidly freeze biological samples so that they can be observed and analyzed in their nearly intact natural state. This technology enables scientists to capture the structure of proteins, viruses, and other biological macromolecules with near-atomic resolution, thereby revealing their functions and mechanisms of action. A major challenge facing cryo-electron microscopy is that it is necessary to accurately select the center position of the particle and determine the particle's posture from a high-noise (low signal-to-noise ratio) image containing target particles in multiple different positions and postures. This task currently relies mainly on deep learning models, and the effectiveness of these models is limited by the lack of high-quality, well-annotated training data sets.

[0003] Although existing physical simulation techniques can simulate cryo-EM images, they are computationally expensive and difficult to effectively reproduce the complex noise characteristics in actual cryo-EM images. Existing generative models fail to fully consider physical constraints, so the images they produce are not physically accurate enough and are not suitable for improving the accuracy of cryo-EM image analysis. Deep generative models, an advanced artificial intelligence technology based on deep learning structures, can generate new samples that are highly similar to the original data by simulating the distribution of learning data, thereby improving the performance of deep learning models. They have demonstrated their broad application prospects in entertainment, visual arts, and broader scientific imaging fields, such as generating high-definition brain magnetic resonance imaging (MRI), computed tomography (CT) images, and even performing complex astrophysical simulations. However, directly applying these methods designed for general image generation or other scientific imaging tasks to cryo-EM image generation tasks requires specially designed methods to meet their specific data characteristics and physical constraints in the face of the unique properties and complexity of cryo-EM images. Summary of the invention

[0004] The present invention aims to provide a method, system, device, chip and medium for generating cryo-electron microscopy images guided by physical information, so as to solve the problems of difficulty in obtaining high-quality annotated cryo-electron microscopy data sets, the shortcomings of traditional methods in simulating the complex noise of real images, the high cost of physical simulation and the failure of existing generation models to fully consider physical constraints.

[0005] In order to achieve the above object, the method provided by the present invention comprises:

[0006] S1, prepare virtual samples containing a large number of target molecular structures;

[0007] S2, performs physics-based simulation on the virtual sample to simulate the cryo-EM imaging process and obtain a two-dimensional image;

[0008] S3, based on slice-by-slice contrast learning, converts simple Gaussian noise in 2D imaging into real cryo-EM noise;

[0009] S4, uses contrastive learning of mask-guided sampling to effectively distinguish particle features from background features, ensuring that negative and positive samples of noise are semantically different, making the final imaging more realistic.

[0010] In S1 of some embodiments, an initial reconstruction is performed using a cryoSPARC platform, and then a continuous heterogeneous reconstruction is performed using a cryoDRGN algorithm to obtain a neural volume representation V(p, w) of the target molecular particle: in, represents the spatial coordinates, Represents high-dimensional configuration embedding;

[0011] During the placement process, the neural volume representations V(p, w) of N particles are added to an empty virtual sample one by one; each time the direction matrix is ​​randomly sampled from the SO(3) space Randomly sample conformations w from the learned configuration space i , translated from the unoccupied area of ​​the virtual sample The virtual sample can be expressed as:

[0012]

[0013] The number N~U(N min , N max ), minimum N min and maximum N max Derived from actual distribution of particle counts in real micrographs.

[0014] In S1 of some embodiments, the position, direction and configuration of the target molecule in the virtual sample are randomly distributed or uniformly distributed.

[0015] In S2 of some embodiments, the simulated cryo-EM imaging process includes the interaction between electrons and samples, and / or the simulation of ice layer gradients, and / or optical aberrations.

[0016] In S2 of some embodiments, a weak phase object approximation method is used to convert the interaction between electrons and the sample into an orthogonal projection of the sample.

[0017] In S2 of some embodiments, the ice gradient is estimated by IceBreaker software to generate an ice weight map W(x, y) to obtain the true distribution of the ice layer gradient.

[0018] In S2 of some embodiments, the point spread function g, i.e., the Fourier transform of the contrast transfer function CTF, is calculated by CTFFIND4 software to estimate the optical aberration of the cryo-EM.

[0019] In some embodiments, according to the following imaging formula, a result of two-dimensional imaging based on physics is obtained: phy (x,y):

[0020]

[0021] In some embodiments, during the simulation process, a pair of weight maps and CTF estimates are randomly selected from a set of real micro-graphs to generate micro-graphs that conform to the physical basis.

[0022] In S3 of some embodiments, I phy ~X Input conditional generative model to learn from real cryo-EM images I real The actual noise distribution of ~Y, outputting a synthetic cryo-EM image with real noise In some embodiments, the conditional generative model includes a discriminator D and a generator Among them G enc is the encoder, G dec It's a decoder.

[0023] In some embodiments, randomness is introduced during the training and inference phases of the conditional generative model by adding zero-mean Gaussian random noise integrated into a clean physics simulation, resulting in To enhance the noise generation process.

[0024] In some embodiments, during the training of the conditional generative model, I is randomly selected inter and I syn The paired patches in are used as positive samples and queries, and then I inter The K patches in are used as negative samples; the encoder G enc Map these to normalized vectors v,v + and The goal is to maximize the likelihood of selecting positive samples by minimizing the cross entropy loss:

[0025]

[0026] Here, τ is a temperature factor that scales the distance between the query and the sample.

[0027] In S4 of some embodiments, the physically based two-dimensional imaging result is converted into a binary particle-background mask M(x, y), where M(x, y)=1 represents a particle and M(x, y)=0 represents a background;

[0028] Define the position on the particle as p∈P and the position on the background as b∈B, where P represents all positions where M=1 and B represents all positions where M=0;

[0029] The feature of the lth layer at the particle position p is expressed as The set is represented as The feature representation of the lth layer at the background position b is The set is represented as

[0030] From the output I syn The encoded features are represented as

[0031] The noise contrast estimation loss function of mask-guided sampling is obtained as:

[0032]

[0033] By minimizing the MaskNCE loss, particle features and background features can be effectively distinguished.

[0034] In some embodiments, the adversarial loss function L is also included. GAN , used to promote the network to produce more realistic physically based 2D imaging results I syn :

[0035]

[0036] In some embodiments, the adversarial loss function L GAN Compared with the estimated loss function L MaskNCE Combined to form the overall training loss function:

[0037]

[0038] The present invention also provides a cryo-electron microscopy image generation system, comprising:

[0039] Particle reconstruction module, used to prepare virtual samples containing a large number of target molecular structures;

[0040] The imaging simulation module is used to perform physics-based simulation on virtual samples, simulate the cryo-EM imaging process, and obtain two-dimensional images;

[0041] The noise conversion module is used to convert simple Gaussian noise in two-dimensional imaging into real cryo-EM noise. The sampling learning module is used to use mask-guided sampling for contrastive learning to ensure that the negative and positive samples of the noise are semantically different.

[0042] The present invention also provides a cryo-electron microscopy image generating device, comprising a memory and a processor, wherein the memory includes a computer program that can be run thereon, and the processor implements any of the described methods when executing the computer program.

[0043] The present invention also provides a chip, comprising one or more processors, for calling and running a computer program from a memory, so that a device equipped with the chip executes any of the methods described above.

[0044] The present invention also provides a storage medium comprising computer executable instructions, which are used to perform any of the methods described above when executed by a computer processor.

[0045] The present invention combines physics-based simulation with contrast noise generation to improve the reconstruction results of cryo-EM images. Starting from the 3D biomolecular structure obtained by primary reconstruction, the present invention applies virtual sample projection, particle and background mask guided contrast learning sampling strategy, and unmatched noise conversion technology based on contrast learning to generate highly realistic synthetic cryo-EM datasets with annotations.

[0046] A method of virtual sample projection was proposed to accurately control the center position of particles in the generated image, solving the technical problem that it is difficult to accurately control the content of high-noise image generation. This technology surpasses existing generation methods in the visual quality of cryo-EM high-noise image generation through a contrastive learning sampling strategy guided by particles and background masks and an unmatched noise conversion technology based on contrastive learning, solving the technical problem that existing generation methods are difficult to both protect the generated image content and model highly random noise at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flowchart of an embodiment of the method of the present invention;

[0048] Figure 2 Flowchart for mask-guided contrast noise generation;

[0049] Figure 3 A schematic diagram illustrating the controllability and effects of model generation;

[0050] Figure 4 A schematic diagram of the qualitative assessment of visual quality;

[0051] Figure 5 Schematic diagram of qualitative evaluation of particle selection. DETAILED DESCRIPTION

[0052] The present invention will now be further described with reference to the accompanying drawings.

[0053] See also Figure 1 , Figure 1The first embodiment of the present invention is presented, a method for generating cryo-electron microscopy images (Cryo-EM) guided by physical information, which specifically includes:

[0054] S1, prepare a virtual sample containing a large number of target molecular structures.

[0055] To simulate the imaging process of cryo-electron microscopy, a virtual sample is first prepared, a large three-dimensional density volume containing the preliminary reconstruction results of hundreds of target molecules. Preferably, the position, orientation and configuration of the molecules in the virtual sample are randomly or evenly distributed.

[0056] This example uses the cryoSPARC platform to perform initial reconstruction, and then uses the cryoDRGN algorithm to perform continuous heterogeneous reconstruction to obtain the neural volume representation V(p, w) of the target molecular particle: in, represents the spatial coordinates, represents a high-dimensional configuration embedding; this neural volume V can be decoded into a density volume given a specific configuration embedding.

[0057] In the process of placing the target molecule, the neural volume representations V(p, w) of N particles are added one by one to an empty virtual sample. Each time it is added, the direction matrix is ​​randomly sampled from the SO(3) space Randomly sample conformations w from the learned configuration space i , translated from the unoccupied area of ​​the virtual sample The extracted directions and positions are further used as true annotations. Therefore, the virtual sample can be expressed as:

[0058]

[0059] The number N~U(N min , N max ), minimum N min and maximum N max Determined based on the distribution of particle counts in actual microscopic images.

[0060] S2, performing a physics-based simulation on the virtual sample to simulate the cryo-EM imaging process to obtain a two-dimensional image. In order to simulate the imaging process of a cryo-EM, this embodiment performs a physics-based simulation on the prepared virtual sample. This process is subdivided into three key steps: the interaction between electrons and samples, the ice layer gradient, and the simulation of the optical aberration of the cryo-EM.

[0061] Considering the computational efficiency, this embodiment uses the weak phase object approximation method to simplify the complex electron-sample interaction and convert it into an orthogonal projection of the sample.

[0062] In order to obtain the true distribution of ice layer gradient, IceBreaker software is used in this embodiment to evaluate the ice layer gradient and generate an ice weight map W(x, y) based on it. Each pixel value in the weight map represents the attenuation ratio relative to the maximum intensity value in the data set.

[0063] This embodiment uses CTFFIND4 software to estimate optical aberrations. The software calculates the point spread function g, which is the Fourier transform of the contrast transfer function (CTF). Based on this, the following imaging formula is followed to obtain the physical-based two-dimensional imaging result I phy (x,y):

[0064]

[0065] Preferably, during the simulation process, the present embodiment also randomly selects a pair of weight maps and CTF estimates from a series of real micro-maps to generate micro-maps that conform to the physical basis.

[0066] S3, based on slice-by-slice contrast learning, converts simple Gaussian noise in 2D imaging into real cryo-EM noise.

[0067] See also Figure 2 , real cryo-electron microscopy images contain high levels of complex random noise, and accurate noise generation is crucial for the generation of realistic synthetic cryo-electron microscopy images. The unpaired noise conversion technology used in this embodiment includes: 1) converting simple Gaussian noise into real low-temperature electromagnetic noise based on slice-by-slice contrast learning; 2) introducing a mask-guided convergence learning process to improve the training robustness and efficiency of model training. In addition, an adversarial loss between synthetic data and real data is also included. Among them, noise generation can be modeled as an unpaired image-to-image conversion task by using a conditional generation model. The conditional generation model includes a discriminator D, and a generator Among them G enc is the encoder, G dec is the decoder. Input I phy ~X learning from real cryo-EM images I real The actual noise distribution of ~Y to generate synthetic cryo-EM images with realistic noise Contrasting unpaired transformations and similar contrastive learning methods have shown significant potential for efficient training and generating high-quality images. However, these methods do not account for the inherent randomness in cryo-EM image noise and can therefore only generate a single synthetic micrograph from a clean physical simulation.

[0068] Furthermore, in order to solve this problem, this embodiment introduces randomness into the framework and adds zero-mean Gaussian random noise to the training and inference stages. integrated into a clean physics simulation, resulting in This simple yet effective strategy significantly enhances the noise generation process, allowing a single clean physics simulation to produce multiple synthetic cryo-EM micrographs, each showing a diverse and realistic noise pattern, including crystalline ice.

[0069] During the training process, I inter and I syn The paired patches in are used as positive samples and queries, and then I inter The K patches in are used as negative samples; the encoder G enc Map these to normalized vectors v,v + and The goal is to maximize the likelihood of selecting positive samples by minimizing the cross entropy loss:

[0070]

[0071] Here, τ is a temperature factor that scales the distance between the query and the sample.

[0072] S4 uses contrastive learning of mask-guided sampling to effectively distinguish particle features from background features, making the final imaging more realistic.

[0073] A basic assumption of contrastive learning is that randomly selected negative samples are often semantically different from positive samples. However, this principle is likely to fail when applied to cryo-EM micrographs, because hundreds of particles are densely packed against a background of almost pure noise, and this density means that randomly selected negative and positive samples may indistinguishably represent particles or noise.

[0074] Furthermore, to solve this problem, the present invention introduces a mask-guided sampling scheme, which uses synthetic data to convert the physically based two-dimensional imaging results into a binary particle-background mask:

[0075] M(x,y)

[0076] M(x, y) = 1 represents a particle and M(x, y) = 0 represents the background; this allows for more accurate identification of relevant sampling locations. This mask guides the selection process, enabling the selection of positive and negative samples from locations with contrasting mask labels. This strategy not only improves the efficiency of the training process, but also improves the quality of the synthesized micrographs.

[0077] Furthermore, the method of this embodiment utilizes G enc Multiple intermediate layers are used to extract multi-scale features from the input patch. enc The first L layers of the network are used to align the receptive field with the particle size. The feature maps of each layer are then passed through a compact two-layer MLP network Hl (l∈{1,2,…,L}) is passed to generate a feature Among them G enc It is the lth layer.

[0078] Preferably, L is set to 5 in all experiments of this embodiment.

[0079] For the particle-background mask position, the position on the particle is defined as p∈P, and the position on the background is defined as b∈B, where P represents all positions where M=1, and B represents all positions where M=0.

[0080] The feature of the lth layer at the particle position p is expressed as The set is represented as The feature representation of the lth layer at the background position b is The set is represented as From the output I syn The encoded features are represented as The noise contrast estimation loss function of mask-guided sampling is obtained as:

[0081]

[0082] By minimizing the MaskNCE loss, particle features and background features can be effectively distinguished. Furthermore, this embodiment extracts multi-scale features for calculating the MaskNCE loss according to a mask-guided sampling scheme, in which the query is in the input domain, while the positive samples and negative samples are located in the output domain.

[0083] Furthermore, the present invention also introduces the adversarial loss function L GAN , used to promote the network to produce more realistic physically based 2D imaging results I syn :

[0084]

[0085] The adversarial loss function L GAN Compared with the estimated loss function L MaskNCE Combined to form the overall training loss function:

[0086]

[0087] Set λ=10.0 as the loss weight.

[0088] The dataset with real annotations generated based on the above method can effectively improve the performance of particle selection and posture estimation models, thereby improving the reconstruction results. As a key step in using deep generative models to generate high-fidelity cryo-EM data, the embodiments of the present invention will demonstrate their important value in multiple application scenarios such as cryo-EM structure exploration.

[0089] The following is an evaluation of the proposed method using five cryo-EM datasets. First, the superiority of the proposed method in visual quality is demonstrated. Then, it is shown that the proposed method can significantly improve the performance of the model in two key cryo-EM downstream tasks - particle selection and pose estimation, and ultimately improve the reconstruction quality. Figure 3 As shown, the method of the present invention is able to generate diverse and high-quality multi-scale cryo-EM content including particles, shot noise, and even crystalline ice in the background.

[0090] Details about the implementation. This example was implemented on an NVIDIA GeForce RTX 3090 GPU. As a lightweight model, this example can complete 100 training cycles on a single dataset in less than an hour.

[0091] This embodiment preferably uses PatchGAN as the discriminator and UNet as the generator. During the training process, the image resolution is fixed to 1024×1024. With the guidance of the particle and background masks, 256 query points are uniformly sampled on the particles and the background to ensure that their corresponding negative samples are located in the opposite area of ​​the label. The training data set (all for evaluation) consists of 100 real images and 100 results based on physical simulation. We perform data augmentation on the data set by rotating the image 90 degrees, 180 degrees, and 270 degrees.

[0092] About the dataset. This example is evaluated on cryo-electron microscopy datasets, each of which contains expert-curated particle selection annotations to achieve high-resolution reconstruction of target molecules. The particle annotations in the datasets are only used in the validation phase of downstream tasks and are not used when training the GenEM model.

[0093] Specifically include:

[0094] 1) The particles in the T20S proteasome (EMPIAR-10025) micro-image are densely packed and prone to occlusion, which poses a significant challenge to particle selection. In addition, the D7 symmetry of its 3D structure adds complexity to the pose estimation task.

[0095] 2) The 80S ribosome (EMPIAR-10028) has a complex 3D structure, which makes it difficult to estimate the particle posture.

[0096] 3) Asymmetric αVβ8 integrin (EMPIAR-10345) contains molecules with different morphologies and requires heterogeneous reconstruction.

[0097] 4) The micrograph of bacteriophage MS2 (EMPIAR-10075) contains a large number of spherical virus particles, which is suitable for evaluating the generalizability of the particle selection model.

[0098] 5) Human endogenous BAF complex (EMPIAR-10590) shows obvious ice gradient artifacts. About baseline comparison. Compared with multiple traditional noise benchmarks and deep generative models, the present invention uses Poisson noise to simulate advanced detector shot noise in cryo-EM images, Gaussian noise to simplify the simulation of general noise of cryo-EM, and Poisson-Gaussian mixed noise as a traditional noise benchmark. CycleGAN and the latest CUT are selected as the baselines of deep generative models.

[0099] For visual quality assessment. Figure 4 , the method of this embodiment achieved the best visual effects in all data sets, especially in terms of overall noise pattern and particle structure information preservation. Although traditional methods are effective in preserving structural information, they fail to achieve the expected results in simulating real noise distribution. Traditional noise models (Gaussian, Poisson, PG) successfully retain structural information but lack a real noise model. CycleGAN encountered convergence difficulties in its application to the proteasome dataset, and CycleGAN and CUT introduced serious artifacts on the ribosome, PhageMS2 and HumanBAF datasets because they cannot stably generate random noise. Compared with CycleGAN, CUT has improved in visual quality, but its performance is still limited due to the use of a random sampling scheme.

[0100]

[0101] As shown in Table 1, this embodiment also uses Frechet Inception Distance (FID), which is a widely used indicator in generative tasks to measure the similarity between generated data and real data. The results of the embodiment of the present invention significantly surpass the existing baseline methods on all five datasets.

[0102] About particle picking evaluation. In order to evaluate the enhancement effect of GenEM on downstream particle picking tasks, this example fine-tuned the advanced particle picking model Topaz using synthetic annotated datasets from different baselines. Compared with the original Topaz and other baseline methods, this example scheme performed well in the quality of visual picking results. See Figure 5, the fine-tuned Topaz model was used for particle selection in five datasets. The original Topaz often incorrectly selected false particles from contaminants, particle accumulation, and ice flakes. Topaz fine-tuned using the dataset significantly improved the accuracy and reduced the number of false positives and redundant particle selections.

[0103]

[0104] Table 2 shows the quantitative results of particle picking, using the area under the precision-recall curve (AUPRC) metric, which is widely used as a particle picking evaluation metric. This metric is particularly suitable for datasets with incomplete positive labels. The original particle picking results compared with the true labels are retained when calculating the AUPRC score without any thresholding.

[0105] Quantitative comparison shows that Topaz, after fine-tuning using the data of the present invention, achieves optimal performance on all datasets. It also proves that the final reconstruction resolution of the present invention is Still surpasses all other baselines.

[0106] About pose estimation evaluation. In the cryo-EM reconstruction process, the accuracy of pose estimation is crucial to the final reconstruction resolution. Considering synthetic datasets generated from different benchmarks, its pose estimation module is directly supervised by minimizing the L2 norm of the rotation matrix. When the domain gap between the real dataset and the synthetic dataset is close enough, the performance of cryoFIRE pre-trained on the real dataset will be enhanced.

[0107] To directly verify the effect of pre-training, the pose estimation module is evaluated on a real dataset to obtain the estimated pose, and the final reconstruction result is obtained through the traditional reconstruction algorithm, filtered back projection (FBP).

[0108]

[0109] As shown in Table 3, the pose estimation module pre-trained with the data of the present invention consistently achieves the best performance in pose accuracy (rotation error in radians) and reconstruction resolution (in pixels). This comprehensive evaluation demonstrates the benefits of the present method in improving the accuracy of pose estimation and the final resolution of cryo-EM reconstruction.

[0110] The embodiment of the present invention creates a virtual sample containing many target particles, and then simulates the cryo-EM imaging process including projection, uneven ice thickness and optical aberration. Then, the simple Gaussian noise is converted into complex real noise using unpaired noise conversion. By introducing a new mask-guided sampling strategy, the training efficiency and performance are greatly improved.

[0111] Through the embodiments of the present invention, a large number of high-quality, well-annotated training data sets can be quickly generated using only a small amount of actual cryo-electron microscopy images and the computing resources of a single GPU, thereby significantly improving the performance of particle selection and pose estimation models on relevant data, thereby improving the resolution of protein structure reconstruction, and providing a powerful tool for understanding the structure and function of biological molecules.

[0112] A second embodiment of the present invention provides a cryo-electron microscopy image generation system, which can execute the method described in the first embodiment, including:

[0113] Particle reconstruction module, used to prepare virtual samples containing a large number of target molecular structures;

[0114] The imaging simulation module is used to perform physics-based simulation on virtual samples, simulate the cryo-EM imaging process, and obtain two-dimensional images;

[0115] The noise conversion module is used to convert simple Gaussian noise in two-dimensional imaging into real cryo-EM noise. The sampling learning module is used to use mask-guided sampling for contrastive learning to ensure that the negative and positive samples of the noise are semantically different.

[0116] The third embodiment of the present invention provides a cryo-electron microscopy image generating device, including a memory and a processor, wherein the memory includes a computer program that can be run thereon, and when the processor executes the computer program, the method described in the first embodiment is implemented.

[0117] The memory may include a random access memory, a flash memory, a read-only memory, a programmable read-only memory, a non-volatile memory or a register, etc. The processor may be a central processing unit (CPU), etc. or a graphic processing unit (GPU). The memory may store executable instructions. The processor may execute the executable instructions stored in the memory to implement the various processes described herein.

[0118] It can be understood that the memory in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a ROM (Read-Only Memory), a PROM (Programmable ROM), an EPROM (Erasable PROM), an EEPROM (Electrically EPROM), or a flash memory. The volatile memory can be a RAM (Random Access Memory), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as SRAM (Static RAM), DRAM (Dynamic RAM), SDRAM (Synchronous DRAM), DDRSDRAM (Double Data Rate SDRAM), ESDRAM (Enhanced SDRAM), SLDRAM (Synchlink DRAM), and DRRAM (Direct Rambus RAM). The memory described herein is intended to include, but is not limited to, these and any other suitable types of memory. In some embodiments, the memory stores the following elements, upgrade packages, executable units, or data structures, or a subset thereof, or an extended set thereof: an operating system and an application.

[0119] Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program includes various application programs for implementing various application services. The program for implementing the method of the embodiment of the present invention may be included in the application program. In the embodiment of the present invention, the processor calls the program or instructions stored in the memory, specifically, the program or instructions stored in the application program, and the processor is used to execute the method steps provided in the first aspect. The fourth embodiment of the present invention provides a chip, including one or more processors, for calling and running a computer program from a memory, so that a device equipped with the chip executes the method described in the first embodiment.

[0120] A fifth embodiment of the present invention provides a storage medium containing computer executable instructions, which are used to perform the method described in the first embodiment when executed by a computer processor. For example, the machine-readable storage medium may include but is not limited to various known and unknown types of non-volatile memories.

[0121] A sixth embodiment of the present invention further provides a computer program product, including computer program instructions, which enable a computer to execute the method described in the first embodiment.

[0122] It will be appreciated by those skilled in the art that the units and steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Claims

1. A method for generating cryo-electron microscopy images guided by physical information, characterized in that include: S1, prepare virtual samples containing a large number of target molecular structures; S2, performs physics-based simulation on the virtual sample to simulate the cryo-EM imaging process and obtain a two-dimensional image; S3, introduces randomness in the training and inference stages of the conditional generative model, integrating zero-mean Gaussian random noise into the clean physical simulation to enhance the noise generation process; During the training of the conditional generative model, pairs of patches in the intermediate image and the synthetic cryo-EM image are randomly selected as positive samples and queries, and then several patches of the intermediate image are selected as negative samples, and the probability of selecting positive samples is maximized by minimizing the cross entropy loss; Using a conditional generative model, we learn the actual noise distribution from real cryo-EM images based on slice-by-slice comparison, and transform simple Gaussian noise in two-dimensional imaging into real cryo-EM noise. S4, uses contrastive learning of mask-guided sampling to effectively distinguish particle features from background features, making the final imaging more realistic.

2. The method according to claim 1, characterized in that: In S1, the neural volume representation V(p, w) of the target molecular particle is obtained through initial reconstruction and continuous isomerization reconstruction: in, represents the spatial coordinates, Represents high-dimensional configuration embedding; During the placement process, the neural volume representations V(p, w) of N particles are added to an empty virtual sample one by one; each time the direction matrix is ​​randomly sampled from the SO(3) space Randomly sample conformations w from the learned configuration space i , translated from the unoccupied area of ​​the virtual sample The virtual sample is represented as: The number N~U(N min , N max ), minimum N min and maximum N max Derived from actual distribution of particle counts in real micrographs.

3. The method according to claim 1, characterized in that: In S1, the position, orientation, and configuration of the target molecules in the virtual sample are randomly distributed or uniformly distributed.

4. The method according to claim 2, characterized in that: In S2, the simulated cryo-electron microscopy imaging process includes the interaction between electrons and samples, and / or the simulation of ice layer gradients, and / or optical aberrations.

5. The method according to claim 4, characterized in that: In S2, the weak phase object approximation method is used to transform the interaction between the electron and the sample into an orthogonal projection of the sample.

6. The method according to claim 4, characterized in that: In S2, the ice gradient is estimated by IceBreaker software to generate the ice weight map W(x, y) and obtain the true distribution of the ice layer gradient.

7. The method according to claim 6, characterized in that: In S2, the point spread function g, i.e., the Fourier transform of the contrast transfer function CTF, was calculated by CTFFIND4 software to estimate the optical aberration of the cryo-EM.

8. The method according to claim 7, characterized in that: According to the following imaging formula, the result of two-dimensional imaging based on physics is obtained: phy (x,y):

9. The method according to claim 8, characterized in that: During the simulation, a pair of weight maps and CTF estimates are randomly selected from the real micro-atlas set to generate micro-atlases that conform to the physical basis.

10. The method according to claim 8, characterized in that: In S3, the physical two-dimensional imaging result I phy ~X Input conditional generative model to learn from real cryo-EM images I real The actual noise distribution of ~Y, outputting a synthetic cryo-EM image with real noise 11. The method according to claim 10, characterized in that: The conditional generation model consists of a discriminator D and a generator Among them G enc is the encoder, G dec It's a decoder.

12. The method according to claim 11, characterized in that: Introducing randomness during the training and inference phases of conditional generative models, 13. The method according to claim 12, characterized in that: During the training of the conditional generative model, the encoder G enc Map positive samples, queries, and negative samples to normalized vectors v, v + and Here, τ is a temperature factor that scales the distance between the query and the sample.

14. The method according to claim 13, characterized in that: In S4, the physical-based two-dimensional imaging result is converted into a binary particle-background mask M(x, y), where M(x, y)=1 represents a particle and M(x, y)=0 represents a background; Define the position on the particle as p∈P and the position on the background as b∈B, where P represents all positions where M=1 and B represents all positions where M=0; The feature of the lth layer at the particle position p is expressed as The set is represented as The feature representation of the lth layer at the background position b is The set is represented as From the output I syn The encoded features are represented as The noise contrast estimation loss function of mask-guided sampling is obtained as: By minimizing the MaskNCE loss, particle features and background features can be effectively distinguished.

15. The method according to claim 14, characterized in that: It also includes the adversarial loss function L GAN , used to promote the network to produce more realistic physically based 2D imaging results I syn :

16. The method according to claim 15, characterized in that: The adversarial loss function L GAN Compared with the estimated loss function L MaskNCE Combined to form the overall training loss function: Where λ is the loss weight.

17. A cryo-electron microscopy image generation system, characterized in that include: Particle reconstruction module, used to prepare virtual samples containing a large number of target molecular structures; The imaging simulation module is used to perform physics-based simulation on virtual samples, simulate the cryo-EM imaging process, and obtain two-dimensional images; A noise conversion module for introducing randomness during the training and inference phases of conditional generative models, integrating zero-mean Gaussian random noise into clean physical simulations to enhance the noise generation process; It is used to randomly select pairs of patches in the intermediate image and the synthetic cryo-EM image as positive samples and queries during the training process of the conditional generation model, and then select several patches in the intermediate image as negative samples, maximizing the probability of selecting positive samples by minimizing the cross entropy loss; Used to use conditional generative models to convert simple Gaussian noise in 2D imaging into realistic cryo-EM noise based on slice-by-slice contrast learning; The sampling learning module is used for contrastive learning with mask-guided sampling to ensure that the noisy negative and positive samples are semantically different.

18. A cryo-electron microscopy image generation device, characterized in that: The method comprises a memory and a processor, wherein the memory comprises a computer program executable thereon, and when the processor executes the computer program, the method according to any one of claims 1 to 16 is implemented.

19. A chip, characterized in that: It comprises one or more processors, which are used to call and run a computer program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 1 to 16.

20. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to perform the method according to any one of claims 1 to 16 when executed by a computer processor.

Images