Cryo-em image processing method and device, terminal and storage medium

Abstract

The present disclosure provides a cryo-EM image processing method and device, a terminal and a storage medium. The cryo-EM image processing method comprises: acquiring a cryo-EM image; encoding the cryo-EM image into a latent variable through an encoder; converting the latent variable into an atomic model structure through a decoder; correcting the atomic model structure by using a loss function to obtain a corrected atomic model structure, wherein the loss function comprises a bond length constraint loss function, a collision constraint loss function and a spring constraint loss function; and converting the corrected atomic model structure into a density map represented by a Gaussian ball through a projector module, and projecting the density map to obtain a projection image. By adopting the bond length constraint, the collision constraint and the spring constraint, the local structure stability can be maintained, the solution space can be greatly reduced, and more efficient and accurate multi-conformation analysis can be realized.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of information technology, and in particular, to a cryo-EM image processing method and device, a terminal and a storage medium. BACKGROUND

[0002] Cryo-EM technology is a biological macromolecular structure analysis technology that has become popular in the past two decades and is growing. It can analyze the structure of dynamic and difficult-to-crystallize biological macromolecular complexes. Cryo-EM is a microscopy technique that uses a transmission electron microscope to observe samples at low temperature. By freezing the sample (protein is constantly moving at room temperature) and keeping it at low temperature in the microscope, a highly coherent electron beam is used as the light source. When passing through the sample and the surrounding ice layer, some electrons are scattered by the heavier atoms in the sample. These scattered signals are imaged and recorded by a detector and lens system, and finally processed to form a projection image, and then the sample structure is obtained.

[0003] The signal-to-noise ratio of cryo-EM images is very low, usually in the order of 0.001-0.0001. For an atomic model with very high degrees of freedom, if the solution space is not limited, it will be easily fitted to the noise. Existing multi-conformation analysis based on atomic models usually limits the solution space by performing normal mode analysis (NMA) on the atomic structure in advance, but NMA itself is very limited by the initial state structure used for analysis, and is suitable for modeling the local vibration of the protein near the steady state. Therefore, it is desirable to develop a method that can accurately and efficiently analyze the dynamic conformation of the cryo-EM image of the protein. SUMMARY

[0004] To solve the existing problems, the present disclosure provides a cryo-EM image processing method and device, a terminal and a storage medium.

[0005] The present disclosure adopts the following technical solutions.

[0006] An embodiment of the present disclosure provides a cryo-EM image processing method, which comprises: acquiring a cryo-EM image; encoding the cryo-EM image into a latent variable through an encoder; converting the latent variable into an atomic model structure through a decoder; modifying the atomic model structure using a loss function to obtain a modified atomic model structure, wherein the loss function comprises a bond length constraint loss function, a collision constraint loss function and a spring constraint loss function; converting the modified atomic model structure into a density map represented by a Gaussian sphere through a projector module, and projecting the density map to obtain a projection image.

[0007] Another embodiment of the present disclosure provides a cryo-EM image processing device, the control device comprising: an image acquisition module configured to acquire a cryo-EM image; an encoding module configured to encode the cryo-EM image into a latent variable through an encoder; a decoding module configured to convert the latent variable into an atomic model structure through a decoder; a correction module configured to correct the atomic model structure using a loss function to obtain a corrected atomic model structure, wherein the loss function comprises a bond length constraint loss function, a collision constraint loss function and a spring constraint loss function; a projection module configured to convert the corrected atomic model structure into a density map represented by a Gaussian ball through a projector module, and project the density map to obtain a projection image.

[0008] In some embodiments, the present disclosure provides a terminal, comprising: at least one memory and at least one processor; wherein the memory is configured to store program code, and the processor is configured to invoke the program code stored in the memory to execute the above-mentioned cryo-EM image processing method.

[0009] In some embodiments, the present disclosure provides a storage medium for storing program code, the program code being used to execute the above-mentioned cryo-EM image processing method.

[0010] The present disclosure corrects the atomic model structure by using a loss function, realizes structure constraint through bond length constraint and collision constraint, so that each Gaussian ball in the Gaussian density representation corresponds to each amino acid of the protein, and the atomic modeling structure at the amino acid level is directly introduced into the three-dimensional reconstruction of cryo-EM. In addition, the spring constraint can maintain the stability of the local structure, greatly reduce the solution space, and realize more efficient and accurate multi-conformation analysis. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings. Throughout the drawings, like or similar reference numerals designate identical or similar elements throughout the several views. It should be understood that the drawings are schematic and elements and features are not necessarily to scale.

[0012] Figure 1 is a flowchart of a cryo-EM image processing method of an embodiment of the present disclosure.

[0013] Figure 2 shows the Gaussian density representation and projection of a protein of an embodiment of the present disclosure.

[0014] Figure 3 is a flowchart of a cryo-EM image processing method of an embodiment of the present disclosure.

[0015] Figure 4is an example constraint way of atomic model structure of a protein of embodiments of the present disclosure.

[0016] Figure 5 An example determination way of a cutoff frequency of low frequency filtering of embodiments of the present disclosure is shown.

[0017] Figure 6 Dynamic revision of spring constraint of embodiments of the present disclosure is shown.

[0018] Figure 7 Atomic model structures and projections of proteins resolved with different methods are shown.

[0019] Figure 8 is a partial module for a cryo-EM image processing device of another embodiment of the present disclosure.

[0020] Figure 9 is a structural schematic diagram of an electronic device of embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described in more detail by referring to the attached drawings. Although certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, but rather the embodiments are provided to more thoroughly and completely understand the present disclosure. It is understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of protection of the present disclosure.

[0022] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be executed in series and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0023] The term "comprising" and variations thereof as used herein are open-ended, that is, "comprising but not limited to." The term "based on" is "based, at least in part, on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The term "some embodiments" means "at least some embodiments." Related definitions are given below in the description of the application.

[0024] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0025] It should be noted that the modification of "one" mentioned in the present disclosure is illustrative but not restrictive, and those skilled in the art should understand that "one or more" should be understood unless otherwise explicitly indicated in the context.

[0026] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0027] Proteins are not static in solution, and some transition state protein conformations are also recorded at the moment of freezing, which makes single-particle cryo-EM not only able to resolve several low-energy conformations of a protein, but also has the potential to resolve the transition conformations between the low-energy conformations. In cryo-EM images, each data set contains many 2D projections from one or more 3D structures in potentially different conformational states, so a cryo-EM image data set can contain rich heterogeneity information at the conformational level. However, existing cryo-EM methods have not been able to fully exploit the conformational information "hidden" in the large amount of cryo-EM projection data.

[0028] For the cryo-EM multi-conformation analysis problem of a given projection direction, the goal is to determine the conformational class of each projection image (particle). Density Figure 1 Generally refers to a three-dimensional object directly reconstructed by back-projection of cryo-EM images. For real cryo-EM data, it is not known how many conformational classes are in the data; the motion of the protein is not abrupt, and the multi-conformation analysis should also be continuous, which is the reason for the generative modeling.

[0029] Some existing methods use a coding-decoding network to predict the NMA coefficients of conformational changes, which obtains some bases of different motion modes by performing NMA on a reference structure (a given initial structure), and the model predicts the linear combination parameters of the bases to represent the conformational changes of the structure itself. The disadvantage of this method is that NMA is strongly dependent on the initial structure, the bases of motion modes are some oscillations near the initial structure, the number of NMA bases also needs to be manually selected, and this method can only be successful on simulated data and cannot be applied to real cryo-EM data.

[0030] The present disclosure proposes a method based on atomic model regularization, which greatly reduces the search space of the algorithm by maintaining the stability of the local structure, and avoids the bias from the initial atomic structure through adaptive relaxation means. The generative model can directly output a reasonable atomic model dynamic at the amino acid level. Therefore, through the atomic model information, the dynamic conformation in the cryo-EM data can be analyzed, and at the same time, a reasonable protein backbone model containing dynamic conformation can be given.

[0031] Figure 1 A flowchart of a cryo-electron microscopy image processing method according to embodiments of the present disclosure is provided. The cryo-electron microscopy image processing method of the present disclosure may include step S101, acquiring a cryo-electron microscopy image. A cryo-electron microscopy image is an image captured by a cryo-electron microscope. When analyzing protein structures using a cryo-electron microscope, a cryo-electron microscopy image of the protein is acquired first. In some embodiments, the cryo-electron microscopy image is a cryo-electron microscopy image of the protein.

[0032] In some embodiments, the method of this disclosure may further include step S102, encoding the cryo-electron microscopy image into latent variables using an encoder. In some embodiments, the encoder is a multilayer neural network that encodes the input low signal-to-noise ratio cryo-electron microscopy image into low-dimensional latent variables. In some embodiments, the latent variables include vectors of shape 8. In some embodiments, the latent variables contain conformational information of the protein, and by analyzing the latent space determined by these latent variable vectors, multi-conformation analysis of the data is achieved.

[0033] In some embodiments, the method of this disclosure may further include step S103, converting latent variables into atomic model structures using a decoder. In some embodiments, the decoder is a multilayer neural network that converts latent variables into different conformations, specifically atomic model structures. In some embodiments, the atomic model structure includes a matrix of shape N×3, where N refers to the number of amino acids and 3 refers to the coordinates (x, y, z) of each amino acid.

[0034] To facilitate understanding, the following will be combined with... Figure 2 Describing the Gaussian density characterization and projection of proteins. For example... Figure 2 As shown in (1), a protein is formed by the folding of amino acids along a single chain. Figure 2 As shown in (2), amino acids are used as basic units, and each amino acid is used as a node to characterize a protein. Nodes are chain-like structures connected in pairs. Figure 2 As shown in (3), a ternary Gaussian distribution is constructed with the nodes as the center and the total number of electrons in the amino acids as the intensity, serving as the potential density of the protein, thus obtaining the Gaussian density representation of the protein. In some embodiments, the Gaussian density representation is assigned by isotropic Gaussians, with each amino acid's Gaussian sphere centered on the C-alpha atom of that amino acid. Its value is positively correlated with the total number of electrons in all atoms composing that amino acid, and its variance is manually given, typically set to 2, thereby obtaining the density map represented by the Gaussian sphere. Figure 2 As shown in (4) in the figure, given the potential density of a protein, the protein can be projected along its z-axis direction according to its orientation to obtain a projected image.

[0035] In some embodiments, the method of the present disclosure can further comprise a step S104 of correcting the atomic model structure by using a loss function to obtain a corrected atomic model structure, wherein the loss function comprises a bond length constraint loss function, a collision constraint loss function, and a spring constraint loss function. In some embodiments, the bond length constraint loss function, the collision constraint loss function, and the spring constraint loss function correspond to the bond length constraint, the collision constraint, and the spring constraint of the atomic model structure of the protein, respectively. Figure 4 An example constraint manner of the atomic model structure of the protein of the embodiments of the present disclosure is shown. Figure 4 (1) in (1) is a reference protein structure, and a, b, c, and d are example amino acids. Figure 4 (2) in (2) is an example of the bond length constraint, which requires the distance between adjacent amino acids of the protein structure to remain a constant. Figure 4 The edges of a and b in (2) are stretched, which does not satisfy the bond length constraint, and the bond length can be shortened by the bond length constraint loss function. In some embodiments, it is assumed that the distance between two amino acids in the predicted protein structure is The mathematical form of the constraint is wherein is the distance between the C-alpha of two adjacent amino acids on a chain of the protein. Figure 4 (3) in (3) is an example of the collision constraint, which requires the distance between non-adjacent amino acids of the protein structure to be not too close, and the distance between them can be stretched by the constraint loss function. Figure 4 a and c in (3) are an example that does not satisfy the collision constraint. In some embodiments, it is assumed that the distance between two amino acids in the predicted protein structure is The mathematical form of the constraint is Figure 4 (4) in (4) is an example of the spring constraint, which establishes a constraint similar to a “spring” between amino acids within a certain distance threshold. The effect of the “spring” is that no matter whether the distance between adjacent amino acids is stretched or compressed, an opposite force will be generated to resist such change. In some embodiments, the spring constraint is only constructed within the same amino acid chain, and is not constructed between chains. In some embodiments, it is assumed that the distance between two amino acids in the predicted protein structure is The distance of the reference structure is d, and the mathematical form of the constraint is

[0036] In some embodiments, the bond length constraint loss function L cont ensures that the connection between two adjacent residues remains intact, and the bond length constraint loss function L cont is expressed as:

[0037]

[0038] where d i，i+1 denotes the distance between two adjacent residues in the reference protein structure, denotes the predicted distance between the same pair of residues.

[0039] In some embodiments, in order to prevent the collision of two residues after the prediction of the deformation, a collision constraint loss function L clash is calculated as follows:

[0040]

[0041]

[0042] where P clash denotes the set of pairs of residues that collide during training, (i, j) is the index of the residues in the structure, denotes the predicted distance between two non-adjacent residues, which is penalized or inversely compensated when the two non-adjacent residues are closer than a length K clash of 4 angstroms. clash

[0043] In some embodiments, in order to maintain the rigidity of the local structure, a spring constraint loss function L EN is adopted, and L EN is calculated as follows:

[0044]

[0045]

[0046] where (i, j) is the index or exponent pair in the spring network P EN , and for each residue with index i, denotes the adjacent index set of the pair distance d ij in the reference structure, where k EN is a constant of 12 angstroms, d ij are the distances between the centers of the two residues of the predicted structure and the reference atomic model structure, respectively.

[0047] In some embodiments, by implementing the structure constraint through the bond length constraint and the collision constraint, each Gaussian sphere in the Gaussian density representation is made to correspond to each amino acid of the protein one by one, and the atomic modeling at the amino acid level is directly introduced into the three-dimensional reconstruction of the cryo- electron microscopy. In addition, by the spring constraint, the local structure is maintained stable, the search space or solution space is greatly reduced, and more efficient and accurate multi-conformation analysis is achieved.

[0048] ​In some embodiments, the method of the present disclosure can further comprise a step S105 of converting the corrected atomic model structure into a density map represented by Gaussian spheres by a projector module, and projecting the density map to obtain a projected image. In some embodiments, the projector module is an integral module for a three-element Gaussian distribution, which converts the atomic model structure into a density map represented by Gaussian spheres, and completes the projection under a given projection visual angle to obtain a noise-free output projected image. It should be understood that the projector module is commonly used and known, and the projection process is not related to machine learning, and there is no parameter to be learned. Figure 3 A flowchart of a cryo-EM image processing method of an embodiment of the present disclosure is shown, and the overall framework of the model is a variational autoencoder (VAE) structure. The main purpose of cryo-EM image reconstruction is to reconstruct the 3D structure of a protein from its 2D projections Reconstructing a 3D body wherein D is the side length of the picture, and M is the size of the data set. Figure 3 (1) is a cryo-EM picture (i.e., an input image ); the cryo-EM image is encoded into a latent variable (3) by an encoder (2), i.e., the role of the encoder is to The latent variable is converted into an atomic model structure (5) by a decoder (4) (i.e., the atomic model structure of the protein is derived by a VAE-based shape prediction network comprising the encoder and the decoder ), i.e., the role of the decoder is to N is the number of amino acids, and 3 corresponds to the coordinates (x, y, z) of the corresponding amino acid; the encoder and the decoder are both multi-layer neural networks (MLP), the MLP is a global feature extractor, and the hidden dimensions of the encoder and the decoder are set to (512, 256, 128, 64, 32) and (32, 64, 128, 256, 512), respectively, and the size of the latent space is 8. The atomic model structure can then be corrected using a loss function; the projector module (7) can convert the corrected atomic model structure into a density map represented by Gaussian spheres (i.e., a volume representation ), and project the density map under a given pose (6) to obtain a projected image (8).

[0049] For single-particle cryo-EM image data containing dynamics, due to the huge potential structure search space, existing methods are difficult to find the correct conformational space to reconstruct its dynamic changes. The present disclosure introduces prior information from the atomic model structure through a reasonable loss function regularization term, reduces the structure search space, and realizes more efficient and accurate multi-conformation analysis to obtain a smooth and reasonable dynamic. When analyzing continuous conformations from cryo-EM data, existing methods can only obtain a series of protein density maps containing dynamics, and reasonable atomic model structures require subsequent tedious manual modeling and adjustment. The present disclosure uses Gaussian balls to associate atomic models and their corresponding density maps, and through bond length constraints and collision constraints, the Gaussian balls and amino acid coordinates are corresponded, so that a reasonable atomic model containing dynamics at the amino acid level is directly obtained by a generative model. Through bond length constraints and collision constraints, structure constraints are realized, so that each Gaussian ball in the Gaussian density representation can correspond to each amino acid of the protein, and atomic modeling at the amino acid level is directly introduced into three-dimensional reconstruction of cryo-EM. In addition, through spring constraints, the local structure is maintained to be stable, the search space or solution space is greatly reduced, and more efficient and accurate multi-conformation analysis is realized.

[0050] In some embodiments, the loss function further includes a reconstruction loss function determined based on a mean square error between the cryo-EM image and the projection image. In some embodiments, the cryo-EM image processing method further includes, before determining the reconstruction loss function, low-frequency filtering the cryo-EM image and the projection image. The method of the present disclosure only fits the low-frequency signals in the original image data, so consistent low-frequency filtering is performed on the input and output images before calculating the similarity loss function of the images. In some embodiments, the low-frequency signal part in the cryo-EM image is sufficient to capture the large dynamics of the protein conformation, and the present disclosure uses a coarse-grained atomic model (Gaussian ball centered on C-alpha at the amino acid level) to cooperate with low-frequency filtering of the image, only fitting the low-frequency signals in the image, and avoiding the shortcomings of modeling the cryo-EM density map by Gaussian balls.

[0051] In some embodiments, the cutoff frequency of the low-frequency filtering is determined based on a Fourier shell correlation (FSC) between the reconstruction result of the cryo-EM image and the density map. In some embodiments, the cutoff frequency of the low-frequency filtering is determined by calculating the Fourier shell correlation (FSC) between the reconstruction result of the cryo-EM traditional software (for example, EMAN2) and the Gaussian density map. Figure 5 An example determination method of the cutoff frequency of the low-frequency filtering of the embodiments of the present disclosure is shown. Figure 5 (a) is the reconstruction density map of the traditional software EMAN2, (b) is the Gaussian density map, and (c) is the FSC calculated between (a) and (b), taking 0.5 as the threshold (the correlation coefficient of the signal below a certain frequency is higher than 0.5), and determining the cutoff frequency as 1 / 16.9.

[0052] In some embodiments, the cryo-EM image processing method of the present disclosure further comprises: when the atomic model structure is modified by using the spring constraint loss function, calculating the variance of the length of the spring between the amino acids, and discarding the springs ranked within a preset percentage when the variance is sorted from large to small. This is a dynamic modification method of spring constraint based on variance. In some embodiments, the preset percentage is 5%. Figure 6 In (1), (2), (3) and (4) of FIG. 1, (1), (2), (3) and (4) are four different structures predicted by the model, wherein the length variance of edges a-c is the largest, and the length variance of other edges is relatively small. The edge with large variance may be caused by protein dynamics, that is, the motion amplitude of amino acid (a) is relatively large, resulting in a relatively large change in the length of a-c. Therefore, the spring constraint of edge a-c can be discarded according to the prediction result. In some embodiments, for n springs (one spring is formed between two amino acids, it should be understood that the spring is not a physical spring, but a hypothetical spring or two that forms a spring force), the variance of the length of each spring is calculated, and then the springs with the largest variance (i.e., ranked within the top 5% when the variance is sorted from large to small) are discarded. This self-adaptive modification of the spring based on variance realizes the automatic relaxation of the spring constraint, which is conducive to promoting more accurate conformation analysis and obtaining smooth and reasonable dynamic conformation. That is, most of the local structures are maintained, and falling into the energy minimum value of the initial structure is avoided.

[0053] In some embodiments, too weak constraint allows free search of the atomic model in the search space, which cannot maintain the stability of the local structure, and thus the analyzed conformation is far from the original conformation of the protein. On the other hand, if the constraint is too strong, it will cause the atomic model structure to vibrate near the low-energy state of the initial structure of the protein. By using the automatic relaxation of the spring constraint, more accurate protein conformation analysis can be performed. Figure 7 (1) of FIG. 1 shows the closed initial state of the protein. Figure 7 (2) of FIG. 1 shows the atomic model structure analyzed by the commonly used weak constraint model that only forces the continuity of the skeleton and its projection, which can be seen that the local structure (for example, the a-helix) of the well-folded protein is destroyed. Figure 7 (3) of FIG. 1 shows the atomic model structure and its projection using the constraint method of the self-adaptive modified spring, which successfully reconstructs the open state of the protein. Figure 7 (4) of FIG. 1 shows the atomic model structure and its projection using the commonly used NMA-based model. The interlocking problem of the NMA-based model (the adjacent nodes of the initial structure tend to remain along the direction of the basis, forcing a large energy barrier from the closed state to the open state) makes it unable to analyze the transition from the closed to the open.

[0054] Embodiments of the present disclosure also provide a cryo-EM image processing apparatus 400. Figure 8 A cryo-EM image processing apparatus 400 according to some embodiments is shown. The cryo-EM image processing apparatus 400 comprises an image acquisition module 401, an encoding module 402, a decoding module 403, a correction module 404, and a projection module 405. In some embodiments, the image acquisition module 401 is configured to acquire a cryo-EM image. In some embodiments, the encoding module 402 is configured to encode the cryo-EM image into a latent variable by an encoder. In some embodiments, the decoding module 403 is configured to convert the latent variable into an atomic model structure by a decoder. In some embodiments, the correction module 404 is configured to correct the atomic model structure by a loss function to obtain a corrected atomic model structure, wherein the loss function comprises a bond length constraint loss function, a collision constraint loss function, and a spring constraint loss function. In some embodiments, the projection module 405 is configured to convert the corrected atomic model structure into a density map represented by Gaussian spheres by a projector module, and project the density map to obtain a projection image.

[0055] It should be understood that what is described with respect to the cryo-EM image processing method also applies to the cryo-EM image processing apparatus 400 herein, which is not described in detail for the purpose of simplicity.

[0056] In some embodiments, the latent variable comprises an 8-dimensional vector. In some embodiments, the atomic model structure comprises an N x 3 matrix, N corresponding to the number of amino acids, and 3 corresponding to the spatial coordinates (x, y, z) of the corresponding amino acid. In some embodiments, the loss function further comprises a reconstruction loss function determined based on the variance between the cryo-EM image and the projection image. In some embodiments, the cryo-EM image processing apparatus further comprises a filtering module configured to low-frequency filter the cryo-EM image and the projection image before determining the reconstruction loss function. In some embodiments, the cut-off frequency of the low-frequency filtering is determined based on the Fourier shell correlation between the reconstruction result of the cryo-EM image and the density map. In some embodiments, the correction module is further configured to, when correcting the atomic model structure by the spring constraint loss function, calculate the variance of the length of the spring between the amino acids, and discard the springs ranked within a preset percentage when the variances are sorted from large to small.

[0057] Further, the present disclosure also provides a terminal comprising at least one memory and at least one processor; wherein the memory is configured to store program code, and the processor is configured to invoke the program code stored in the memory to perform the above-mentioned cryo-EM image processing method.

[0058] In addition, the disclosure also provides a computer storage medium, which stores program codes for executing the above-mentioned cryo-EM image processing method.

[0059] The above describes the cryo-EM image processing method and device of the disclosure based on the embodiments and application examples. In addition, the disclosure also provides a terminal and a storage medium, which are described below.

[0060] The following refers to Figure 9 which shows a structural schematic diagram of an electronic device (such as a terminal device or a server) 500 suitable for implementing the embodiments of the disclosure. The terminal device in the embodiments of the disclosure can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 9 The electronic device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the disclosure.

[0061] As shown in Figure 9 , the electronic device 500 can include a processing device (such as a central processor, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 502 or programs loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0062] Generally, the following devices can be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 508 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 509. The communication devices 509 can allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 9 The electronic device 500 is shown with various devices, but it should be understood that all the shown devices are not required to be implemented or provided. More or fewer devices can be alternatively implemented or provided.

[0063] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0064] It should be noted that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to wire, cable, optical fiber, RF, etc., or any suitable combination of the above.

[0065] In some embodiments, the client, server, or other computing machines utilized by the system can communicate information using any known or future developed end-to-end communication protocol, such as the HyperText Transfer Protocol (HTTP), and can be interconnected via any form or medium of digital data communication (for example, a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet, and peer-to-peer networks (for example, ad hoc peer-to-peer networks), as well as any current or future developed network.

[0066] The computer-readable medium described above can be included in the electronic device described above; alternatively, the computer-readable medium can exist as a standalone entity.

[0067] The computer-readable medium described above can be included in the electronic device described above; alternatively, the computer-readable medium can exist as a standalone entity.

[0068] Computer program code for carrying out operations of the present disclosure can be written in any of one or more programming languages, including object oriented programming languages such as Java, Smalltalk, C++, or conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network ("LAN") or a wide area network ("WAN"), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0069] The computer program product of the first aspect can include one or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the operations of the first aspect. The computer program product of the first aspect can include a non-transitory computer-readable medium storing code that, when executed, causes a computer to perform operations for the first aspect.

[0070] The units described in the embodiments of the present disclosure can be implemented by hardware, software, or a combination of hardware and software. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0071] The functions described in this document can be performed at least in part by one or more hardware logic components. For example, and without limitation, example types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0072] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0073] According to one or more embodiments of the present disclosure, a cryo-EM image processing method is provided, which includes: acquiring a cryo-EM image; encoding the cryo-EM image into a latent variable by an encoder; converting the latent variable into an atomic model structure by a decoder; correcting the atomic model structure by a loss function to obtain a corrected atomic model structure, wherein the loss function includes a bond length constraint loss function, a collision constraint loss function, and a spring constraint loss function; converting the corrected atomic model structure into a density map represented by a Gaussian sphere by a projector module, and projecting the density map to obtain a projection image.

[0074] According to one or more embodiments of the present disclosure, the latent variable includes an 8-dimensional vector.

[0075] According to one or more embodiments of the present disclosure, the atomic model structure includes an N x 3 matrix, N corresponding to the number of amino acids, and 3 corresponding to the spatial coordinates (x, y, z) of the corresponding amino acids.

[0076] According to one or more embodiments of the present disclosure, the loss function further includes a reconstruction loss function determined based on the variance between the cryo-EM image and the projection image.

[0077] According to one or more embodiments of the present disclosure, the cryo-EM image processing method further includes: before determining the reconstruction loss function, low-frequency filtering the cryo-EM image and the projection image.

[0078] According to one or more embodiments of the present disclosure, the cutoff frequency of the low-frequency filtering is determined based on the Fourier shell correlation between the reconstruction result of the cryo-EM image and the density map.

[0079] According to one or more embodiments of the present disclosure, the cryo-EM image processing method further includes: when the atomic model structure is corrected by the spring constraint loss function, calculating the variance of the length of the spring between the amino acids, and discarding the springs ranked within a preset percentage when the variances are sorted from large to small.

[0080] According to one or more embodiments of this disclosure, a cryo-electron microscopy image processing apparatus is provided, the apparatus comprising: an image acquisition module configured to acquire cryo-electron microscopy images; an encoding module configured to encode the cryo-electron microscopy images into latent variables using an encoder; a decoding module configured to convert the latent variables into an atomic model structure using a decoder; a correction module configured to correct the atomic model structure using a loss function to obtain a corrected atomic model structure, wherein the loss function includes a bond length constraint loss function, a collision constraint loss function, and a spring constraint loss function; and a projection module configured to convert the corrected atomic model structure into a density map represented by a Gaussian sphere using a projector module, and to project the density map to obtain a projected image.

[0081] According to one or more embodiments of the present disclosure, a terminal is provided, comprising: at least one memory and at least one processor; wherein the at least one memory is used to store program code, and the at least one processor is used to invoke the program code stored in the at least one memory to execute the method described in any one of the above descriptions.

[0082] According to one or more embodiments of the present disclosure, a storage medium is provided for storing program code for performing the methods described above.

[0083] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0084] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0085] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A cryo-electron microscopy image processing method, characterized by, The cryo-EM image processing method comprises: obtaining a cryo-EM image, the cryo-EM image being a cryo-EM image for a protein taken by a cryo-electron microscope; encoding the cryo-EM image into a latent variable by an encoder; converting the latent variable into an atomic model structure by a decoder; correcting the atomic model structure by a loss function to obtain a corrected atomic model structure, wherein the loss function comprises a bond length constraint loss function, a collision constraint loss function and a spring constraint loss function, in the process of correcting the atomic model structure, the bond length constraint loss function indicates shortening the distance between adjacent amino acids in the protein that do not satisfy the bond length constraint, and the spring constraint loss function indicates that when the distance between adjacent amino acids in the protein is stretched or shortened, an opposite force is generated; transforming the corrected atomic model structure into a density map represented by Gaussian spheres by a projector module, and projecting the density map to obtain a projection image.

2. The cryo-EM image processing method of claim 1, wherein, The latent variable comprises an 8-dimensional vector.

3. The cryo-EM image processing method of claim 1, wherein, The atomic model structure comprises an N×3 matrix, N corresponding to the number of amino acids, and 3 corresponding to the spatial coordinates (x, y, z) of the corresponding amino acids.

4. The cryo-EM image processing method of claim 1, wherein, The loss function further comprises a reconstruction loss function determined based on the variance between the cryo-EM image and the projection image.

5. The cryo-EM image processing method of claim 4, wherein, Further comprising: Before determining the reconstruction loss function, low-frequency filtering is performed on the cryo-EM image and the projection image.

6. The cryo-EM image processing method of claim 5, wherein, The cutoff frequency of the low-frequency filtering is determined based on the Fourier shell correlation between the reconstruction result of the cryo-EM image and the density map.

7. The cryo-EM image processing method of claim 1, wherein, Further comprising: When the atomic model structure is corrected by the spring constraint loss function, the variance of the length of the spring between the amino acids is calculated, and the springs ranked within a preset percentage in the order of variance from large to small are discarded.

8. A cryo-EM image processing device, comprising: The cryo-EM image processing device comprises: an image acquisition module configured to obtain a cryo-EM image, the cryo-EM image being a cryo-EM image for a protein taken by a cryo-electron microscope; an encoding module configured to encode the cryo-EM image into a latent variable by an encoder; a decoding module configured to convert the latent variable into an atomic model structure by a decoder; a correction module configured to correct the atomic model structure by a loss function to obtain a corrected atomic model structure, wherein the loss function comprises a bond length constraint loss function, a collision constraint loss function and a spring constraint loss function, in the process of correcting the atomic model structure, the bond length constraint loss function indicates shortening the distance between adjacent amino acids in the protein that do not satisfy the bond length constraint, and the spring constraint loss function indicates that when the distance between adjacent amino acids in the protein is stretched or shortened, an opposite force is generated; a projection module configured to transform the corrected atomic model structure into a density map represented by Gaussian spheres by a projector module, and project the density map to obtain a projection image.

9. A terminal comprising: at least one memory and at least one processor; The at least one memory is configured to store program code, and the at least one processor is configured to invoke the program code stored in the at least one memory to execute the cryo-EM image processing method in any one of claims 1 to 7. 10.A storage medium configured to store program code, the program code being configured to execute the cryo-EM image processing method in any one of claims 1 to 7.

Images