Information processing program, information processing method, and information processing device

Abstract

When a computer trains an autoencoder that outputs the three-dimensional density structure of a molecule by inputting, into a decoder, a latent variable obtained by inputting a particle image of the molecule into an encoder, and three-dimensional information obtained from a value found by correcting azimuth information of the particle image of the molecule via a trainable parameter, this information processing device optimizes a parameter of the encoder, a parameter of the decoder, and the trainable parameter so that a loss function of the autoencoder is minimized.

Description

Information processing program, information processing method, and information processing device.

[0001] This invention relates to information processing programs, etc.

[0002] CryoEM (Cryogenic Electron Microscopy) is being used to improve the efficiency of drug discovery and other related processes. CryoEM is a device (or technique) for observing biomolecules such as proteins by irradiating them with an electron beam under liquid nitrogen cooling.

[0003] For example, a conventional technology related to cryoEM is called CryoTWIN (PaStEL). CryoTWIN is a technology that predicts the three-dimensional density structure by inputting a particle image and the orientation information of the image into a machine learning model. CryoTWIN trains a machine learning model using multiple pre-prepared particle images, and then uses the latent distribution obtained from the trained machine learning model to automatically calculate a reasonable continuous deformation of the three-dimensional density structure.

[0004] Furthermore, CryoTWIN uses technologies such as RELION and CryoSPARC to pre-determine the orientation information of the particle image.

[0005] International Publication No. 2021 / 176841, U.S. Patent Application Publication No. 2023 / 0260085, U.S. Patent Application Publication No. 2023 / 0132893

[0006] However, the conventional techniques described above have a problem in that if the accuracy of the particle image orientation information is low, the accuracy of the three-dimensional density structure also decreases.

[0007] In one aspect, the present invention aims to provide an information processing program, an information processing method, and an information processing apparatus that can improve the accuracy of particle image orientation information and the accuracy of three-dimensional density structure.

[0008] In the first approach, the computer performs the following process: The computer inputs latent variables obtained by inputting molecular particle images into the encoder, and three-dimensional information obtained from correcting the orientation information of the molecular particle images with trainable parameters, into the decoder. The computer then optimizes the encoder parameters, decoder parameters, and trainable parameters so that the loss function of the autoencoder is minimized when training an autoencoder that outputs the three-dimensional density structure of a molecule.

[0009] This improves the accuracy of particle image orientation information and thus the accuracy of the three-dimensional density structure.

[0010] Figure 1 is a diagram illustrating CryoTWIN (PaStEL). Figure 2 is a diagram illustrating isomegaly. Figure 3 is a diagram illustrating the process of calculating the continuous deformation of the most likely path of the latent variable z. Figure 4 is a diagram illustrating the Cryo-EM image. Figure 5 is a diagram illustrating the information processing device according to this embodiment 1. Figure 6 is a functional block diagram showing the configuration of the information processing device according to this embodiment 1. Figure 7 is a flowchart illustrating the processing procedure of the information processing device according to this embodiment 1. Figure 8 is a diagram illustrating the information processing device according to this embodiment 2. Figure 9 is a functional block diagram showing the configuration of the information processing device according to this embodiment 2. Figure 10 is a flowchart illustrating the processing procedure of the information processing device according to this embodiment 2. Figure 11 is a diagram illustrating the information processing device according to this embodiment 3. Figure 12 is a functional block diagram showing the configuration of the information processing device according to this embodiment 3. Figure 13 is a flowchart illustrating the processing procedure of the information processing device according to this embodiment 3. Figure 14 is a diagram illustrating an example of a computer hardware configuration that realizes the same functions as the information processing device of the embodiment.

[0011] The following describes in detail, with reference to the drawings, embodiments of the information processing program, information processing method, and information processing apparatus disclosed in this application. However, this invention is not limited to these embodiments.

[0012] First, before describing the information processing device according to this embodiment 1, we will provide some supplementary information about CryoTWIN (PaStEL). CryoTWIN is a combination of spatial-RaDOGAGA ​​(DeepTWIN), etc. PaStEL is an abbreviation for Generator of Pathways with Structural Change on Pseudo Free-Energy Landscape from Cryo-EM Images.

[0013] Figure 1 is a diagram illustrating CryoTWIN (PaStEL). As shown in Figure 1, CryoTWIN 10 includes an encoder 11 and a decoder 12. CryoTWIN 10 applies cryoEM to DeepTWIN. For the sake of explanation, the device that performs the processing related to CryoTWIN 10 will be referred to as "device".

[0014] First, we will explain the process by which the device predicts the three-dimensional density structure based on multiple particle images.

[0015] For example, by using cryoEM to image protein 6 from various angles, multiple particle images are generated. The instrument generates a Fourier image X by performing a Fourier Transform (FT) on the particle images 7 obtained from the Cryo-EM images. The instrument calculates the latent variable z by inputting the Fourier image X into the encoder 11. The latent variable z is P ψ The method shall follow the (GMM) model, which is an abbreviation for Gaussian Mixture Model. In this Example 1, proteins are used as an example, but macromolecules may also be used, such as nucleic acids, glycans, lipids, etc.

[0016] Next, the device inputs the latent variables z and v into the decoder 12, and then X' ｚ(v) is calculated. v represents the three-dimensional position and is defined by equation (1). In equation (1), R' represents the orientation information of the particle image 7. This orientation information R' is estimated in advance using technologies such as RELION and CryoSPARC. The v on the right side of equation (1) represents the position (two-dimensional position) of the Fourier image X.

[0017]

[0018] X' ｚ (v) represents the value of the three-dimensional position v in the three-dimensional Fourier volume. The instrument generates a three-dimensional Fourier volume 8 by repeatedly performing the above process on multiple particle images obtained from the same protein 6. The instrument predicts the three-dimensional density structure 9 by performing IFT (Inverse fast Fourier Transform) on the three-dimensional Fourier volume.

[0019] Here, CryoTWIN is trained to minimize the objective function shown in equation (2) while changing the parameters of the encoder 11 and decoder 12 using the training dataset. In equation (2), "X" is the Fourier image of the particle image. "X'" is the estimate of X by CryoTWIN. "W" is the metric correction matrix. "β" is a positive hyperparameter. "z" is the CryoTWIN latent variable. logQ z This is the rate. The symbol to the right of W indicates the Hadamard product.

[0020]

[0021] For example, a training dataset contains multiple training data. For instance, the explanatory variables (input data) in the training data are Fourier images of particle images. The target variable (ground truth data) in the training data is the estimated value.

[0022] The device repeatedly performs the above process using multiple training data points included in the training dataset, and in the process, obtains the distribution of the latent variable z output from the encoder 11. In the following description, the distribution of the latent variable z will be referred to as the "latent distribution".

[0023] Note that the latent distribution obtained in the process of training the encoder 11 and the decoder 12 using the training dataset has "isometry".

[0024] FIG. 2 is a diagram for explaining isometry. In FIG. 2, graphs G1, G2, and G3 are shown. Graph G1 is a graph of the latent distribution obtained from the structure of the original protein. Graph G2 is a graph of the latent distribution obtained as a result of applying spatial-VAE (Variational AutoEncoder) to a plurality of proteins. Graph G3 is a graph of the latent distribution obtained from the encoder 11 described in FIG. 1 for a plurality of proteins.

[0025] The horizontal axis of graphs G1, G2, and G3 corresponds to the first principal component (PC1) in principal component analysis. The vertical axis of graphs G1, G2, and G3 corresponds to the second principal component (PC2) in principal component analysis. One plot on graphs G1, G2, and G3 corresponds to the structure of one protein.

[0026] In graphs G1 and G3, the plots of proteins with similar structures are dense and there is isometry. On the other hand, in graph G2, the plots of proteins with dissimilar structures are arranged close to each other and there is no isometry. The reason for the lack of isometry is that the structure of the original protein is distorted by N(z;0,I d ).

[0027] Here, in CryoTWIN (PaStEL), based on the latent distribution obtained using the training dataset, a continuous deformation of the plausible path of the latent variable z is calculated. FIG. 3 is a diagram for explaining the process of calculating the continuous deformation of the plausible path of the latent variable z. In the latent space of FIG. 3, the latent distribution obtained during training is arranged. A probability is set for each latent variable z included in the latent distribution. On the latent space, a darker color indicates a higher probability of the latent variable z.

[0028] The device is based on the first criterion and the second criterion shown below, μ ＊ ｉ from μ ＊ ｊGenerate a plausible path z0 up to. For example, the path z0 is shown as in Equation (3).

[0029]

[0030] The first criterion is the criterion of making the total value of the probability of the latent variable z on the path z0 as large as possible. For example, the total value of the probability on the path z0 is shown as in Equation (4).

[0031]

[0032] The second criterion is the criterion of making the path length as short as possible. For example, the path length is shown as in Equation (5).

[0033]

[0034] The device inputs the path z0 into the trained decoder 12, thereby obtaining a continuous deformation of the three-dimensional density structure as shown in Equation (6).

[0035]

[0036] For example, by inputting the latent variable z obtained during training into the trained decoder 12, the three-dimensional density structure V’ z can be started up, so the latent variable z and the three-dimensional density structure V’ z can be identified as the same.

[0037] Furthermore, the latent distribution is a Gaussian distribution P ψ ’(z) as shown in Equation (7), and has isotropy as described in FIG. 2. Therefore, the latent distribution can be interpreted as the existence distribution of the three-dimensional density structure V’ z and can be defined as in Equation (8).

[0038]

[0039]

[0040] Above, CryoTWIN (PaStEL) has been supplemented and explained.

[0041] Next, I will provide a supplementary explanation regarding the challenges of CryoTWIN (PaStEL). In the above-mentioned CryoTWIN 10, the encoder 11 and decoder 12 are trained using orientation information estimated in advance by RELION and CryoSPARC. If the accuracy of the pre-estimated orientation information is insufficient, the accuracy of the 3D density structure output from the trained CryoTWIN 10 will be insufficient, and it may not be possible to obtain a 3D density structure with sufficient resolution. Furthermore, if the resolution of the 3D structure is insufficient, the accuracy of the continuous deformation of the 3D density structure will also decrease.

[0042] Next, we will provide supplementary explanation regarding the orientation information described in Figure 1. Biomolecules, including proteins, have a three-dimensional structure, and cryo-EM imaging captures a projected image of it. The camera is pointed at the three-dimensional structure from a certain direction, and a projected image (particle image) is obtained. The direction in which the camera is pointed consists of three parameters in Euler angles: ψ, θ, and φ. The orientation is (ψ, θ, φ).

[0043] The parameter ψ represents the angle of rotation around the z-axis in the xy-plane. The parameter θ represents the angle of rotation around the y-axis. The parameter φ represents the angle of rotation around the x-axis.

[0044] Figure 4 is a diagram illustrating the Cryo-EM image. The Cryo-EM image Im1 in Figure 4 is an image projected from a certain orientation (ψ, θ, φ). Since the Cryo-EM image Im1 contains multiple biomolecules, CryoTWIN 10 extracts one biomolecule from the Cryo-EM image Im1 to obtain a single image Im2. The single image Im2 corresponds to a particle image. In the single image Im2, particle 15 is located in the upper left of the single image Im2, and CryoTWIN 10 performs a process that moves the image around this particle 15. This process of moving the image around particle 15 is called translation and is represented by the parameters (x, y).

[0045] In this embodiment 1, the azimuth parameters (ψ, θ, φ) and the translation parameters (x, y) are combined and treated as azimuth information.

[0046] Next, the information processing device according to this embodiment 1 will be described. In the following description, the information processing device according to this embodiment 1 will be referred to as "information processing device 100". Figure 5 is a diagram illustrating the information processing device according to this embodiment 1. The information processing device 100 adds an angle correction model 20 to the CryoTWIN 10 described in Figure 1.

[0047] When the angle correction model 20 receives pre-estimated azimuth information R' as input, it corrects the azimuth information R based on the parameters of the angle correction model 20. ＊ This is a model that outputs [the specified value]. The angle correction model 20 is an NN (Neural Network), etc.

[0048] The information processing device 100 generates a Fourier image X by performing a Fourier Tectonic Process (FT) on the particle image 7 obtained from the Cryo-EM image. The information processing device 100 calculates the latent variable z by inputting the Fourier image X into the encoder 11. In Figure 5, a protein is used as an example, but other biomolecules may also be used.

[0049] The information processing device 100 inputs the latent variables z and v to the decoder 12, and then generates X' ｚ Calculate (v). v represents the three-dimensional position and is defined by equation (9). R in equation (9) ＊ This is the output result of the angle correction model 20, and the corrected azimuth information R ＊ The rest of the explanation for equation (9) is the same as the explanation for equation (1).

[0050]

[0051] The information processing device 100 corrects the previously estimated direction information R' using the angle correction model 20, thereby correcting the direction information R ＊ Generate and use equation (9) to calculate the three-dimensional position v. Directional information R' and directional information R ＊ This includes the azimuth parameters (ψ, θ, φ) and the translation parameters (x, y).

[0052] X' ｚ(v) represents the value of the three-dimensional position v in the three-dimensional Fourier volume. The information processing device 100 generates a three-dimensional Fourier volume 8 by repeatedly performing the above process on multiple particle images obtained from the same protein 6. The information processing device 100 predicts the three-dimensional density structure 9 by performing IFT on the three-dimensional Fourier volume.

[0053] The information processing device 100 is trained to minimize the objective function shown in equation (2) using the training dataset. The explanation of equation (2) is the same as the explanation of equation (2) above. The information processing device 100 trains the encoder 11, decoder 12, and angle correction model 20 to minimize the objective function shown in equation (2) while changing the parameters of the encoder 11, decoder 12, and angle correction model 20.

[0054] Here, the information processing device 100 performs processing to reduce the training load when training the encoder 11, decoder 12, and angle correction model 20.

[0055] For example, the training load refers to the following first load and second load.

[0056] The first load indicates the difficulty level of the training required for the problem setting itself. It means how difficult the problem setting is in the first place, and how complex the problem being attempted is. For example, in a classification problem, with images as input, problem setting (a2) is more demanding in terms of the problem setting than problem setting (a1) in the following problem settings (a2).

[0057] Problem setting (a1): Determine whether it is a dog. Problem setting (a2): Determine whether it is a dog, and then determine the breed.

[0058] The second load indicates the difficulty of training based on input information, network configuration, number of parameters, etc. It represents the difficulty of training in solving a problem; that is, how appropriate the input information is, and how appropriately the network is configured to facilitate training. In a classification problem, under the following problem settings (b1) and (b2), training is expected to proceed more easily with problem setting (b1). Problem setting (b2) has a lower resolution, which may necessitate image interpolation or training to unify object positions, as they may differ, thus resulting in a higher load.

[0059] Problem setting (b1): The image has high resolution (sharp image), and an object is in the center of the image. The object represents either a dog or something other than a dog. Problem setting (b2): The image has low resolution (coarse image), and the location of the object varies from image to image. A variation in object location means, for example, that the object may be in the center of the image or at the edge. The object represents either a dog or something other than a dog.

[0060] As explained above, high training loads can lead to problems such as increased time and cost to converge to optimization, and failure to converge to the optimal solution.

[0061] For example, typically, the information processing device 100 trains the parameters of the encoder 11, the decoder 12, and the angle correction model 20 simultaneously. In contrast, the information processing device 100 reduces the training load by alternately training the parameters of the encoder 11 and decoder 12 with the parameters of the angle correction model 20 at predetermined epoch intervals.

[0062] The information processing device 100 fixes the parameters of the angle correction model 20 while training the parameters of the encoder 11 and the decoder 12. Meanwhile, the information processing device 100 fixes the parameters of the encoder 11 and the decoder 12 while training the parameters of the angle correction model 20.

[0063] As described above, the information processing device 100 adds an angle correction model 20 to correct the azimuth information and trains the parameters of the encoder 11, the decoder 12, and the angle correction model 20. Furthermore, when training the encoder 11, decoder 12, and angle correction model 20, the information processing device 100 performs processing to reduce the training load. This makes it possible to train the parameters of the angle correction model 20 for correcting the azimuth information with a reduced training load, thereby improving the accuracy of the azimuth information.

[0064] Next, an example of the configuration of the information processing device 100 according to this embodiment will be described. Figure 6 is a functional block diagram showing the configuration of the information processing device according to this embodiment 1. As shown in Figure 6, the information processing device 100 has a communication unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150.

[0065] The communication unit 110 performs data communication with external devices, etc., via a network. The communication unit 110 may also receive the learning dataset 142, etc., from the external device.

[0066] The input unit 120 inputs various types of information to the control unit 150. The user may operate the input unit 120 to issue instructions for executing training.

[0067] The display unit 130 displays the information output from the control unit 150.

[0068] The memory unit 140 includes an autoencoder 141 and a training dataset 142. The memory unit 140 is a memory or the like.

[0069] The autoencoder 141 corresponds to the CryoTWIN described in Figure 5. More specifically, the autoencoder 141 includes an encoder 11, a decoder 12, and an angle correction model 20.

[0070] The training dataset 142 is used to train the autoencoder 141. The training dataset 142 contains multiple training data. For example, the explanatory variables (input data) of the training data are the particle images of proteins. The target variable (ground truth data) of the training data is the three-dimensional density structure of proteins. The ground truth data may also be the three-dimensional Fourier volume corresponding to the three-dimensional density structure.

[0071] We will now move on to the explanation of the control unit 150. The control unit 150 has a training unit 151. The control unit 150 is a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc.

[0072] The training unit 151 trains the autoencoder 141 (encoder 11, decoder 12, angle correction model 20) using the training dataset 142. The process by which the training unit 151 trains the autoencoder 141 is the same as described in Figure 5.

[0073] Furthermore, in order to reduce the training load, the training unit 151 alternately trains the parameters of the encoder 11 and decoder 12, and trains the parameters of the angle correction model 20, at predetermined epochs (predetermined number of training data).

[0074] Next, an example of the processing procedure of the information processing device 100 according to this embodiment 1 will be described. Figure 7 is a flowchart showing the processing procedure of the information processing device according to this embodiment 1. For example, M shown in Figure 7 is a pre-set natural number. The information processing device 100 alternately trains the parameters of the encoder 11 and the decoder 12, and trains the parameters of the angle correction model 20 every M times.

[0075] As shown in Figure 7, the training unit 151 of the information processing device 100 fixes the parameters of the angle correction model 20 of the autoencoder 141 (step S101). The training unit 151 sets i to 0 (step S102).

[0076] The training unit 151 obtains training data from the training dataset 142 (step S103). The training unit 151 increments i (step S104). The training unit 151 trains the autoencoder 141 (step S105). If the condition i ≥ M is not met (step S106, No), the training unit 151 proceeds to step S103.

[0077] On the other hand, if the condition i ≥ M is met (step S106, Yes), the training unit 151 releases the fixation of the parameters of the angle correction model 20 (step S107). The training unit 151 then fixes the parameters of the encoder 11 and the decoder 12 (step S108).

[0078] The training unit 151 sets i to 0 (step S109). The training unit 151 obtains training data from the training dataset 142 (step S110). The training unit 151 increments i (step S111). The training unit 151 trains the autoencoder 141 (step S112).

[0079] If the training unit 151 does not satisfy the condition i ≥ M (step S113, No), it proceeds to step S110.

[0080] On the other hand, if the training unit 151 satisfies the condition i ≥ M (step S113, Yes), it releases the fixation of the encoder and decoder parameters (step S114). If the training unit 151 satisfies the termination condition (step S115, Yes), it terminates the process.

[0081] On the other hand, if the training unit 151 does not meet the termination conditions (step S115, No), it proceeds to step S101.

[0082] The termination condition for step S115 is a pre-set condition. For example, the termination condition may be that the parameters of the encoder 11, decoder 12, and angle correction model 20 have converged.

[0083] As described above, the information processing device 100 adds an angle correction model 20 to correct the azimuth information and trains the parameters of the encoder 11, the decoder 12, and the angle correction model 20. Furthermore, when the information processing device 100 trains the encoder 11, decoder 12, and angle correction model 20, it alternately performs training of the encoder 11 and decoder 12 and training of the angle correction model 20. This makes it possible to train the parameters of the angle correction model 20 for correcting the azimuth information with a reduced training load, thereby improving the accuracy of the azimuth information.

[0084] Next, the information processing device according to this second embodiment will be described. In the following description, the information processing device according to this second embodiment will be referred to as "information processing device 200". Figure 8 is a diagram illustrating the information processing device according to this second embodiment. The information processing device 200 adds an angle estimation model 21 to the CryoTWIN 10 described in Figure 1.

[0085] The angle estimation model 21 is a model that estimates orientation information R' based on the parameters of the angle estimation model 21 when a particle image 7 is input. The angle estimation model 21 is an NN, etc. The orientation information R' includes orientation parameters (ψ, θ, φ) and translation parameters (x, y).

[0086] The information processing device 200 generates a Fourier image X by performing a Fourier transform on the particle image 7 obtained from the Cryo-EM image. The information processing device 200 calculates the latent variable z by inputting the Fourier image X to the encoder 11. In Figure 8, a protein is used as an example, but other biomolecules may also be used.

[0087] The information processing device 200 inputs the latent variables z and v to the decoder 12, and then generates X' ｚ (v) is calculated. v represents the three-dimensional position and is defined by equation (1). In equation (1), R' is the azimuth information R' estimated by the angle estimation model 21. Further explanation of equation (1) is the same as explained above.

[0088] X' ｚ(v) represents the value of the three-dimensional position v in the three-dimensional Fourier volume. The information processing device 200 generates a three-dimensional Fourier volume 8 by repeatedly performing the above process on multiple particle images obtained from the same protein 6. The information processing device 200 predicts the three-dimensional density structure 9 by performing IFT on the three-dimensional Fourier volume.

[0089] The information processing device 200 is trained to minimize the objective function shown in equation (2) using the training dataset. The explanation of equation (2) is the same as the explanation of equation (2) above. The information processing device 200 trains the encoder 11, decoder 12, and angle estimation model 21 to minimize the objective function shown in equation (2) while changing the parameters of the encoder 11, decoder 12, and angle estimation model 21.

[0090] Here, the information processing device 200 performs processing to reduce the training load when training the encoder 11, decoder 12, and angle estimation model 21.

[0091] The information processing device 200 fixes the parameters of the angle estimation model 21 while training the parameters of the encoder 11 and the decoder 12.

[0092] As described above, the information processing device 200 adds an angle estimation model 21 for estimating directional information and trains the parameters of the encoder 11, the decoder 12, and the angle estimation model 21. Furthermore, when training the encoder 11, decoder 12, and angle correction model 200, the information processing device 200 performs processing to reduce the training load. This makes it possible to train the parameters of the angle estimation model 21 for estimating directional information with a reduced training load, thereby improving the accuracy of the directional information.

[0093] Next, an example of the configuration of the information processing device 200 according to this embodiment will be described. Figure 9 is a functional block diagram showing the configuration of the information processing device according to this second embodiment. As shown in Figure 9, the information processing device 200 includes a communication unit 210, an input unit 220, a display unit 230, a storage unit 240, and a control unit 250.

[0094] The description of the communication unit 210, input unit 220, and display unit 230 is the same as the description of the communication unit 110, input unit 120, and display unit 130 described in Figure 6.

[0095] The memory unit 240 includes an autoencoder 241 and a training dataset 242. The memory unit 240 is a memory or the like.

[0096] The autoencoder 241 corresponds to CryoTWIN as described in Figure 8. More specifically, the autoencoder 241 includes an encoder 11, a decoder 12, and an angle estimation model 21.

[0097] The training dataset 242 is used to train the autoencoder 241. The training dataset 242 contains multiple training data. For example, the explanatory variables (input data) of the training data are the particle images of proteins. The target variable (ground truth data) of the training data is the three-dimensional density structure of proteins. Note that the ground truth data may also be the three-dimensional Fourier volume corresponding to the three-dimensional density structure.

[0098] We will now move on to the explanation of the control unit 250. The control unit 250 has a training unit 251. The control unit 250 is a CPU, GPU, etc.

[0099] The training unit 251 trains the autoencoder 241 (encoder 11, decoder 12, angle estimation model 21) using the training dataset 242. The process by which the training unit 251 trains the autoencoder 241 is the same as described in Figure 8.

[0100] Furthermore, in order to reduce the training load, the training unit 251 alternately trains the parameters of the encoder 11 and decoder 12, and trains the parameters of the angle estimation model 21, at predetermined epochs (predetermined number of training data).

[0101] Next, an example of the processing procedure of the information processing device 100 according to this second embodiment will be described. Figure 10 is a flowchart showing the processing procedure of the information processing device according to this second embodiment. For example, M shown in Figure 10 is a pre-set natural number. The information processing device 200 alternately trains the parameters of the encoder 11 and the decoder 12, and trains the parameters of the angle estimation model 21 every M times.

[0102] As shown in Figure 10, the training unit 251 of the information processing device 200 fixes the parameters of the angle estimation model 21 of the autoencoder 241 (step S201). The training unit 251 sets i to 0 (step S202).

[0103] The training unit 251 obtains training data from the training dataset 242 (step S203). The training unit 251 increments i (step S204). The training unit 251 trains the autoencoder 241 (step S205). If the condition i ≥ M is not met (step S206, No), the training unit 251 proceeds to step S203.

[0104] On the other hand, if the condition i ≥ M is met (step S206, Yes), the training unit 251 releases the fixation of the parameters of the angle estimation model 21 (step S207). The training unit 251 then fixes the parameters of the encoder 11 and the decoder 12 (step S208).

[0105] The training unit 251 sets i to 0 (step S209). The training unit 251 obtains training data from the training dataset 242 (step S210). The training unit 251 increments i (step S211). The training unit 251 trains the autoencoder 241 (step S212).

[0106] If the training unit 251 does not satisfy the condition i ≥ M (step S213, No), it proceeds to step S210.

[0107] On the other hand, if the training unit 251 satisfies the condition i ≥ M (step S213, Yes), it releases the fixation of the encoder and decoder parameters (step S214). If the training unit 251 satisfies the termination condition (step S215, Yes), it terminates the process.

[0108] On the other hand, if the training unit 251 does not meet the termination conditions (step S215, No), it proceeds to step S201.

[0109] The termination condition for step S215 is a pre-set condition. For example, the termination condition may be that the parameters of the encoder 11, decoder 12, and angle estimation model 21 have converged.

[0110] As described above, the information processing device 200 adds an angle estimation model 21 to correct the azimuth information and trains the parameters of the encoder 11, the decoder 12, and the angle estimation model 21. Furthermore, when the information processing device 200 trains the encoder 11, decoder 12, and angle correction model 20, it alternately performs training of the encoder 11 and decoder 12 and training of the angle estimation model 21. This makes it possible to train the parameters of the angle estimation model 21 for correcting the azimuth information with a reduced training load, thereby improving the accuracy of the azimuth information.

[0111] Next, an information processing device according to this third embodiment will be described. In the following description, the information processing device according to this third embodiment will be referred to as "information processing device 300". Figure 11 is a diagram illustrating the information processing device according to this third embodiment.

[0112] The information processing device 300 is configured to allow modification of the directional information R' used by CryoTWIN 10, as described in Figure 1. For example, the information processing device 300 updates the parameters (ψ, θ, φ) and (x, y) included in the directional information R' to optimal parameters during the training process. The initial value of the directional information R' is set to a pre-estimated value.

[0113] The information processing device 300 generates a Fourier image X by performing a Fourier transform on the particle image 7 obtained from the Cryo-EM image. The information processing device 300 calculates the latent variable z by inputting the Fourier image X to the encoder 11. In Figure 11, a protein is used as an example, but other biomolecules may also be used.

[0114] The information processing device 300, by inputting the latent variables z and v to the decoder 12, generates X' ｚ Calculate (v). v represents the three-dimensional position and is defined by equation (1). In equation (1), R' is the updatable orientation information R'. Further explanation of equation (1) is the same as the explanation above.

[0115] X' ｚ (v) represents the value of the three-dimensional position v in the three-dimensional Fourier volume. The information processing device 300 generates a three-dimensional Fourier volume 8 by repeatedly performing the above process on multiple particle images obtained from the same protein 6. The information processing device 300 predicts the three-dimensional density structure 9 by performing IFT on the three-dimensional Fourier volume.

[0116] The information processing device 300 is trained to minimize the objective function shown in equation (2) using the training dataset. The explanation of equation (2) is the same as the explanation of equation (2) above. The information processing device 300 trains the encoder 11, decoder 12, and direction information R' to minimize the objective function shown in equation (2) while changing the parameters of the encoder 11, decoder 12, and direction information R'.

[0117] Here, when the information processing device 300 trains the encoder 11, decoder 12, and direction information R', it performs processing to reduce the training load.

[0118] The information processing device 300 fixes the parameters of the azimuth information R' while training the parameters of the encoder 11 and the decoder 12. Meanwhile, the information processing device 200 fixes the parameters of the encoder 11 and the decoder 12 while training the parameters of the azimuth information R'.

[0119] Furthermore, the information processing device 300 may perform a process to unify one of the directional parameters (ψ, θ, φ) included in the directional information R' to a certain value, making the other parameters updatable. Unifying one parameter to a certain value means reducing the degree of freedom by one, which can reduce the training load.

[0120] As described above, the information processing device 300 sets the modifiable azimuth information R' and trains the parameters of the encoder 11, the decoder 12, and the azimuth information R'. Furthermore, when training the encoder 11, decoder 12, and azimuth information R', the information processing device 200 performs processing to reduce the training load. This makes it possible to improve the accuracy of the azimuth information while lowering the training load.

[0121] Next, an example of the configuration of the information processing device 300 according to this embodiment will be described. Figure 12 is a functional block diagram showing the configuration of the information processing device according to this embodiment 3. As shown in Figure 12, the information processing device 300 has a communication unit 310, an input unit 320, a display unit 330, a storage unit 340, and a control unit 350.

[0122] The description of the communication unit 310, input unit 320, and display unit 330 is the same as the description of the communication unit 110, input unit 120, and display unit 130 described in Figure 6.

[0123] The memory unit 340 includes an autoencoder 341 and a training dataset 342. The memory unit 340 is a memory or the like.

[0124] The autoencoder 341 corresponds to the CryoTWIN described in Figure 11. More specifically, the autoencoder 341 has an encoder 11 and a decoder 12. In addition, the autoencoder 341 treats the orientation information R' as a changeable parameter.

[0125] The training dataset 342 is used to train the autoencoder 341. The training dataset 342 contains multiple training data. For example, the explanatory variables (input data) of the training data are the particle images of proteins. The target variable (ground truth data) of the training data is the three-dimensional density structure of proteins. The ground truth data may also be the three-dimensional Fourier volume corresponding to the three-dimensional density structure.

[0126] We will now move on to the explanation of the control unit 350. The control unit 350 has a training unit 351. The control unit 350 is a CPU, GPU, etc.

[0127] The training unit 351 trains the autoencoder 341 (encoder 11, decoder 12, direction information R') using the training dataset 342. The process by which the training unit 351 trains the autoencoder 341 is the same as described in Figure 11.

[0128] Furthermore, in order to reduce the training load, the training unit 351 alternately trains the parameters of the encoder 11 and the decoder 12, and trains the direction information R' at predetermined epochs (predetermined number of training data).

[0129] Furthermore, the training unit 351 may perform a process to unify one of the directional parameters (ψ, θ, φ) included in the directional information R' to a certain value, and update the other parameters.

[0130] Next, an example of the processing procedure of the information processing device 300 according to this embodiment 3 will be described. Figure 13 is a flowchart showing the processing procedure of the information processing device according to this embodiment 3. For example, M shown in Figure 13 is a pre-set natural number. The information processing device 300 alternately trains the parameters of the encoder 11 and the decoder 12, and trains the parameters of the direction information R' every M times.

[0131] As shown in Figure 13, the training unit 351 of the information processing device 300 fixes the parameters of the direction information R' of the autoencoder 341 (step S301). The training unit 351 sets i to 0 (step S302).

[0132] The training unit 351 obtains training data from the training dataset 342 (step S303). The training unit 351 increments i (step S304). The training unit 351 trains the autoencoder 341 (step S305). If the condition i ≥ M is not met (step S306, No), the training unit 351 proceeds to step S303.

[0133] On the other hand, if the condition i ≥ M is met (step S306, Yes), the training unit 351 releases the fixation of the parameters of the direction information R' (step S307). The training unit 351 then fixes the parameters of the encoder 11 and the decoder 12 (step S308).

[0134] The training unit 351 sets i to 0 (step S309). The training unit 351 obtains training data from the training dataset 342 (step S310). The training unit 351 increments i (step S311). The training unit 351 trains the autoencoder 341 (step S312).

[0135] If the training unit 351 does not satisfy the condition i ≥ M (step S313, No), it proceeds to step S310.

[0136] On the other hand, if the training unit 351 satisfies the condition i ≥ M (step S313, Yes), it releases the fixation of the encoder and decoder parameters (step S314). If the training unit 351 satisfies the termination condition (step S315, Yes), it terminates the process.

[0137] On the other hand, if the training unit 351 does not meet the termination conditions (step S315, No), it proceeds to step S301.

[0138] The termination condition for step S315 is a pre-set condition. For example, the termination condition may be that the parameters of the encoder 11, decoder 12, and direction information R' have converged.

[0139] As described above, the information processing device 300 allows the directional information R' to be changed and trains the parameters of the encoder 11, the decoder 12, and the directional information R'. Furthermore, when the information processing device 300 trains the encoder 11, the decoder 12, and the directional information R', it alternately performs training of the encoder 11 and the decoder 12 and training of the directional information R'. This improves the accuracy of the directional information.

[0140] The information processing devices 100 of Examples 1 to 3 have been described above. The parameters of the angle correction model 20, the angle estimation model 21, and the orientation information R' described above correspond to "trainable parameters".

[0141] The processes described in Examples 1 to 3 above are examples, and the information processing device 100 (200, 300) may perform other processes. The following describes other processes (1) to (3) of the information processing device 100.

[0142] Other processing (1) will be explained. When the information processing device 100 trains the autoencoder 141, it uses a training dataset 142 in which the explanatory variable is "protein particle image" and the target variable is "protein three-dimensional density structure", but it is not limited to this.

[0143] For example, the information processing device 100 may further train the autoencoder 141 using an additional training dataset in which the explanatory variables are "first latent variable and first orientation information R'" and the target variable is "second latent variable and second orientation information R'". For example, with the parameters of the decoder 12 fixed, the information processing device 100 inputs "first latent variable and first orientation information R'" to the decoder 12 and obtains an output image from the decoder 12. The information processing device 100 inputs a particle image based on this output image to the encoder 11 to obtain a third latent variable. The information processing device 100 also obtains pre-prepared orientation information R' as the third latent variable.

[0144] The information processing device 100 replaces "X'" in the objective function shown in equation (2) with the third latent variable and the third directional information R', and replaces "X" in the objective function shown in equation (2) with the second latent variable and the second directional information R', and trains the parameters of the encoder 11 and the angle correction model 20 so as to minimize the objective function. For example, the information processing device 100 trains the parameters of the encoder 11 and the angle correction model 20 so as to minimize the difference between the second latent variable and the third latent variable, and the difference between the second directional information R' and the third directional information R'.

[0145] The information processing device 100 alternately performs training of the autoencoder 141 using the above-mentioned training dataset 142 and training of the autoencoder 141 using the additional training dataset.

[0146] Next, other processing (2) will be explained. After the training of the autoencoder 141 has progressed to a certain extent, the information processing device 100 may perform training of the autoencoder 141 using neighboring points in the latent space as described below. Note that the training of the autoencoder 141 has progressed to a certain extent means that training has been performed using a predetermined number or more of training data.

[0147] With respect to the particle images included in the training dataset 142, the first particle image and the second particle image are defined as pairs of particle images in which the distance between each latent variable in the latent space obtained when input to the encoder 11 is less than a threshold, and in which the correlation between the two particle images is low.

[0148] The three-dimensional density structure obtained by inputting the first particle image into an encoder 11 that has been trained to a certain extent is almost the same as the three-dimensional density structure obtained by inputting the second particle image into an encoder 11 that has been trained to a certain extent. The information processing device 300 obtains the third particle image by projecting the three-dimensional density structure obtained from the first particle image using the orientation information R' of the second particle image. The information processing device 100 trains the parameters of the angle correction model 20 so as to minimize the difference between the second particle image and the third particle image.

[0149] Next, other processing (3) will be explained. If the target biomolecule is a biomolecule with symmetry, the information processing device 100 may narrow the search range of the orientation information R' and train the orientation information R'. For example, if the biomolecule is "S-pro", since there is a symmetry every 120°, the structure will match by rotating it 120° in the axial direction, and the search range of (ψ, θ, φ) of the orientation information R' is set to 0° or more and less than 120°, respectively. Narrowing the search range makes it possible to further reduce the training load.

[0150] The other processes (1) to (3) have been explained above.

[0151] Next, we will explain an example of how to use different optimization schedules. Here, we define "alternating optimization" as optimization that alternates between training the encoder 11 and decoder 12 and training the direction information R'. We define "direct optimization" as optimization that simultaneously trains the encoder 11 and decoder 12 and the direction information R'.

[0152] Alternating optimization takes approximately twice as long as direct optimization in simple calculations (because it performs backpropagation twice when gradients are passed), but it has the advantage of being more stable because the learning load for each step is lower than that of direct optimization.

[0153] While direct optimization is expected to have approximately half the computation time of alternating optimization, it has the characteristic of being more prone to unstable learning results or failure to converge to the desired solution compared to alternating optimization due to the larger amount of data to be learned.

[0154] When prioritizing learning time, it is preferable to perform only direct optimization, and when prioritizing stability, it is preferable to perform only alternating optimization. When balancing the advantages of both stability and time, learning is started with alternating optimization, and then switched to direct optimization once stability is achieved. For example, the user operates the input unit 120 to give instructions to the information processing device 100 to perform alternating optimization or direct optimization.

[0155] Next, I will provide some additional explanation regarding the optimization schedule and convergence. If the initial values ​​are too far from the optimal solution, there is a higher probability that the model will not converge to the desired value. In such cases, it may be advisable to reduce the number of parameters in the latent variables during training and then increase the number of parameters once the model stabilizes.

[0156] For example, the parameters of the encoder 11 and decoder 12, including the latent space, are defined as "first parameters," and the parameters of the orientation information R' are defined as "second parameters." If "the number of first parameters >> the number of second parameters," then the training priority for the second parameters is low. If the distance between the second parameters and the initial values ​​is greater than or equal to a threshold, the information processing device 100 may take measures such as reducing the number of first parameters. For example, the information processing device 100 may perform processing such as reducing the dimensionality of the parameters of the encoder 11 and decoder 12, and the latent variables.

[0157] Next, we will provide a supplementary explanation regarding the learning rates for the orientation information R' and the latent variables. Training the orientation information (plus translation) has a small dimension and low complexity. On the other hand, training the latent variables (parameters of encoder 11 and decoder 12) has a large dimension and high complexity. Therefore, the information processing device 100 sets a large learning rate for the orientation information and a small learning rate for the latent variables. This allows for efficient training of the autoencoder 141. The learning rate corresponds to the magnitude of the change from before the update to after the update.

[0158] Next, we will describe in order an example of a computer hardware configuration that realizes the same functions as the information processing device 100 shown in the above embodiment.

[0159] Figure 14 shows an example of a computer hardware configuration that realizes similar functions to the information processing device of the embodiment. As shown in Figure 14, the computer 400 has a CPU 401 that performs various calculations, an input device 402 that receives data input from the user, and a display 403. The computer 400 also has a communication device 404 and an interface device 405 that exchange data with external devices via a wired or wireless network. The computer 400 also has a RAM 406 for temporarily storing various information and a hard disk drive 407. Each of the devices 401 to 407 is connected to a bus 408.

[0160] The hard disk drive 407 contains a training program 407a. The CPU 401 reads the training program 407a and loads it into the RAM 206.

[0161] The training program 407a functions as a training process 406a. The processing of the training process 406a corresponds to the processing of the training units 151, 251, and 351.

[0162] Furthermore, the training program 407a does not necessarily have to be stored in the hard disk drive 407 from the beginning. For example, the training program 407a may be stored in a "portable physical medium" such as a flexible disk (FD), CD-ROM, DVD, magneto-optical disk, or IC card inserted into the computer 400. Then, the computer 400 may read and execute the training program 407a.

[0163] 100 Information Processing Device 110, 210, 310 Communication Unit 120, 220, 320 Input Unit 130, 230, 330 Display Unit 140, 240, 340 Storage Unit 141, 241, 341 Autoencoder 142, 242, 342 Training Dataset 150, 250, 350 Control Unit 151, 251, 351 Training Unit

Claims

1. An information processing program characterized in that, when training an autoencoder that outputs the three-dimensional density structure of a molecule by inputting latent variables obtained from inputting a particle image of a molecule into an encoder and three-dimensional information obtained from values ​​corrected by trainable parameters for the orientation information of the particle image of the molecule, the computer performs a process to optimize the encoder parameters, the decoder parameters, and the trainable parameters so as to minimize the loss function of the autoencoder.

2. The information processing program according to claim 1, characterized in that the optimization process alternately optimizes the encoder parameters, the decoder parameters, and the trainable parameters.

3. The information processing program according to claim 1, characterized in that the optimization process optimizes the encoder parameters, decoder parameters, and trainable parameters so as to minimize the loss function, using a plurality of training data sets in which the particle image of the molecule is the explanatory variable and the three-dimensional density structure of the molecule is the objective variable.

4. The information processing program according to claim 1, characterized in that the optimization process uses a first latent variable obtained by inputting a particle image of the molecule to the encoder and first orientation information of the particle image as explanatory variables, a second latent variable obtained by inputting a particle image of the molecule to the encoder and second orientation information of the particle image as objective variables, inputs the output image obtained by inputting the first latent variable and the first orientation information to the decoder to the encoder to obtain a third latent variable, and optimizes the encoder parameters and the trainable parameters so as to minimize the difference between the third latent variable and the pre-estimated third orientation information and the second latent variable and the second orientation information.

5. The information processing program according to claim 1, characterized in that it identifies a first particle image and a second particle image in which the distance of latent variables in the latent space is less than a threshold, and the first particle image and the second particle image have low correlation with each other, and further causes the computer to perform a process to optimize the trainable parameters so as to minimize the error between a third particle image obtained by projecting a three-dimensional density structure estimated based on the first particle image with the orientation information and the second particle image.

6. The information processing program according to claim 1, wherein the trainable parameters are parameters of an angle correction model that outputs corrected azimuth information when the azimuth information is input, and the optimization process optimizes the encoder parameters, the decoder parameters, and the angle correction model parameters so that the loss function of the autoencoder is minimized.

7. The information processing program according to claim 1, wherein the trainable parameters are the orientation information, and the optimization process optimizes the encoder parameters, the decoder parameters, and the orientation information so as to minimize the loss function of the autoencoder.

8. The information processing program according to claim 7, characterized in that the optimization process fixes a portion of the orientation information and optimizes the encoder parameters, the decoder parameters, and the orientation information so as to minimize the loss function of the autoencoder.

9. The information processing program according to claim 7, characterized in that the optimization process limits the search range of the direction information and optimizes the encoder parameters, the decoder parameters, and the direction information so as to minimize the loss function of the autoencoder.

10. The information processing program according to claim 1, wherein the trainable parameters are parameters of an angle estimation model that outputs orientation information when a particle image is input, and the optimization process optimizes the encoder parameters, the decoder parameters, and the angle estimation model parameters so that the loss function of the autoencoder is minimized.

11. An information processing method characterized in that, when training an autoencoder that outputs the three-dimensional density structure of a molecule by inputting latent variables obtained from inputting a particle image of a molecule into an encoder and three-dimensional information obtained from values ​​obtained by correcting the orientation information of the particle image of the molecule with trainable parameters, a computer performs a process to optimize the encoder parameters, the decoder parameters, and the trainable parameters so as to minimize the loss function of the autoencoder.

12. The information processing method according to claim 11, characterized in that the optimization process alternately optimizes the encoder parameters, the decoder parameters, and the trainable parameters.

13. The information processing method according to claim 11, characterized in that the optimization process optimizes the encoder parameters, the decoder parameters, and the trainable parameters so as to minimize the loss function, using a plurality of training data sets in which the particle image of the molecule is the explanatory variable and the three-dimensional density structure of the molecule is the objective variable.

14. The information processing method according to 11, characterized in that the optimization process uses a first latent variable obtained by inputting a particle image of the molecule to the encoder and first orientation information of the particle image as explanatory variables, a second latent variable obtained by inputting a particle image of the molecule to the encoder and second orientation information of the particle image as objective variables, inputs the output image obtained by inputting the first latent variable and the first orientation information to the decoder to the encoder to obtain a third latent variable, and optimizes the encoder parameters and trainable parameters so as to minimize the difference between the third latent variable and the pre-estimated third orientation information and the second latent variable and the second orientation information.

15. The information processing method according to claim 11, characterized in that a computer further performs a process to optimize the trainable parameters so as to minimize the error between a third particle image, which is obtained by projecting a three-dimensional density structure estimated based on the first particle image using the orientation information, and the second particle image, wherein the distance between the latent variables in the latent space is less than a threshold, and the first and second particle images have low correlation with each other.

16. The information processing method according to claim 11, wherein the trainable parameters are parameters of an angle correction model that outputs corrected azimuth information when the azimuth information is input, and the optimization process optimizes the encoder parameters, the decoder parameters, and the angle correction model parameters so that the loss function of the autoencoder is minimized.

17. The information processing method according to claim 11, wherein the trainable parameters are the orientation information, and the optimization process optimizes the encoder parameters, the decoder parameters, and the orientation information so as to minimize the loss function of the autoencoder.

18. The information processing method according to 17, characterized in that the optimization process fixes a portion of the orientation information and optimizes the encoder parameters, the decoder parameters, and the orientation information so as to minimize the loss function of the autoencoder.

19. The information processing method according to 17, characterized in that the optimization process limits the search range of the orientation information and optimizes the encoder parameters, the decoder parameters, and the orientation information so as to minimize the loss function of the autoencoder.

20. The information processing method according to claim 11, wherein the trainable parameters are parameters of an angle estimation model that outputs orientation information when a particle image is input, and the optimization process optimizes the encoder parameters, the decoder parameters, and the angle estimation model parameters so that the loss function of the autoencoder is minimized.

21. An information processing device having a control unit that optimizes the parameters of the encoder, the parameters of the decoder, and the trainable parameters so as to minimize the loss function of the autoencoder when training an autoencoder that outputs the three-dimensional density structure of a molecule by inputting latent variables obtained from inputting a particle image of a molecule into an encoder and three-dimensional information obtained from values ​​obtained by correcting the orientation information of the particle image of the molecule with trainable parameters into a decoder.

22. The information processing apparatus according to claim 21, characterized in that the optimization process alternately optimizes the encoder parameters, the decoder parameters, and the trainable parameters.

23. The information processing apparatus according to 21, characterized in that the optimization process optimizes the encoder parameters, the decoder parameters, and the trainable parameters so as to minimize the loss function, using a plurality of training data sets in which the particle image of the molecule is the explanatory variable and the three-dimensional density structure of the molecule is the objective variable.

24. The information processing apparatus according to 21, characterized in that the optimization process uses a first latent variable obtained by inputting a particle image of the molecule to the encoder and first orientation information of the particle image as explanatory variables, a second latent variable obtained by inputting a particle image of the molecule to the encoder and second orientation information of the particle image as objective variables, inputs the output image obtained by inputting the first latent variable and the first orientation information to the decoder to the encoder to obtain a third latent variable, and optimizes the encoder parameters and the trainable parameters so as to minimize the difference between the third latent variable and the pre-estimated third orientation information and the second latent variable and the second orientation information.

25. The information processing apparatus according to claim 21, wherein the control unit further performs a process to optimize the trainable parameters so as to minimize the error between the second particle image and the third particle image obtained by projecting the three-dimensional density structure estimated based on the first particle image with the orientation information.

26. The information processing apparatus according to claim 21, wherein the trainable parameters are parameters of an angle correction model that outputs corrected azimuth information when the azimuth information is input, and the optimization process optimizes the encoder parameters, the decoder parameters, and the angle correction model parameters so that the loss function of the autoencoder is minimized.

27. The information processing apparatus according to claim 21, wherein the trainable parameters are the orientation information, and the optimization process optimizes the encoder parameters, the decoder parameters, and the orientation information so as to minimize the loss function of the autoencoder.

28. The information processing apparatus according to 27, characterized in that the optimization process fixes a portion of the orientation information and optimizes the encoder parameters, the decoder parameters, and the orientation information so as to minimize the loss function of the autoencoder.

29. The information processing apparatus according to 27, characterized in that the optimization process limits the search range of the direction information and optimizes the encoder parameters, the decoder parameters, and the direction information so as to minimize the loss function of the autoencoder.

30. The information processing apparatus according to claim 21, wherein the trainable parameters are parameters of an angle estimation model that outputs orientation information when a particle image is input, and the optimization process optimizes the encoder parameters, the decoder parameters, and the angle estimation model parameters so that the loss function of the autoencoder is minimized.

Images