Light curve automatic classification method and system based on multi-modal machine learning

Abstract

The application discloses a kind of light variation curve automation classification method and system based on multi-modal machine learning.It reads the image data in fits file and carries out pre-processing;Image feature extraction model is inputted to the image data after pre-processing, and image feature vector is outputted;Read the light variation curve data to be classified in the same fits file and carry out pre-processing;The light variation curve data after pre-processing is matched with image data;Matched light variation curve data is inputted into light variation curve-image transfer learning model, and simulated image feature vector is outputted;The light variation curve data after pre-processing and simulated image feature vector are sent into multi-modal light variation curve classification model, and the binary classification result of planet and non-planet is outputted.The application can well utilize multi-modal advantage to improve the accuracy of light variation curve data classification.

Description

Technical Field

[0001] This invention relates to an automated classification method and system for light curves based on multimodal machine learning, belonging to the field of intelligent astronomical image processing technology. Background Technology

[0002] In recent years, with the rapid advancement of science and technology, human exploration of outer space has been increasing.

[0003] The observation of large-scale transits has revolutionized our understanding of exoplanets. Many ground-based and space-based telescopes have made outstanding contributions to this, such as OGLE, TrES, HATNET / HATS, WAS, KELT, CoRoT, and Kepler, providing us with an unprecedented number and speed of new discoveries.

[0004] Among the most noteworthy is the rising star of telescopes—NASA's Transiting Exoplanet Survey Satellite (TESS). This space telescope, dedicated to the search for exoplanets, is considered the successor to NASA's Kepler Space Telescope. In its three years of operation, TESS has completed scanning over 80% of the sky, acquiring a vast amount of high-precision light curves for stars. It is the world's largest photometric survey project in terms of the number of bright stars, the coverage area, and the accuracy.

[0005] This project provided a massive amount of sky survey astronomical data; however, mining this massive amount of data manually by astronomers is clearly impractical. Therefore, astronomers turned their attention to an automatic classification method.

[0006] In recent years, machine learning and deep learning methods have been explored and applied in the field of light curve classification. The Autovetter project used a random forest model to classify TCEs based on the statistical features of Kepler pipelines. Thompson et al. and Armstrong et al. used unsupervised learning to cluster Kepler light curves with similar shapes and defined a classification metric using the distance between new light curves and TCEs with known labels. The LPP metric defined by Thompson et al. was used by Robovetter to filter light curves with shapes that "don't look like transits". In addition to using machine learning on transiting planets in Kepler, others have used machine learning to identify different types of candidate signals in Kepler and other datasets. Millholland and Laughlin used supervised learning to identify candidate non-transiting planets in Kepler data, and Dittmann et al. used neural networks to identify the most likely true transits among many candidate events in Earth datasets. Chrisyopher J. Shallue and Andrew Vaanderburg proposed a deep neural network that automatically examines Kepler TCEs, and its model can accurately distinguish the subtle differences between transiting exoplanets and false positives.

[0007] There are many studies on the classification of light curves both at home and abroad, but most of them only use the characteristics of the light curves themselves for classification. A few studies have added multiple views of the light curves, and a very few have added time series. However, there are still problems such as low classification accuracy and inability to overcome overfitting. Summary of the Invention

[0008] Purpose of the invention: With the advent of the era of astronomical big data, there is a wealth of sky survey astronomical data available. However, some single photometric data suffer from problems such as limited features and incompleteness, resulting in low data mining efficiency. The purpose of this invention is to provide an automated classification method and system for light curves based on multimodal machine learning. By learning features and knowledge from image data with rich features, the method transfers these features to the task of mining light curve data, thus solving the problem of accurately classifying single light curves with limited features.

[0009] Technical solution: To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0010] On the one hand, an automated classification method for light curves based on multimodal machine learning includes:

[0011] Read image data from the fits file and preprocess the image data;

[0012] The preprocessed image data is input into a pre-trained image feature extraction model, and the output is the image feature vector.

[0013] Read the light curve data to be classified from the same fits file and preprocess the light curve data;

[0014] The preprocessed optical curve data is matched with the preprocessed image data;

[0015] The matched optical curve data is input into the pre-trained optical curve-image transfer learning model, and the output is a simulated image feature vector.

[0016] The preprocessed light curve data and the simulated image feature vector are fed into a pre-trained multimodal light curve classification model, which outputs a binary classification result of planets and non-planetary objects.

[0017] Furthermore, the image feature extraction model consists of an input layer, a reshape layer, a residual block containing 16 convolutional kernels, a max pooling layer, a residual block containing 32 convolutional kernels, a max pooling layer, a residual block containing 64 convolutional kernels, a max pooling layer, a flatten layer, two dense layers, and an output layer, all connected in sequence. Each of the three residual blocks consists of three convolutional layers with a kernel size of 1×1, and each convolutional layer uses the ReLU activation function.

[0018] Furthermore, the image feature extraction model is trained through the following steps:

[0019] Obtain the image dataset from the fits file;

[0020] The acquired image dataset is preprocessed;

[0021] The preprocessed image dataset was used for model training and validation in a 7:3 ratio. The cross-entropy loss function was used as the objective function, and the goal was to minimize the objective function value. The Adam optimization algorithm was used to optimize the parameters, and the image feature extraction model was trained and the image feature vector set was output.

[0022] Furthermore, the preprocessing of the light curve data includes:

[0023] The light curve data is normalized, mean filtered / linearly interpolated, and then phase-folded to obtain a standardized input form of the light curve.

[0024] Furthermore, the optical curve-image transfer learning model consists of an input layer, a reshape layer, a convolutional layer containing 16 5×5 convolutional kernels, a max pooling layer, a convolutional layer containing 32 5×5 convolutional kernels, a max pooling layer, a convolutional layer containing 64 5×5 convolutional kernels, a max pooling layer, a convolutional layer containing 128 5×5 convolutional kernels, a max pooling layer, a convolutional layer containing 256 5×5 convolutional kernels, a max pooling layer, a Flatten layer, a Dense layer, and an output layer, wherein each convolutional layer uses the ReLU activation function.

[0025] Furthermore, the optical curve-image transfer learning model is trained through the following steps:

[0026] Obtain the categorized optical curve dataset from the same fits file;

[0027] Preprocess the classified optical curve dataset;

[0028] The preprocessed optical curve dataset is matched with the preprocessed image dataset;

[0029] The matched light curve datasets were used for model training and validation at a ratio of 6:4. The comparison error function was used as the objective function. The image feature vectors output by the image feature extraction model and the simulated image feature vectors output by the light curve-image transfer learning model were compared. The goal was to minimize the objective function value. The parameters were optimized using the Adam optimization algorithm to train the light curve-image transfer learning model and generate a simulated image feature vector set.

[0030] Furthermore, the multimodal optical curve classification model consists of an optical curve feature extraction module, a feature fusion module, and a Sigmoid classifier. The optical curve feature extraction module is used to extract features from the preprocessed input optical curve data to obtain an optical curve feature vector. The feature fusion module is used to fuse the optical curve feature vector with the simulated image feature vector output by the optical curve-image transfer learning model. The Sigmoid classifier is used to classify the fused feature vector.

[0031] Furthermore, the optical curve feature extraction module consists of an input layer, two convolutional layers containing 16 5×5 kernels each, one max pooling layer, two convolutional layers containing 32 5×5 kernels each, one max pooling layer, two convolutional layers containing 64 5×5 kernels each, one max pooling layer, two convolutional layers containing 128 5×5 kernels each, one max pooling layer, two convolutional layers containing 256 5×5 kernels each, one max pooling layer, one Flatten layer, and one Dense layer, wherein each convolutional layer uses the ReLU activation function; the feature fusion module consists of four fully connected layers connected in sequence.

[0032] Furthermore, the multimodal optical curve classification model is trained through the following steps:

[0033] The preprocessed light curve dataset and the simulated image feature vector set output by the light curve-image transfer learning model are randomly allocated in a ratio of 8:1:1 for model training, validation, and testing, respectively. The cross-entropy loss function is used as the objective function, and the goal is to minimize the objective function value. The parameters are optimized using the Adam optimization algorithm to obtain the multimodal light curve classification model.

[0034] On the other hand, an automated classification system for light curves based on multimodal machine learning includes:

[0035] The first reading and preprocessing module is used to read image data from the fits file and preprocess the image data.

[0036] The image feature extraction module is used to input pre-processed image data into a pre-trained image feature extraction model and output image feature vectors.

[0037] The second reading and preprocessing module is used to read the light curve data to be classified in the same fits file and preprocess the light curve data.

[0038] The matching module is used to match the preprocessed optical curve data with the preprocessed image data;

[0039] The image feature simulation generation module is used to input the matched light curve data into the pre-trained light curve-image transfer learning model and output the simulated image feature vector.

[0040] The classification module is used to input the preprocessed light curve data and the simulated image feature vector into the pre-trained multimodal light curve classification model, and output a binary classification result of planets and non-planetary objects.

[0041] The present invention also provides a computer-readable storage medium for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the aforementioned automated classification method for light curves based on multimodal machine learning.

[0042] The present invention also provides a computing device, comprising:

[0043] One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs including a method for performing the aforementioned automated classification method for light curves based on multimodal machine learning.

[0044] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0045] (1) Light curve data suffers from problems such as limited and incomplete data features. Currently, the classification of single light curve data still suffers from low accuracy and the inability to overcome overfitting. However, image data has richer features. This invention designs a light curve-image transfer learning model so that the features extracted from image data can be used for light curve classification, thus making up for the problem of limited features in light curve data.

[0046] (2) For the classification of optical curves, most classifications are achieved by directly inputting the optical curve. This invention proposes a multimodal optical curve classification model that combines the features of time-domain signals and image data, which effectively improves the classification efficiency of optical curves. Attached Figure Description

[0047] Figure 1 This is a flowchart of the method of the present invention;

[0048] Figure 2 This is a structural diagram of the image feature extraction model in an embodiment of the present invention;

[0049] Figure 3 This is a structural diagram of the optical curve-image transfer learning model in an embodiment of the present invention.

[0050] Figure 4 This is a structural diagram of the multimodal optical curve classification model in an embodiment of the present invention. Detailed Implementation

[0051] The present invention will be further described below with reference to specific embodiments. These embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0052] As mentioned earlier, with the advent of the era of astronomical big data, there is a wealth of sky survey astronomical data available. However, some single photometric data suffer from problems such as limited features and incompleteness, resulting in low data mining efficiency and inaccurate classification of light curves.

[0053] Therefore, embodiments of the present invention provide an automated classification method for light curves based on multimodal machine learning, such as... Figure 1 As shown, it includes the following steps:

[0054] Step 1: Read the image data from the fits file and preprocess the image data;

[0055] Step 2: Input the preprocessed image data into the pre-trained image feature extraction model, and output the image feature vector;

[0056] Step 3: Read the light curve data to be classified from the same fits file and preprocess the light curve data;

[0057] Step 4: Match the preprocessed optical curve data with the preprocessed image data;

[0058] Step 5: Input the matched optical curve data into the pre-trained optical curve-image transfer learning model, and output the simulated image feature vector.

[0059] Step 6: Input the preprocessed light curve data and the simulated image feature vector into the pre-trained multimodal light curve classification model, and output the binary classification results of planets and non-planetary objects.

[0060] When using a model for classification and recognition, and when training the model, it is necessary to preprocess the image data and light curve data.

[0061] In this embodiment of the invention, TESS data is used. Since the optical curve can generate images of different sizes, the image data needs to be preprocessed.

[0062] Image data preprocessing includes cropping the image data to 11*11 pixels by the center point and then performing dimensionality reduction to obtain 11*11 image data.

[0063] In addition, since the lengths of the optical curve data are not uniform, some preprocessing of the optical curve data is required.

[0064] The preprocessing of the light curve data includes: normalizing the light curve data, mean filtering / linear interpolation, and then phase folding to obtain a standardized input form of the 201-dimensional light curve.

[0065] In step 4, the light curve data is matched with the image data, meaning there is a one-to-one correspondence between the light curve data and the image data. Matching the light curve data and the image data is necessary to minimize the error between the generated simulated image data and the real image.

[0066] In step 5, the matched light curve data is input into the pre-trained light curve-image transfer learning model, and the output is a simulated image feature vector generated based on the light curve data.

[0067] In order to obtain effective spectral data features, this embodiment of the invention builds an image feature extraction model, which ultimately yields a 256-dimensional feature vector of the image data.

[0068] like Figure 2 As shown, the image feature extraction model consists of an input layer, a reshape layer, a residual block containing 16 convolutional kernels, a max pooling layer, a residual block containing 32 convolutional kernels, a max pooling layer, a residual block containing 64 convolutional kernels, a max pooling layer, a flatten layer, two dense layers, and an output layer, connected in sequence.

[0069] More specifically, the structure of the image feature extraction model includes:

[0070] Part 1: Input layer, the input data is the preprocessed image data, the input dimension is 11*11;

[0071] Part Two: Reshape Layer. After the reshape layer, the output data has a dimension of 11*11*1.

[0072] The third part is a residual block containing 16 convolutional kernels, which consists of 3 convolutional layers with a kernel size of 1×1. Each convolutional layer is processed by the ReLU activation function to obtain data with a dimension of 11*11*16.

[0073] Part 4: Maxpooling layer, which produces data with a dimension of 5*5*16 after max pooling operation;

[0074] The fifth part is a residual block containing 32 convolutional kernels, which consists of 3 convolutional layers with a kernel size of 1×1. Each convolutional layer is processed by the ReLU activation function to obtain data with a dimension of 5*5*32.

[0075] Part 6: Max pooling layer, which produces data with a dimension of 2*2*64 after max pooling operation;

[0076] Part 7: is a residual block containing 64 convolutional kernels, consisting of 3 convolutional layers with a kernel size of 1×1. Each convolutional layer is processed by the ReLU activation function to obtain data with a dimension of 2*2*64.

[0077] Part 8: Max pooling layer, which produces data with a dimension of 1*1*64 after max pooling operation;

[0078] Part 9: Flatten layer, which produces data with a dimension of 64;

[0079] Part 10: Dense Layer, which produces data with a dimension of 128;

[0080] Part 11: Dense Layer. The data obtained after passing through the Dense layer has a dimension of 256.

[0081] The image feature extraction model can be trained through the following steps:

[0082] Step 21: Obtain the image dataset from the fits file;

[0083] Step 22: Preprocess the acquired image dataset;

[0084] Each image in the image dataset is cropped to 11*11 pixels by the center point and then dimensionality is reduced to obtain an 11*11 image dataset.

[0085] The 11*11 image data obtained by segmentation was identified using the OpenCV recognition program.

[0086] Step 23: Train and validate the model using the preprocessed image dataset in a 7:3 ratio. Use the cross-entropy loss function as the objective function and minimize the objective function value. Optimize the parameters using the Adam optimization algorithm to train the image feature extraction model and output the image feature vector set.

[0087] The training sample batch size was set to 128, the cross-entropy loss function was selected, the ReLU activation function was used in the residual blocks, and the parameters were optimized using the Adam optimization algorithm. The network was iteratively trained using deep learning to obtain a well-trained image feature extraction model with optimized weights.

[0088] In order to make the image data simulated by the input optical curve data simulate real image data as closely as possible, this embodiment builds an optical curve-image transfer learning model, which will eventually obtain a 256-dimensional feature vector set of simulated image data.

[0089] like Figure 3As shown, the optical curve-image transfer learning model consists of an input layer, a reshape layer, a convolutional layer with 16 kernels, a max pooling layer, a convolutional layer with 32 kernels, a max pooling layer, a convolutional layer with 64 kernels, a max pooling layer, a convolutional layer with 128 kernels, a max pooling layer, a convolutional layer with 256 kernels, a max pooling layer, a flatten layer, a dense layer, and an output layer.

[0090] More specifically, the structure of the optical curve-image transfer learning model includes:

[0091] Part 1: Input layer, whose input data is preprocessed light curve data, with an input dimension of 1*201;

[0092] Part Two: Reshape Layer. After the reshape layer, the output data has a dimension of 1*201*1.

[0093] The third part is a convolutional layer with 16 kernels, each kernel being 5×5 in size. After processing with the ReLU activation function, the resulting data has a dimension of 1*201*16.

[0094] Part 4: Max pooling layer, which performs max pooling to obtain data with dimensions 1*101*16;

[0095] Part 5: This is a convolutional layer with 32 kernels, each kernel size of 5×5. After processing with the ReLU activation function, the resulting data has a dimension of 1*101*32.

[0096] Part 6: Max pooling layer, which produces data with dimensions 1*51*32 after max pooling operation;

[0097] Part 7: This is a convolutional layer with 64 kernels, each kernel size of 5×5. After processing with the ReLU activation function, the resulting data has a dimension of 1*51*64.

[0098] Part 8: Max pooling layer, which produces data with a dimension of 1*26*64 after max pooling operation;

[0099] Part 9: This is a convolutional layer with 128 kernels, each kernel size of 5×5. After processing with the ReLU activation function, it yields data with dimensions of 1*26*128.

[0100] Part 10: Max Pooling Layer, which produces data with a dimension of 1*13*128 after max pooling operation;

[0101] Part 11: This is a convolutional layer with 256 kernels, each kernel being 5×5 in size. After processing with the ReLU activation function, the resulting data has a dimension of 1*13*256.

[0102] Part 12: Max pooling layer, which produces data with a dimension of 1*7*256 after max pooling operation;

[0103] Part Thirteen: Flatten Layer, which produces data with a dimension of 7*256;

[0104] Part Fourteen: Dense Layer. The data obtained after passing through the Dense layer has a dimension of 256.

[0105] The optical curve-image transfer learning model can be trained through the following steps:

[0106] Step 51: Obtain the classified light curve dataset from the same fits file in Step 21;

[0107] Step 52: Preprocess the obtained light curve dataset;

[0108] Based on the processing of light curves in Shallue & Verderburg (2018), the light curve data to be processed is normalized, mean filtered / linearly interpolated, and then phase folded to obtain a standardized input form of 201-dimensional light curves.

[0109] Step 53: Match the preprocessed optical curve dataset from step 52 with the preprocessed image dataset from step 22.

[0110] Step 54: The matched light curve dataset is used for model training and validation in a 6:4 ratio. The comparison error function is used as the objective function. The image feature vector output by the image feature extraction model is compared with the simulated image feature vector output by the light curve-image transfer learning model. The objective function value is minimized. The parameters are optimized by the Adam optimization algorithm to train the light curve-image transfer learning model and generate a simulated image feature vector set.

[0111] The training sample batch size was set to 128. The objective function was a comparison error function, comparing the image feature vector output by the image feature extraction model with the simulated image feature vector output by the optical curve-image transfer learning model. The goal was to minimize the objective function value, and the parameters were optimized using the Adam optimization algorithm. The trained optical curve-image transfer learning model was obtained through iterative deep learning training.

[0112] Then, the preprocessed light curve dataset from step 52 and the simulated image feature vector set generated in step 54 are input into the multimodal light curve classification model to complete the binary classification of planets and non-planetary objects.

[0113] like Figure 4 As shown, the multimodal optical curve classification model consists of an optical curve feature extraction module, a feature fusion module, and a Sigmoid classifier. The optical curve feature extraction module extracts features from the preprocessed input optical curve data to obtain optical curve feature vectors. The feature fusion module fuses the obtained optical curve feature vectors with the simulated image feature vectors output by the optical curve-image transfer learning model. The Sigmoid classifier classifies the fused feature vectors.

[0114] Specifically, the optical curve feature extraction module consists of an input layer, two convolutional layers with 16 kernels each, one max-pooling layer, two convolutional layers with 32 kernels each, one max-pooling layer, two convolutional layers with 64 kernels each, one max-pooling layer, two convolutional layers with 128 kernels each, one max-pooling layer, two convolutional layers with 256 kernels each, one max-pooling layer, one Flatten layer, and one Dense layer, all connected in sequence. The feature fusion module consists of four fully connected layers.

[0115] More specifically, such as Figure 4 As shown, the structure of the multimodal optical curve classification model includes:

[0116] Part 1: Input layer, the input data is the preprocessed light curve data, the input dimension is 1*201;

[0117] The second part is a convolutional layer with 16 kernels, each kernel being 5×5 in size. After processing with the ReLU activation function, the resulting data has a dimension of 1*201*16.

[0118] The third part is a convolutional layer with 16 kernels, each kernel being 5×5 in size. After processing with the ReLU activation function, the resulting data has a dimension of 1*201*16.

[0119] Part 4: Max pooling layer, which performs max pooling to obtain data with dimensions 1*101*16;

[0120] Part 5: This is a convolutional layer with 32 kernels, each kernel size of 5×5. After processing with the ReLU activation function, the resulting data has a dimension of 1*101*32.

[0121] Part 6: This is a convolutional layer with 32 kernels, each 5×5 in size. After processing with the ReLU activation function, the resulting data has a dimension of 1*101*32.

[0122] Part 7: Max pooling layer, which produces data with dimensions 1*51*32 after max pooling operation;

[0123] Part 8: This is a convolutional layer with 64 kernels, each kernel size of 5×5. After processing with the ReLU activation function, the resulting data has a dimension of 1*51*64.

[0124] Part 9: This is a convolutional layer with 64 kernels, each kernel being 5×5 in size. After processing with the ReLU activation function, the resulting data has dimensions of 1*51*64.

[0125] Part 10: Max Pooling Layer, which yields data with a dimension of 1*26*64 after max pooling operation;

[0126] Part 11: This is a convolutional layer with 128 kernels, each kernel being 5×5 in size. After processing with the ReLU activation function, the resulting data has a dimension of 1*26*128.

[0127] Part 12: This is a convolutional layer with 128 kernels, each kernel being 5×5 in size. After processing with the ReLU activation function, the resulting data has a dimension of 1*26*128.

[0128] Part Thirteen: Max Pooling Layer, which yields data with a dimension of 1*13*128 after max pooling operation;

[0129] Part Fourteen: This is a convolutional layer with 256 kernels, each kernel being 5×5 in size. After processing with the ReLU activation function, it yields data with dimensions 1*13*256.

[0130] Part 15: This is a convolutional layer with 256 kernels, each kernel being 5×5 in size. After processing with the ReLU activation function, it yields data with dimensions 1*13*256.

[0131] Part 16: Max pooling layer, which produces data with a dimension of 1*7*256 after max pooling operation;

[0132] Part 17: Flatten layer, which produces data with a dimension of 7*256;

[0133] Part 18: Dense Layer, which produces data with a dimension of 256;

[0134] Part 19: The input data is simulated image data generated by the optical curve-image transfer learning model, with an input dimension of 1*256;

[0135] Part 20: Fully Connected Layer (FC), the data obtained after passing through the FC layer has a dimension of 512;

[0136] Part 21: FC layer, the data obtained after passing through the FC layer has a dimension of 512;

[0137] Part 22: FC layer, which produces data with a dimension of 512;

[0138] Part 23: FC layer, which produces data with a dimension of 512;

[0139] Part 24: Sigmoid classifier, which outputs a binary classification result of planets and non-planetary objects using the Sigmoid function.

[0140] The multimodal optical curve classification model can be trained through the following steps:

[0141] The preprocessed light curve dataset from step 52 and the simulated image feature vector set output by the light curve-image transfer learning model from step 54 are randomly allocated in a ratio of 8:1:1 for training, validation, and testing, respectively. The cross-entropy loss function is used as the objective function, and the goal is to minimize the objective function value. The parameters are optimized using the Adam optimization algorithm to train and obtain a multimodal light curve classification model.

[0142] The training sample batch size was set to 10, the cross-entropy loss function was selected, and the ReLU activation function was used in the residual blocks. The Adam optimization algorithm was used to optimize the parameters: learning rate 1e-05, first-moment decay rate = 0.9, second-moment exponential decay rate = 0.999, and decay term 1e-08. The network was iteratively trained using deep learning to obtain a multimodal optical curve classification model with well-trained weights.

[0143] In another embodiment, an automated classification system for light curves based on multimodal machine learning includes:

[0144] The first reading and preprocessing module is used to read image data from the fits file and preprocess the image data.

[0145] The image feature extraction module is used to input pre-processed image data into a pre-trained image feature extraction model and output image feature vectors.

[0146] The second reading and preprocessing module is used to read the optical curve data from the same fits file and preprocess the optical curve data.

[0147] The matching module is used to match the preprocessed optical curve data with the preprocessed image data;

[0148] The image feature simulation generation module is used to input the matched light curve data into the pre-trained light curve-image transfer learning model and output the simulated image feature vector.

[0149] The classification module is used to input the preprocessed light curve data and the simulated image feature vector into the pre-trained multimodal light curve classification model, and output a binary classification result of planets and non-planetary objects.

[0150] The present invention also provides a computer-readable storage medium for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the aforementioned automated classification method for light curves based on multimodal machine learning.

[0151] The present invention also provides a computing device, comprising:

[0152] One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs including a method for performing the aforementioned automated classification method for light curves based on multimodal machine learning.

[0153] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] The present invention has been disclosed above with reference to preferred embodiments, but it is not intended to limit the present invention. All technical solutions obtained by adopting equivalent substitutions or equivalent transformations fall within the protection scope of the present invention.

Claims

1. A method for automated classification of light curve based on multi-modal machine learning, characterized in that, The method comprises the following steps: reading image data in a fits file and pre-processing the image data; inputting the pre-processed image data into an image feature extraction model obtained by pre-training to output an image feature vector; reading light curve data to be classified in the same fits file and pre-processing the light curve data; matching the pre-processed light curve data with the pre-processed image data; inputting the matched light curve data into a light curve-image transfer learning model obtained by pre-training to output a simulated image feature vector; inputting the pre-processed light curve data and the simulated image feature vector into a multi-modal light curve classification model obtained by pre-training to output a binary classification result of planets and non-planets; The multi-modal light curve classification model comprises a light curve feature extraction module, a feature fusion module and a Sigmoid classifier. The light curve feature extraction module is configured to extract features from the input pre-processed light curve data to obtain a light curve feature vector. The feature fusion module is configured to fuse the light curve feature vector with the simulated image feature vector output by the light curve-image transfer learning model. The Sigmoid classifier is configured to classify the fused feature vector.

2. The method of claim 1, wherein, The image feature extraction model comprises an input layer, a reshape layer, one residual block comprising 16 convolution kernels, one max-pooling layer, one residual block comprising 32 convolution kernels, one max-pooling layer, one residual block comprising 64 convolution kernels, one max-pooling layer, one Flatten layer, two Dense layers and an output layer connected in sequence. Each of the three residual blocks comprises three convolution layers with a kernel size of 1x1, wherein each convolution layer uses a ReLU activation function. 3.The method of claim 1, wherein, The image feature extraction model is obtained by the following steps: obtaining an image data set from a fits file; pre-processing the obtained image data set; training and verifying the pre-processed image data set in a ratio of 7:3, using a cross-entropy loss function as the objective function, aiming to minimize the objective function value, and performing parameter optimization through an Adam optimization algorithm to obtain the image feature extraction model and output an image feature vector set. 4.The method of claim 1, wherein, The pre-processing of the light curve data comprises: normalizing, mean filtering / linear interpolation and phase folding the light curve data to obtain a standardized input form of the light curve data.

5. The method of automated classification of light curves based on multi-modal machine learning according to claim 1, wherein, The light curve-image transfer learning model is composed of an input layer, a reshape layer, a convolutional layer containing 16 5*5 convolutional kernels, a maximum pooling layer, a convolutional layer containing 32 5*5 convolutional kernels, a maximum pooling layer, a convolutional layer containing 64 5*5 convolutional kernels, a maximum pooling layer, a convolutional layer containing 128 5*5 convolutional kernels, a maximum pooling layer, a convolutional layer containing 256 5*5 convolutional kernels, a maximum pooling layer, a Flatten layer, a Dense layer, and an output layer, wherein each convolutional layer uses a ReLU activation function.

6. The method of automated classification of light curves based on multi-modal machine learning according to claim 3, wherein, The light curve-image transfer learning model is trained by the following steps: Obtaining the classified light curve dataset from the same fits file; Preprocessing the classified light curve dataset; Matching the preprocessed light curve dataset with the preprocessed image dataset; Training and verifying the matched light curve dataset in a ratio of 6:4, taking the contrast image feature extraction model output image feature vector and the light curve-image transfer learning model output simulated image feature vector as the objective function, and taking the target function value tending to be the minimum as the goal, and optimizing the parameters through the Adam optimization algorithm to train the light curve-image transfer learning model and generate the simulated image feature vector set.

7. The method of automated classification of light curves based on multi-modal machine learning according to claim 1, wherein, The light curve feature extraction module is composed of an input layer, two convolutional layers containing 16 5*5 convolutional kernels, a maximum pooling layer, two convolutional layers containing 32 5*5 convolutional kernels, a maximum pooling layer, two convolutional layers containing 64 5*5 convolutional kernels, a maximum pooling layer, two convolutional layers containing 128 5*5 convolutional kernels, a maximum pooling layer, two convolutional layers containing 256 5*5 convolutional kernels, a maximum pooling layer, a Flatten layer, and a Dense layer, wherein each convolutional layer uses a ReLU activation function; and the feature fusion module is composed of four fully connected layers. 8.The method of claim 6, wherein, The multi-modal light curve classification model is trained by the following steps: Randomly allocating the preprocessed light curve dataset and the simulated image feature vector set output by the light curve-image transfer learning model in a ratio of 8:1:1 for model training, verification, and testing, respectively, taking the cross-entropy loss function as the objective function, taking the target function value tending to be the minimum as the goal, and optimizing the parameters through the Adam optimization algorithm to train the multi-modal light curve classification model.

9. A photometric light curve automated classification system based on multi-modal machine learning, characterized in that, It includes: A first reading and preprocessing module for reading image data in a fits file and preprocessing the image data; An image feature extraction module for inputting the preprocessed image data into a pre-trained image feature extraction model to output an image feature vector; A second reading and preprocessing module for reading light curve data to be classified in the same fits file and preprocessing the light curve data; The matching module is configured to match the preprocessed light curve data with the preprocessed image data; The image feature simulation generation module is configured to input the matched light curve data into a pre-trained light curve-image transfer learning model to output a simulated image feature vector; The classification module is configured to input the preprocessed light curve data and the simulated image feature vector into a pre-trained multi-modal light curve classification model to output a binary classification result of the planet and the non-planet. The multi-modal light curve classification model comprises a light curve feature extraction module, a feature fusion module and a Sigmoid classifier. The light curve feature extraction module is configured to extract features of the input preprocessed light curve data to obtain a light curve feature vector. The feature fusion module is configured to fuse the light curve feature vector with the simulated image feature vector output by the light curve-image transfer learning model. The Sigmoid classifier is configured to classify the fused feature vector.

Images