A fine-grained galaxy denoising image classification method based on deep learning

Abstract

The present invention belongs to the field of image processing technology, and discloses a fine-grained galaxy denoising image classification method based on deep learning. A transformer with strong modeling ability is used as a backbone network to construct a galaxy classification model. In order to reduce the amount of calculation, a dynamic fine-grained partitioning module is used to focus on the foreground area, reduce the redundant calculation caused by the invalid background area, and consider the noise added to the original image at the same time. The galaxy image is denoised to improve the classification accuracy, and the global information q and the local information k and v are calculated by the transformer to complete the interaction between the information, thereby realizing the fine-grained partition galaxy denoising image classification based on the transformer. The present invention effectively improves the classification accuracy, broadens the scope of galaxy image classification in actual astronomical application scenarios, and makes the constructed galaxy classification model more robust.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and specifically relates to a fine-grained galaxy denoising image classification method based on deep learning. Background Art

[0002] Galaxies are gravitationally bound systems composed of stars, stellar remains, dark matter, and interstellar gas. In traditional studies of galactic astronomy, galaxies are usually classified according to their visual morphology. The morphology of a galaxy reflects the comprehensive physical characteristics of the galaxy, such as its structure, and is therefore closely related to the formation and evolution of the galaxy. For example, studies have shown that elliptical galaxies are more likely to be formed by the merger of galaxies. Therefore, classifying galaxies according to their morphological characteristics is a routine and important part of galactic astronomy research. As the survey depth and detection efficiency of large-scale sky survey projects continue to increase, the number of galaxies obtained in survey detection is also increasing, even reaching nearly 100 million, which poses new challenges to conventional data processing methods.

[0003] For the task of galaxy classification, automated classification methods for galaxy morphology have developed rapidly. The paper (Hocking A, Geach J E, Sun Y, et al. An automatic taxonomy of galaxy morphology using unsupervised machine learning [J]. Monthly Notices of the Royal Astronomical Society, 2018, 473 (1): 1108-1129.) proposed an unsupervised machine learning technique. Unlike traditional unsupervised machine learning methods, this method can separate early and late types of galaxies without any form of pre-training. The machine-based classification was compared with the manual classification results of Galaxy Zoo, proving the good consistency between manual and machine classification. The paper (Gauci A, Adami K Z, Abela J. Machine learning for galaxy morphology classification [J]. arXiv preprint arXiv: 1005.0390, 2010.) proposed different classification models based on machine learning for galaxy morphology classification, and conducted a comparative analysis including Classification And Regression Tree (CART), C4.5 (which can generate decision trees for classification), random forest and fuzzy logic algorithms.

[0004] The astronomical image denoising and classification algorithms based on machine learning proposed above have achieved the denoising and classification tasks to a certain extent, but the time taken will be longer, and even beyond the reconstruction capabilities of some algorithms. With many upcoming large-scale astronomical surveys, it is crucial to find more time-saving and efficient methods to analyze data.

[0005] Traditional galaxy morphology classification methods, on the one hand, adopt a classification method based on the original image, that is, only the most original galaxy image is used as data in the feature extraction step, and the galaxy features are learned from the original image through a neural network. The galaxy classification model is trained according to the loss function, and the most commonly used classification accuracy index is selected to judge the quality of the model. However, the classification method based on the original image ignores the noise information of the original galaxy image, and the real noise will affect the classification accuracy. On the other hand, most traditional methods adopt an evaluation method based on CNN, that is, using a convolutional neural network as the backbone network to complete the feature extraction operation; and the evaluation method based on CNN has limitations in global modeling and long-distance modeling capabilities. For example, patent application CN111027608A discloses a method and device for galaxy classification and recognition based on deep learning, which can realize automatic, rapid and accurate prediction of galaxy morphology, so as to quickly process images containing a large number of galaxies; but it does not take into account the noise added to the actual image, and does not take into account the global and local features of the galaxy image, and reflects the limitations of traditional CNN. In addition, due to the particularity of astronomical galaxy images, such as the high proportion of black background areas, a lot of redundant calculations will occur during model calculation.

[0006] With the introduction of ViT, more and more tasks in the image field have shifted their attention to transformers. For example, CN 115601772 A applied transformers to the field of aesthetics and achieved excellent results. However, for fine-grained tasks such as astronomical galaxy classification, transformers will cause a lot of redundant calculations. Therefore, in the field of galaxy image classification, it is crucial to find methods with relatively low complexity. Summary of the invention

[0007] To solve the above technical problems, the present invention provides a fine-grained partitioning galaxy denoising image classification method based on deep learning, which uses the transformer with strong modeling ability as the backbone network to construct a galaxy classification model; utilizes a dynamic fine-grained partitioning module to reduce redundant calculations, improve the accuracy of astronomical image classification, and reduce the complexity of astronomical galaxy image classification; it is conducive to enhancing the significance of real galaxy classification tasks, promoting the application of astronomical images, and accelerating the development of deep learning in the field of astronomical images in the future.

[0008] The present invention provides a deep learning-based fine-grained galaxy denoising image classification method, comprising the following steps:

[0009] Step 1: According to the Galaxy-zoo dataset, randomly select 90% of the galaxy images as the training set and the remaining 10% as the test set, and generate the training set and validation set CSV files;

[0010] Step 2: Add Gaussian noise to each image and perform denoising based on the NAFNet model to obtain the denoised galaxy image;

[0011] Step 3: Reshape the training image and the verification image using the transform image exchange processing method, and reshape the image corresponding to the local information and the global information in the training image;

[0012] Step 4: Divide the image corresponding to the global information into H×H patches through non-overlapping windows, and then divide each H×H patch into smaller patches through a dynamic fine-grained partitioning method. Then, pool the final partitioned patches to obtain a sparse query vector q.

[0013] Step 5: Calculate the key vector k and value vector v according to the image corresponding to the local information;

[0014] Step 6: Cross-send the sparse query vector q, key vector k and value vector v from different images into the transformer for training to obtain the image features of each image;

[0015] Step 7: Send the image features to the classification layer of the transformer model to obtain the predicted category distribution;

[0016] Step 8: Use cross entropy as the loss function to calculate the transformer model loss to complete the training;

[0017] Step 9: Use the predicted category distribution obtained in step 7 to predict the category through the torch.max function model to complete the galaxy denoised image classification.

[0018] Furthermore, in step 1, the divided galaxy images are generated into a CSV file, which includes index, image path and image label.

[0019] Furthermore, in step 2, Gaussian noise is added to simulate real noise; in the NAFNet model, its input is the noise observation value y=x+a; where x represents the real image and a represents the noise value, and the L1 loss and MSE loss functions are used for training in turn to learn the potential clean image; specifically as follows:

[0020]

[0021] where x′ i represents the i-th real image, x i represents the network output of the i-th image, and N represents the number of images.

[0022] Furthermore, in step 3, in the training set, each image corresponds to two transform image exchange processing methods; an image containing local information is formed by center cropping and random horizontal flipping; an image containing global information is formed by random reshaping cropping and random horizontal flipping; in the validation set, each image is reshaped using center cropping.

[0023] Furthermore, in step 4, the dynamic fine-grained division method will automatically divide the image into patches, and then further divide it into small patches based on the features. Specifically, the image is first divided into H×H patches using a non-overlapping window. For each patch, due to the difference in the amount of information between the background area and the foreground area, an average pooling layer and a linear mapping are used to predict the number of fine-grained divisions to be selected. The number of sparse query vectors q after the transformation will be reduced to a certain extent according to the fine-grained division. The larger the fine-grainedness, the smaller the amount of calculation.

[0024] Furthermore, in step 8, the cross entropy function is as follows:

[0025]

[0026] Where p(x i ) and q(x i ) represents the predicted category distribution and the actual category distribution.

[0027] The beneficial effects described in the present invention are as follows: the method described in the present invention does not consider the traditional CNN method, does not consider the image prior brought by CNN, but uses the transformer with strong modeling ability as the backbone network to construct a galaxy classification model, which makes up for the limitations of the traditional CNN and brings a brand new path for the transformer in the field of galaxy classification; the present invention uses a dynamic fine-grained partitioning method to focus on important foreground areas, reduce redundant calculations brought by invalid background areas, and greatly reduce the complexity of astronomical image classification; at the same time, considering the noise added to the original image, using the denoising mechanism in the astronomical image, before using the original image for training, first add Gaussian noise, and then obtain a cleaner galaxy image through denoising, integrate astronomical denoising into classification, improve classification accuracy, and broaden the scope of galaxy image classification in actual astronomical application scenarios; through the mid-term fusion of global information and local information, the constructed galaxy classification model is more robust, which makes up for the shortcomings of insufficient pertinence of traditional astronomical image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a schematic diagram of the structure of the transformer-based fine-grained galaxy denoising image classification model;

[0029] Figure 2 This is a flowchart of the transformer-based fine-grained galaxy denoising image classification method. DETAILED DESCRIPTION

[0030] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments in conjunction with the accompanying drawings.

[0031] like Figure 2 As shown, the method for classifying galaxy denoising images with fine-grained division based on deep learning in the present invention comprises the following steps:

[0032] Step 1: According to the (Galaxy-zoo) dataset, 90% of the images are randomly selected as the training set and the remaining 10% as the test set according to the standard dataset division method, and the training set and validation set CSV files are generated; the CSV files include indexes, image paths, and image labels;

[0033] Step 2: Add Gaussian noise to each image and perform denoising based on NAFNet to obtain the denoised galaxy image;

[0034] Step 3: Use transform to reshape the training image and the verification image, and reshape the image corresponding to the local information and the global information in the training image;

[0035] Step 4: Divide the image corresponding to the global information into H×H patches through non-overlapping windows, further divide it into small patches through a dynamic fine-grained partitioning method, and pool it to obtain a sparse query vector q;

[0036] Step 5: Calculate the key vector k and value vector v according to the image corresponding to the local information;

[0037] Step 6: Cross-send q, k, and v from different images to the transformer for training to obtain image features.

[0038] Step 7: Send the image features to the classification layer to obtain the predicted category distribution;

[0039] Step 8: Use cross entropy as the loss function to calculate the model loss to complete the training;

[0040] Step 9: Use the predicted category distribution obtained in step 7 to predict the category through the torch.max function model to complete the galaxy denoised image classification;

[0041] In step 2, Gaussian noise is added to simulate real noise. In the NAFNet model, the input of the model is the noise observation y=x+a; where x represents the real image and a represents the noise value; the L1 loss and MSE loss functions are used to learn the potential clean image; the L1 loss function is used for the first 40 rounds of training, and the MSE loss function is used for the next 60 rounds of training:

[0042]

[0043] where x′ i represents the i-th real image, x i represents the network output of the i-th image, and N represents the number of images.

[0044] In step 3, in the training set, each image corresponds to two transforms. An image containing local information is formed by center cropping and random horizontal flipping; an image containing global information is formed by random reshaping and random horizontal flipping; in the validation set, each image is reshaped by center cropping;

[0045] In step 4, the dynamic fine-grained partitioning module automatically divides the image into patches, and then further divides it into small patches based on the features. Specifically, the image is first divided into H×H patches using a non-overlapping window. For each patch, due to the difference in the amount of information between the background area and the foreground area, an average pooling layer and linear mapping are used to predict the number of fine-grained partitions to be selected. The number of query vectors q after transformation will be reduced to a certain extent according to the fine-grained partitioning. The larger the fine-grainedness, the smaller the amount of calculation.

[0046] In step 6, the cross entropy function is as follows. i ) and q(x i ) represents the predicted category distribution and the actual category distribution;

[0047]

[0048] The structure of the transformer-based galaxy denoising image classification model described in the present invention is as follows: Figure 1 As shown, the present invention conducted experiments on the Galaxy-zoo dataset, which is the largest public galaxy morphology classification database. The dataset contains 61,578 images, each with a resolution of 424×424, including five categories: Cigar galaxies, Circular disk galaxies, EdgeOn galaxies, Intermediate galaxies, and spiral galaxies. We verified the effectiveness on this dataset.

[0049] Taking a set of images in the training set as an example, the images will go through a denoising process before reshaping, adding Gaussian noise with an exponent of 25, and then denoising through NAFNet to obtain a clean image. The actual resolution of the image is 424×424, and after reshaping, a 224×224 size local information image and a global information image are obtained, and then a 224×224 size input image is obtained after random horizontal flipping.

[0050] In the dynamic fine-grained segmentation module, the image containing global information will be divided into H×H patches. For each patch, the patch is first average pooled and linearly mapped to obtain the predicted fine-grained value θ, then each patch will be divided into H / θ 2 This means that the astronomical background, which is mainly black, will get a larger granularity value, which means there will be fewer query vectors q and less calculation.

[0051] The calculated query vector q and the key vector k and value vector v containing the local information image are sent to the transformer for calculation, and finally the image features of batchsize×768 are obtained.

[0052] After obtaining the image features, they are input into the classification layer and the galaxy predicted distribution a is output. k =[a1,a2,...,a5], and compare it with the actual label distribution to calculate the classification accuracy and loss.

[0053] In view of the low accuracy, poor modeling ability, and complex transformer calculation of existing galaxy morphology classification methods, the present invention utilizes the transformer's long-distance feature extraction ability and strong modeling ability, dynamic fine-grained division method, the fusion of local features and global features, and the effectiveness of astronomical denoising to design a solution that can effectively construct an astronomical galaxy classification model based on fine-grained division of galaxy denoising images, thereby realizing a method suitable for astronomical galaxy morphology classification tasks. As can be seen from the above, the method described in the present invention utilizes the high quality of denoised images to effectively improve the accuracy of astronomical galaxy morphology classification and reduce computational complexity; it is conducive to promoting the application of astronomical images and accelerating the development of deep learning in the field of astronomical images in the future.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to be a further limitation of the present invention. All equivalent changes made using the contents of the present specification and drawings are within the protection scope of the present invention.

Claims

1. A fine-grained galaxy denoising image classification method based on deep learning, characterized in that: The method comprises: Step 1: According to the Galaxy-zoo dataset, randomly select 90% of the galaxy images as the training set and the remaining 10% as the test set, and generate the training set and validation set CSV files; Step 2: Add Gaussian noise to each image and perform denoising based on the NAFNet model to obtain the denoised galaxy image; Step 3: Reshape the training image and the verification image using the transform image exchange processing method, and reshape the image corresponding to the local information and the global information in the training image; Step 4: Divide the image corresponding to the global information into H×H patches through non-overlapping windows, and then divide each H×H patch into smaller patches through a dynamic fine-grained partitioning method. Then, pool the final partitioned patches to obtain a sparse query vector q. Step 5: Calculate the key vector k and value vector v according to the image corresponding to the local information; Step 6: Cross-send the sparse query vector q, key vector k and value vector v from different images into the transformer for training to obtain the image features of each image; Step 7: Send the image features to the classification layer of the transformer model to obtain the predicted category distribution; Step 8: Use cross entropy as the loss function to calculate the transformer model loss to complete the training; Step 9: Use the predicted category distribution obtained in step 7 to predict the category through the torch.max function model to complete the galaxy denoised image classification.

2. The method for classifying galaxy denoising images with fine-grained division based on deep learning according to claim 1, characterized in that: In step 1, the divided galaxy images are generated into a CSV file, which includes the index, image path and image label.

3. The method for classifying galaxy denoising images with fine-grained division based on deep learning according to claim 1, characterized in that: In step 2, Gaussian noise is added to simulate real noise. In the NAFNet model, the input is the noise observation value y=x+a, where x represents the real image and a represents the noise value. The L1 loss and MSE loss functions are used for training in sequence to learn the potential clean image. The L1 loss function is used for the first 40 rounds of training, and the MSE loss function is used for the next 60 rounds of training, as follows: Among them, x′ i represents the i-th real image, x i represents the network output of the i-th image, and N represents the number of images.

4. The method for classifying galaxy denoising images with fine-grained division based on deep learning according to claim 1, characterized in that: In step 3, in the training set, each image corresponds to two transform image exchange processing methods; an image containing local information is formed through center cropping and random horizontal flipping; an image containing global information is formed through random reshaping cropping and random horizontal flipping; in the validation set, each image is reshaped using center cropping.

5. The method for classifying galaxy denoising images with fine-grained division based on deep learning according to claim 1, characterized in that: In step 4, the dynamic fine-grained segmentation method automatically divides the image into patches, and then further divides it into small patches based on the features. Specifically, a non-overlapping window is first used to divide the image into H×H patches. For each patch, due to the difference in the amount of information between the background area and the foreground area, an average pooling layer and linear mapping are used to predict the number of fine-grained segmentations to be selected.

6. The method for classifying galaxy denoising images with fine-grained division based on deep learning according to claim 1, characterized in that: In step 8, the cross entropy function is as follows: Where p(x i ) and q(x i ) represents the predicted category distribution and the actual category distribution, x i Represents the network output for the ith image.

Images