Sun multiband / sequence image registration method and system based on deep learning
Through the deep learning image registration method, the key point extraction model, descriptor extractor and matching algorithm are used to solve the field of view offset problem in solar multi-band image registration, and high-precision image alignment is achieved.
Patent Information
- Application Number
- CN202510502056.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-21
- Publication Date
- 2025-08-01
AI Technical Summary
Due to the differences in different telescope equipment and observation bands, there is a field of view offset in solar observation images, and existing image registration methods are not effective in solar multi-band and sequence image registration.
The image registration method based on deep learning is adopted, and the key point extraction model, descriptor extractor and bidirectional matching brute force algorithm are combined with the RANSAC algorithm to achieve high-quality image registration.
It improves the accuracy and stability of image registration, especially in solar active and quiet areas image registration, and is suitable for registration of images in different bands and same band sequences.
Smart Images
Figure CN120411191A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a solar multi - band / sequence image registration method based on deep learning, belonging to the fields of astronomical technology and digital image processing. Background Art
[0002] The sun is the central celestial body of the solar system and the nearest star to us. The sun generates solar activities such as sunspots, solar flares, coronal mass ejections, and prominence eruptions. These evolutions have a great impact on the earth. Therefore, real - time observation and research of the sun are crucial.
[0003] Multi - band, full - time - domain, and high - resolution observation of the sun has always been a key topic in solar physics and astronomy research. Solar observation mainly relies on the imaging of ground - based and space - based solar telescopes. However, due to different telescope devices and different pixel resolutions of the images obtained in different observation bands, the observed field - of - view differences cause field - of - view offsets in the observed images. Therefore, it is necessary to use image registration to align these images. Summary of the Invention
[0004] The present invention provides a solar multi - band / sequence image registration method based on deep learning, which can be used not only for solar multi - band image registration, but also for the registration of quiet - region images and active - region images in the same band sequence.
[0005] The technical solution of the present invention is as follows:
[0006] According to the first aspect of the present invention, there is provided a solar multi - band / sequence image registration method based on deep learning, including the following steps:
[0007] S1. Obtain the solar - band reference image m c and the image m to be registered p ; the solar - band reference image and the image to be registered are images of different bands or sequence images of the same band;
[0008] S2. Use the key - point extraction model to extract the key - point heat map and intermediate feature representation of the solar - band reference image m c and the image m to be registered p respectively; perform local maximum operation on the key - point heat maps of the solar - band reference image m c , the image m to be registered p respectively to find the local maximum values in the key - point heat maps; compare the local maximum values with a preset pixel threshold, and take the pixels greater than the pixel threshold as key points;
[0009] S3. Use the descriptor extractor to extract the solar - band reference image m cDescriptor of key points, and extracting the image m to be registered by using a descriptor extractor p Descriptor of key points;
[0010] S4. For the solar band reference image m c Descriptor of key points, the image m to be registered p Using the brute-force algorithm with bidirectional matching for the descriptors of key points to determine the most matching key point pairs; using the random sample consensus algorithm for the most matching key point pairs to determine the transformation parameters for image registration; transforming the image m to be registered according to the transformation parameters to obtain the registered image. p Perform transformation to obtain the registered image.
[0011] Furthermore, the key point extraction model consists of an encoder and a decoder. A skip connection is introduced between the encoder and the decoder. The output obtained after passing the output of the encoder through a first convolutional module and a second convolutional module is used as the intermediate feature representation. The output of the encoder is used as the input of the decoder, and the key point heat map is obtained through the decoder.
[0012] Furthermore, the encoder uses 3 downsampling blocks, and each downsampling block includes two first convolutional modules and a max pooling layer connected in sequence; the decoder uses two first upsampling blocks and a second upsampling block. The first upsampling block includes a first upsampling operation and two first convolutional modules connected in sequence. The second upsampling block includes a second upsampling operation, a first convolutional module, and a third convolutional module connected in sequence. The first upsampling operation uses the bilinear interpolation method, and the second upsampling operation uses the bicubic interpolation method; wherein, the first convolutional module consists of a two-dimensional convolutional layer, a batch normalization layer, and a ReLU activation function; the second convolutional module consists of a two-dimensional convolutional layer and a batch normalization layer; the third convolutional module uses a two-dimensional convolutional layer.
[0013] Furthermore, the descriptor extractor includes:
[0014] Step 1: Extract a 64-dimensional feature vector as the descriptor d1 of the key points by performing bilinear interpolation on the intermediate feature representation extracted in S2;
[0015] Step 2: Input the 32*32 local block around each key point into the deep learning network model to extract the 64-dimensional key point descriptor d2;
[0016] Step 3: Weightedly fuse the descriptor d1 extracted in Step 1 and the descriptor d2 extracted in Step 2 as the descriptor of the key points.
[0017] Further, the deep learning network model structure includes six sequentially connected convolutional blocks, an average pooling block, and three convolutional blocks; each convolutional block consists of a two-dimensional convolutional layer, a BatchNorm2d layer, and a ReLU activation function, and the average pooling block includes a Dropout layer, an AvgPool2d layer, a BatchNorm2d layer, and a ReLU activation function.
[0018] Further, the specific steps of the bidirectional matching brute-force algorithm are as follows:
[0019] Step4.1: For the image m to be registered p Calculate the Euclidean distance between the descriptors of each key point and the descriptors of each key point in the reference image m c and select the one with the smallest distance as the matching pair;
[0020] Step4.2: For the reference image m c Calculate the Euclidean distance between the descriptors of each key point and the descriptors of each key point in the image m to be registered p and select the one with the smallest distance as the matching pair;
[0021] Step4.3: Denote the key point in the reference image m c whose descriptor is most matched with the key point P p in the image m to be registered as P j ; when the key point obtained from the image m to be registered in step Step4.2 i whose descriptor is most matched with the key point P p in the reference image m c is P i , take P j P i P j as the most matched key point pair.
[0022] According to the second aspect of the present invention, there is provided a solar multi-band / sequence image registration system based on deep learning, including the module of the solar multi-band / sequence image registration method according to any one of the above.
[0023] The beneficial effects of the present invention are:
[0024] First, the present invention extracts the key point heat maps and intermediate feature representations of the reference image and the image to be registered through the key point extraction model respectively; based on the key point heat maps, combining the local maximum operation and the set pixel threshold, significant key points are screened out, and unrepresentative weak feature points are eliminated, so as to obtain the final key point sequence; further, the descriptor extractor weights and fuses the descriptors of the above key points extracted from the intermediate feature representation and the descriptors of the above key points extracted from the deep learning network model as the descriptors of the final key points. The descriptors extracted from the intermediate feature representation have stronger discrimination ability and can effectively distinguish key points at different positions, while the descriptors extracted from the deep learning network model have stronger robustness, so that the key points can still maintain a high degree of consistency under different imaging conditions (such as illumination changes, perspective differences); the combination of the two not only enhances the distinguishability of the key points, but also improves the consistency of the descriptors between key points at the same physical position, providing a more reliable basis for the subsequent matching process. Subsequently, a brute-force algorithm with two-way matching is used to construct the key point matching relationship between the reference image and the image to be registered. In the whole processing flow, the key point extraction model, the descriptor extractor and the brute-force algorithm with two-way matching form a bottom-up collaborative mechanism: high-quality key points provide a basis for feature description, highly robust descriptors ensure the accuracy of matching, and the matching algorithm realizes the matching of key points between the two images. The three cooperate closely to effectively improve the overall accuracy and stability of key point matching in the registration process. Finally, the present invention uses the RANSAC algorithm to exclude outliers from the initial key point matching pairs, selects the model with the most inliers as the final registration transformation parameters, and finally realizes the registration of the image according to the registration transformation parameters. The method proposed by the present invention solves the problem that the registration effect of previous methods (such as cross-correlation method, feature point detection-based method, phase correlation method and optical flow method) is not good in the registration of time series images in the solar quiet region, and has good effects in the registration of images in the solar active region and quiet region, and can also be used for the registration of images in different bands. Brief Description of the Drawings
[0025] Figure 1 is the flowchart of the method of the present invention;
[0026] Figure 2 is a set of solar images in the TiO band and Hα band;
[0027] Figure 3 is the structural diagram of the key point extraction model;
[0028] Figure 4 is the structural diagram of the deep learning network model in the descriptor extractor;
[0029] Figure 5 (a) is the key point matching pairs of the brute-force algorithm, Figure 5(b) is a comparison graph of key point matching pairs of the RANSAC algorithm;
[0030] Figure 6 (a) is the image after TiO band registration; Figure 6 (b) is the spliced image of the registered TiO band image and the Hα band;
[0031] Figure 7 is the Hα band time series quiet region image;
[0032] Figure 8 are the registration results of different methods for the time series quiet region images of the same band. (a) is the registration result of a multi-band solar image registration method, (b) is the registration result of the SIFT method, and (c) is the registration result of the method of the present invention. Specific implementation manner
[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other arbitrarily.
[0034] Example 1: As Figures 1 - 8 shown, according to the first aspect of the embodiments of the present invention, a method for registering multi-band / sequence images of the sun based on deep learning is provided, including the following steps:
[0035] S1. Obtain the sun band reference image m c and the image m to be registered p ; the sun band reference image and the image to be registered are images of different bands or sequence images of the same band;
[0036] It should be noted that the registration method of the present invention can be used for both images of different bands and sequence images of the same band, and has relatively good registration effects on both the time series quiet region images and active region images of the same band.
[0037] S2. Use the key point extraction model to extract the key point heat maps and intermediate feature representations of the sun band reference image m c and the image m to be registered p respectively; for the sun band reference image m c , the image m to be registered pThe key-point heatmaps respectively use non-maximum Suppression to find the local maxima in the key-point heatmaps; compare the local maxima with a preset pixel threshold, and regard the pixels greater than the pixel threshold as key points;
[0038] Further, the key-point extraction model consists of an encoder and a decoder. A skip connection is introduced between the encoder and the decoder. The output obtained after passing the output of the encoder through a first convolutional module and a second convolutional module is used as the intermediate feature representation. The output of the encoder is used as the input of the decoder, and the key-point heatmap is obtained through the decoder; the encoder uses 3 downsampling blocks, and each downsampling block includes two first convolutional modules and a max pooling layer connected in sequence; the decoder uses two first upsampling blocks and a second upsampling block. The first upsampling block includes a first upsampling operation and two first convolutional modules connected in sequence. The second upsampling block includes a second upsampling operation, a first convolutional module, and a third convolutional module connected in sequence. The first upsampling operation uses the bilinear interpolation method, and the second upsampling operation uses the bicubic interpolation method; wherein, the first convolutional module consists of a 3×3 two-dimensional convolutional layer, a batch normalization layer (BatchNorm2d), and a ReLU activation function; the second convolutional module consists of a 3×3 two-dimensional convolutional layer and a batch normalization layer; the third convolutional module uses a 1×1 two-dimensional convolutional layer; the key-point extraction network is as Figure 3 shown. The output of the first downsampling block is concatenated with the output after the upsampling operation of the second first upsampling block through a skip connection. The output of the second downsampling block is concatenated with the output after the upsampling operation of the first first upsampling block through a skip connection.
[0039] Exemplarily, the present invention uses a 3×3 non-maximum operation to find the local maxima in the key-point heatmap, and sets the pixel threshold to 30; the local maxima obtained through the non-maximum operation are further compared with the pixel threshold, and it can be realized that only those pixels that have both local maxima and are greater than the threshold are retained as key points. In this way, a set of important key points can be obtained for each image for subsequent feature extraction and matching. The specific expression is as follows:
[0040]
[0041] where x i,j represents the pixel value at the (i, j) position in the key-point heatmap, y i,j =1 indicates that the pixel is retained as a key point, y i,j =0 indicates that the pixel is not retained, and γ is the threshold.
[0042] S3. Use the descriptor extractor to extract the solar band reference image mc Descriptor of key points, and use a descriptor extractor to extract the image m to be registered p Descriptor of key points;
[0043] S4. For the solar band reference image m c Descriptor of key points, the image m to be registered p Use the bidirectional matching brute-force algorithm for the descriptor of key points to determine the most matching key point pairs; use the Random Sample Consensus (RANSAC) algorithm for the most matching key point pairs to determine the transformation parameters for image registration; according to the transformation parameters, transform the image m to be registered p to obtain the registered image.
[0044] Furthermore, the descriptor extractor includes:
[0045] Step 1: Extract a 64-dimensional feature vector as the descriptor d1 of key points by performing bilinear interpolation on the intermediate feature representation extracted in S2;
[0046] Step 2: Input the 32*32 local block around each key point into the deep learning network model to extract a 64-dimensional key point descriptor d2;
[0047] Step 3: Use weighted fusion of the descriptor d1 extracted in Step 1 and the descriptor d2 extracted in Step 2 as the descriptor of key points.
[0048] Furthermore, the structure of the deep learning network model includes 6 consecutive convolutional blocks, an average pooling block, and 3 convolutional blocks; each convolutional block consists of a two-dimensional convolutional layer, a BatchNorm2d layer, and a ReLU activation function, and the average pooling block includes a Dropout layer, an AvgPool2d layer, a BatchNorm2d layer, and a ReLU activation function.
[0049] Exemplarily, refer to Figure 4The deep learning network model includes six first convolutional blocks, an average pooling block, two fourth convolutional blocks, and a fifth convolutional block connected in sequence. The feature vector obtained by removing the last two dimensions through a squeeze operation after the last convolutional block is used as the key point descriptor extracted by the deep learning network model. The first convolutional block consists of a 3×3 two-dimensional convolutional layer, a BatchNorm2d layer, and a ReLU activation function. The fourth convolutional blocks each consist of a 1×3 two-dimensional convolutional layer, a BatchNorm2d layer, and a ReLU activation function. The fifth convolutional blocks each consist of a 1×2 two-dimensional convolutional layer, a BatchNorm2d layer, and a ReLU activation function. The average pooling block includes a Dropout layer, an AvgPool2d layer, a BatchNorm2d layer, and a ReLU activation function. Among them, the dropout rate of the Dropout layer is 0.1, and the pooling window size of the AvgPool2d layer is (8,1).
[0050] Further, the specific steps of the bidirectional matching brute-force algorithm are as follows:
[0051] Step4.1: For the image m to be registered p Calculate the Euclidean distance between the descriptor of each key point and the descriptor of each key point in the reference image m c and select the one with the smallest distance as the matching pair;
[0052] Step4.2: For the reference image m c Calculate the Euclidean distance between the descriptor of each key point and the descriptor of each key point in the image m to be registered p and select the one with the smallest distance as the matching pair;
[0053] Step4.3: Denote the key point in the reference image m c whose descriptor is the most matched with the key point P p in the image m to be registered obtained from the descriptor in Step4.1 as P j ; when the key point obtained from the image m to be registered in Step4.2 i whose descriptor is the most matched with the key point P p in the reference image m c is P i , take P j P i P j as the most matched key point pair.
[0054] Further, the RANSAC algorithm is specifically as follows: First, randomly extract n samples from the key point pairs matched in the previous step, and use the least squares method to calculate the optimal registration parameters to align the key points of the image to be registered with the corresponding key points of the reference image. Then, test the remaining matched key points. Use the currently obtained model to transform all points and test which points conform to this transformation model. If the Euclidean distance between the predicted position and the true position is less than the set inlier and outlier threshold, it means this point is an inlier, otherwise it is an outlier. Then, record the number of inliers, repeat this process multiple times, and select the model with the largest number of inliers as the final transformation model. Exemplarily, the thresholds for inliers and outliers are set to 3, and the number of iterations is 1000 times.
[0055] According to the second aspect of the embodiments of the present invention, there is provided a solar multi-band / sequence image registration system based on deep learning, including: an acquisition module for acquiring a solar band reference image m c and the image m to be registered p ; the solar band reference image and the image to be registered are images of different bands or sequence images of the same band; a first extraction module for respectively extracting the key point heat map and the intermediate feature representation of the solar band reference image m c and the image m to be registered p ; for the key point heat maps of the solar band reference image m c and the image m to be registered p respectively perform a local maximum operation to find the local maximum in the key point heat map; compare the local maximum with a preset pixel threshold, and use the pixel greater than the pixel threshold as the key point; a second extraction module for using a descriptor extractor to extract the descriptors of the key points of the solar band reference image m c and using the descriptor extractor to extract the descriptors of the key points of the image m to be registered p ; an obtaining module for using the brute-force algorithm of bidirectional matching for the descriptors of the key points of the solar band reference image m c and the descriptors of the key points of the image m to be registered p to determine the most matching key point pairs; using the random sample consensus algorithm for the most matching key point pairs to determine the transformation parameters for image registration; performing transformation on the image m to be registered according to the transformation parameters p to obtain the registered image. For the parts not detailed in each module, reference can be made to the relevant descriptions of this embodiment.
[0056] Embodiment 2: The following combines different band active region simulation data to illustrate a solar multi-band / sequence image registration method proposed by the present invention as follows:
[0057] 1. Obtain a solar band reference image m cand the image m to be registered p ; the solar band reference image and the image to be registered are images of different bands; for example Figure 2 is a pair of images obtained by the BBSO telescope of the Big Bear Solar Observatory in the United States at 17:17:10 on May 22, 2012. These two images are respectively two images in the Hα band and the TiO band. The size of the TiO band image is 1844*1844, and the size of the Hα band image is 920*920. As can be seen from the figure, their resolutions and fields of view are different.
[0058] II. Use the key point extraction model to extract the key point heat maps and intermediate feature representations of the solar band reference image m c and the image m to be registered p ; for the key point heat maps of the solar band reference image m c , the image m to be registered p , respectively use the local maximum operation to find the local maxima in the key point heat maps; compare the local maxima with the preset pixel threshold of 30, and use the pixels greater than the pixel threshold as key points;
[0059] III. Use the descriptor extractor to extract the descriptors of the key points of the solar band reference image m c , use the descriptor extractor to extract the descriptors of the key points of the image m to be registered p ;
[0060] IV. Use the two-way matching brute-force algorithm to determine 651 pairs of the most matching key point pairs for the descriptors of the key points of the solar band reference image m c , the descriptors of the key points of the image m to be registered p , as shown in Figure 5 (a); as shown in Figure 5 (b), use the RANSAC algorithm for the most matching key point pairs. The thresholds of the inliers and outliers of the RANSAC algorithm are set to 3. After 1000 iterations, finally 128 pairs of stable matching key points are retained for calculating the transformation parameters, that is, the TiO image to be registered is reduced to 0.7433 times of the original image, rotated clockwise by 0.958°, and translated by (231.26, 157.18). The registered TiO image is shown in Figure 6 (a). In order to show the registration accuracy, therefore Figure 2 the left half of Figure 6 (a) and Figure 6 (a) the right half of are stitched together, and the stitched image is shown in 6(b). The registered TiO image and the Hα image can be accurately aligned.
[0061] Embodiment 3: Considering that the registration of active regions in different bands is achieved in Embodiment 2, the following describes a method for registering solar multi-band / sequence images based on deep learning in combination with simulation data of quiet regions in the same band:
[0062] I. Obtain the solar band reference image m c and the image m to be registered p ; the solar band reference image and the image to be registered are images in different bands; for example Figure 7 They are a pair of images obtained by the NVST of the Yunnan Observatory at 1:24:20 and 1:25:27 on May 16, 2020 respectively. These two images are two quiet region images in the Hα band of the solar chromosphere. The size of these two images is 950*950.
[0063] II. Use the key point extraction model to extract the key point heat maps and intermediate feature representations of the solar band reference image m c and the image m to be registered p ; for the key point heat maps of the solar band reference image m c and the image m to be registered p , respectively adopt the local maximum operation to find the local maxima in the key point heat maps; compare the local maxima with the preset pixel threshold of 30, and take the pixels greater than the pixel threshold as key points;
[0064] III. Use the descriptor extractor to extract the descriptors of the key points of the solar band reference image m c and use the descriptor extractor to extract the descriptors of the key points of the image m to be registered p ;
[0065] IV. Use the two-way matching brute force algorithm to determine 828 pairs of the most matching key point pairs for the descriptors of the key points of the solar band reference image m c and the descriptors of the key points of the image m to be registered p ; for the most matching key point pairs, adopt the RANSAC algorithm. The thresholds of the inliers and outliers in the RANSAC algorithm are set to 3, and after 1000 iterations, finally 67 pairs of stable matching key points are retained for calculating the transformation parameters, that is, the TiO image to be registered is not scaled, rotated clockwise by 0.026°, and translated (-4.21, 7.74). To show the registration accuracy, the method proposed in the present invention is compared with the registration results of a multi-band solar image registration method and the SIFT method. Under each method, the solar band reference image shown in Figure 7 (a) and the registered image obtained by the corresponding method in Figure 7 (b) are stacked together, and the superimposed images of different methods are shown in Figure 8 . Figure 8(a) is a multi - band solar image registration method, Figure 8 (b) is the SIFT method, Figure 8 (c) is the method proposed by the present invention. The results show that the bright core pointed by the red arrow of the method proposed by the present invention is clearly visible, and the spicule structure is clear, and it can be registered Figure 7 , while the other two methods cannot be registered Figure 7 .
[0066] The specific embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above - mentioned embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art, various changes can be made without departing from the gist of the present invention.
Claims
1. A deep learning-based solar multi-band / sequence image registration method, characterized in that It includes the following steps: S1. Obtain the solar band reference image m c and the image m to be registered p ; the solar band reference image and the image to be registered are images of different bands or sequential images of the same band; S2. Use the key point extraction model to extract the key point heat maps and intermediate feature representations of the solar band reference image m c and the image m to be registered p respectively; For the solar band reference image m c and the image m to be registered p The key point heat maps are respectively processed by local maximum operation to find the local maxima in the key point heat maps; the local maxima are compared with a preset pixel threshold, and the pixels greater than the pixel threshold are taken as key points; S3. Use a descriptor extractor to extract the solar band reference image m c The descriptors of key points, and use a descriptor extractor to extract the image m to be registered p The descriptors of key points; S4. For the solar band reference image m c Descriptors of key points, the image m to be registered p Using the two-way matching brute-force algorithm for the descriptors of key points to determine the most matching key point pairs; using the random sample consensus algorithm for the most matching key point pairs to determine the transformation parameters for image registration; according to the transformation parameters, the image m to be registered p is transformed to obtain the registered image.
2. The method for registering solar multi-band / sequence images based on deep learning according to claim 1, wherein The key point extraction model consists of an encoder and a decoder. A skip connection is introduced between the encoder and the decoder. The output obtained after passing the output of the encoder through a first convolutional module and a second convolutional module is used as the intermediate feature representation. The output of the encoder is used as the input of the decoder, and the key point heat map is obtained through the decoder.
3. The multi-band / sequence image registration method of the sun based on deep learning according to claim 2, characterized in that The encoder uses 3 downsampling blocks. Each downsampling block includes two first convolutional modules and a max pooling layer connected in sequence. The decoder uses two first upsampling blocks and a second upsampling block. The first upsampling block includes a first upsampling operation and two first convolutional modules connected in sequence. The second upsampling block includes a second upsampling operation, a first convolutional module, and a third convolutional module connected in sequence. The first upsampling operation uses the bilinear interpolation method, and the second upsampling operation uses the bicubic interpolation method. Among them, the first convolutional module consists of a two-dimensional convolutional layer, a batch normalization layer, and a ReLU activation function. The second convolutional module consists of a two-dimensional convolutional layer and a batch normalization layer. The third convolutional module uses a two-dimensional convolutional layer.
4. The method for registering solar multi-band / sequence images based on deep learning according to claim 1, wherein The descriptor extractor includes: Step 1: Extract a 64-dimensional feature vector as the descriptor d1 of the key point by performing bilinear interpolation on the intermediate feature representation extracted in S2. Step 2: Input the 32*32 local block around each key point into the deep learning network model to extract a 64-dimensional key point descriptor d2. Step 3: The descriptor d1 extracted in Step 1 and the descriptor d2 extracted in Step 2 are weighted and fused as the descriptor of the key point.
5. The method for registering solar multi-band / sequence images based on deep learning according to claim 4, wherein, The structure of the deep learning network model includes 6 convolutional blocks, an average pooling block, and 3 convolutional blocks connected in sequence. Each convolutional block consists of a two-dimensional convolutional layer, a BatchNorm2d layer, and a ReLU activation function. The average pooling block includes a Dropout layer, an AvgPool2d layer, a BatchNorm2d layer, and a ReLU activation function.
6. The method for registering solar multi-band / sequence images based on deep learning according to claim 1, characterized in that The specific steps of the bidirectional matching brute force algorithm are as follows: Step 4.1: For the image m to be registered p Calculate the Euclidean distance between the descriptors of each key point and the descriptors of each key point in the reference image m c and select the one with the minimum distance as the matching pair; Step4.2: For the reference image m c Calculate the Euclidean distance between the descriptors of each key point and the descriptors of each key point in the image m to be registered p and select the one with the smallest distance as the matching pair; Step4.3: Obtain from the descriptors in Step4.1 of the reference image m c the key point P p of the image m j to be registered that has the best match with the descriptor, and denote it as P i ; when the key point obtained from the image m p to be registered in Step4.2 has the best match with the descriptor of the key point P c of the reference image m i and is P j , then use P i P j as the best-matched key point pair.
7. A solar multi-band / sequence image registration system based on deep learning, characterized in that, It includes the module of the deep learning-based solar multi-band / sequence image registration method according to any one of claims 1-6.