A method of air-to-ground geopositioning

Abstract

The application discloses a ground-to-sky geographic positioning method, characterized in that the method comprises the following steps: extracting local features of a ground image and a satellite image; transmitting the local features of the ground image to a satellite image domain to realize feature alignment; and using a triple loss function to calculate the difference between the features of the ground image and the satellite image and reversely updating network parameters. The method of the application transmits the local features of the ground image to the satellite image domain, improves the accuracy of cross-view geographic positioning, and is robust to direction disturbance. When the ground image does not specify a north direction, the method still has good positioning effect.

Description

TECHNICAL FIELD

[0001] The present application relates to cross-domain recognition technology, and in particular to a ground-to-air geographic positioning method. BACKGROUND

[0002] At present, the image information observed in cross-domain recognition comes from different cameras. Due to the difference of each camera itself, the re-identification has several characteristics. First, due to the difference of camera height and shooting direction, the images of different targets may be more similar in appearance than the images of the same target in different poses. Second, in actual shooting, due to the limitation of clarity and shooting angle, it is difficult to obtain the front information, and only the appearance features can be used for judgment. Third, under different cameras, the appearance features of the same target are different due to the influence of light, angle and pose. Influenced by the above characteristics, the re-identification of cross-domain targets has always been a hot spot of attention, and people are constantly exploring and researching to find better methods. In recent years, under the premise of single domain task, the constantly improved feature extraction and metric learning method has greatly improved the re-identification accuracy rate of large-scale data set. However, when directly using the model in different domains, the performance will be greatly reduced, which is very obvious when testing the model on new data set. For example, the top-1 recall rate of the current Market1501 can reach more than 95%, but the model with similar precision directly used in non-homologous tasks such as Duke MTMC reid, the top-1 recall rate is often less than 40%. This is the domain adaptation problem in cross-domain re-identification, which exists universally in the field of machine learning. And considering the relatively small data set and single scene in the field of cross-domain recognition, and in the cross-domain re-identification task, there is usually a significant domain difference between the source domain and the target domain, this problem is particularly serious. The main problem studied by this patent is the cross-domain positioning of ground images and corresponding satellite images, which is also essentially a cross-domain re-identification problem, and the source domain and the target domain differ by 90° in perspective. The task is: given a street scene ground image (such as a street view image), it is hoped that the shooting position of the image can be estimated according to the estimated position of the satellite map. Due to the wide availability of high-resolution satellite images, this cross-view positioning problem not only has practical significance, but also has important scientific significance, such as target positioning when a UAV is flying in the air, satellite positioning according to a geographic 7 picture, etc. But also has great challenge, the challenge mainly comes from the significant difference of the perspective of different images (one from the ground and the other from the top) and the great difference of visual appearance.

[0003] In the early stage, people use traditional features commonly used in the computer vision community for cross-view image matching. However, due to the huge difference in the viewpoint, the satellite image and the ground image at the same location are very different in features, which leads to the failure of direct matching of traditional local features. Therefore, many places warp the ground image into a top-down view to improve feature matching. For example, if the satellite image is tilted and the building facade is visible, geo-localization can be achieved through facade patch matching.

[0004] Since deep learning methods have proven to be very successful in image / video classification and recognition tasks, people have tried to introduce deep learning into the cross-view image matching and retrieval field. Workman and Jacobs fine-tuned AlexNet on ImagNet and Places, and then applied neural network to extract features for cross-view matching / localization. They showed that deep features for ordinary image classification are significantly better than hand-crafted features. Some researchers pointed out that better localization performance can be achieved by fine-tuning the satellite branch by minimizing the distance between satellite and ground images. Later, Workman et al. further improved the matching accuracy by training a convolutional neural network on the aerial branch. There have been thorough experiments on existing classification and retrieval networks, including binary classification networks, Siamese networks, and triplet networks. The most important part of image retrieval is to find a good image descriptor, which should have good discriminability and fast comparability. Some researchers proposed a bag of visual word descriptor, which aggregates a set of local features into a histogram of visual words, i.e., a global descriptor. The results show that the descriptor has partial viewpoint and occlusion invariance, which is better than local feature matching. Some researchers created a tree-structured vocabulary to support more visual words. Some researchers proposed a VLAD descriptor, which aggregates the residuals of local features to the cluster centroids instead of histograms. Based on this work, some researchers proposed a learnable VLAD layer, i.e., Net-VLAD, which can be embedded into a deep network for end-to-end training. In their extended paper, they showed that Net-VLAD outperforms multiple fully connected layers, max pooling, and VLAD. Due to the superior performance of Net-VLAD, CVM-NET adopted a Net-VLAD layer in the network. Hu et al. embedded a Net-VLAD layer on top of the VGG network based on the Siamese network to extract descriptors that are invariant to large angular changes. Some researchers incorporated per-pixel orientation information into CNN to learn the orientation selection function for cross-view localization. Through a new soft-margin triplet loss function and an exhaustive mini-batch training strategy, they achieved significant improvement in retrieval accuracy. On the other hand, some researchers proposed a weakly supervised training network to obtain the semantic layout of satellite images. These layouts are used as image descriptors for retrieval from a database.

[0005] Although the existing methods can achieve good results in cross-view geolocation tasks, they all have a shortcoming, that is, they all ignore the spatial layout information of the ground image. They only focus on what the features are, but not the relationship between the features, which leads to poor positioning effect in some application scenarios. For example, positioning in the wild forest or wilderness, because there is no prominent feature, it is easy to misjudge. In fact, the spatial layout information of the ground image is very important for positioning, such as positioning according to the layout information of houses and cars in reality. SUMMARY

[0006] The main purpose of the present application is to provide a ground-to-air geolocation method, which transmits the local features of the ground image to the satellite image domain, improves the accuracy of cross-view geolocation, and is robust to direction disturbance. When the ground image has no specified north direction, it still has good positioning effect.

[0007] The technical scheme adopted by the present application is: a ground-to-air geolocation method, comprising:

[0008] extracting local features of the ground image and the satellite image;

[0009] transmitting the local features of the ground image to the satellite image domain to realize feature alignment;

[0010] using a triple loss function to calculate the difference between the features and update the network parameters in reverse.

[0011] Further, the extracting local features of the ground image and the satellite image comprises:

[0012] using a VGG16 network to extract local features of the ground image and the satellite image;

[0013] The VGG16 network is composed of five convolutional layers and four pooling layers;

[0014] In the first convolutional layer, the input is the ground image and the satellite image, whose dimensions are 112x616x3 and 256x256x3 respectively. Two convolutional operations are performed on the image, and the convolution kernel dimensions are 3x3x3x64 and 3x3x64x64, that is, the convolution kernel length and width are 3, 3, the input depth is 3, the output depth is 64, and the input depth is 64, the output depth is 64. After the first convolutional layer, a maximum pooling layer is connected.

[0015] In the second layer convolution layer, the input is the output of the first layer convolution layer, the input depth is 64, two convolution operations are performed, the convolution kernel dimensions are 3*3*64*128 and 3*3*128*128 respectively, and then a maximum pooling layer is connected;

[0016] In the third layer convolution layer, the input is the output of the second layer convolution layer, the input depth is 128, three convolution operations are performed, the convolution kernel dimensions are 3*3*128*256, 3*3*256*256 and 3*3*256*256 respectively, and then a maximum pooling layer is connected;

[0017] In the fourth layer convolution layer, the input is the output of the third layer convolution layer, three convolution operations are performed, the convolution kernel dimensions are 3*3*256*512, 3*3*512*512 and 3*3*512*512 respectively, and a dropout function is used after each convolution, the keep prob is set to 0.8, and then a maximum pooling layer is connected;

[0018] In the fifth layer convolution layer, the input is the output of the fourth layer convolution layer, three convolution operations are performed, the convolution kernel dimensions are 3*3*512*512, 3*3*512*512 and 3*3*512*512 respectively, and a dropout function is used after each convolution, the keep prob is set to 0.8;

[0019] In all convolution operations, the step is set to 1, the activation function is relu, and the padding mode is set to SAME;

[0020] In all pooling operations, the pooling kernel dimension is 2*2; After five layers of convolution and four layers of pooling, the final output dimension is 8*8*512 feature map.

[0021] Further, the transmission of the local features of the ground image to the satellite image domain to realize feature alignment includes:

[0022] The transmission and alignment of features are performed using the sinkhorn-knopp algorithm; The definition of regularization is as follows:

[0023]

[0024] Wherein Obtained by the regularization operation of the cost function, the optimal transmission problem after regularization can be described as:

[0025]

[0026] where is a regularization coefficient, which can affect the strength of regularization on the optimal transport problem; , we have When transferring elements, if the mass is distributed on a few , the value of each will be large, while if the mass is distributed on multiple , the value of each will be small;

[0027] The Sinkhorn-Knopp algorithm first applies the exponential kernel to the cost matrix to obtain

[0028]

[0029] Second, the Sinkhorn-Knopp algorithm iteratively normalizes the rows and columns of to transform into a doubly stochastic matrix; the normalization of the rows and columns is as follows:

[0030]

[0031] where represents an element in ;

[0032] For the mth iteration, the output of the Sinkhorn algorithm can be recursively represented by equations 3-4:

[0033]

[0034] When the iteration converges, the feature transport matrix can be obtained:

[0035]

[0036] After obtaining the transport matrix, feature transport can be performed between the ground image and the satellite image, as shown in the following equation:

[0037]

[0038] where , represents the index of the channel, and represent the number of features in the satellite image and the ground image in the th channel, respectively;

[0039] After the feature maps of the ground image and the satellite image are extracted by the VGG16 network, another convolution layer is first used to reduce the feature dimension along the channel dimension instead of the spatial dimension, the spatial layout information of the extracted features is retained, and the final output feature dimension is 8x8x64=4096. Figure One Then the feature map of the ground image is adjusted to the same dimension as the satellite image feature map by the bilinear interpolation method; after the adjustment, the dimensions of the ground image feature map and the satellite image feature map are both 8x8x64; then a convolution operation is performed on the extracted ground feature map, the convolution kernel dimension is 1x1x64x1, the step is 1, the padding mode is SAME, and the activation function is relu; after the convolution, an 8x8x1 matrix is obtained, and then the matrix is passed through a fully connected layer to obtain a 1x64x64 matrix.

[0040] The matrix is subjected to Sinkhorn entropy regularization operation to obtain a feature transmission matrix; entropy regularization can effectively reduce the computational complexity, and the feature map matrix 1x8x8x64 of the ground image is adjusted to a 1x64x64 dimension matrix, three dimensions represent batch, channel and input dimension respectively, then matrix multiplication is performed with the feature transmission matrix 1x64x64 to obtain a 1x64x64 matrix, and the dimension is adjusted to 1x8x8x64, i.e. the satellite domain feature after the ground image feature transmission is obtained.

[0041] The transmitted ground image feature matrix and the satellite image feature matrix are both adjusted to 1x4096, and then L2 norm regularization is performed along the row coordinates; finally, the transmitted feature matrix is compared with the feature matrix of the satellite domain image, the loss is calculated, and the network parameters are updated in reverse propagation.

[0042] Further, the use of a triple loss function to calculate the difference between the two features and update the network parameters in reverse includes:

[0043] A weighted soft boundary ternary loss function is used, and the loss function is scaled by the coefficient when calculating :

[0044]

[0045] Twelve image pairs are input at a time, each ground image has a matching satellite image and 11 non-matching satellite images, and the ground image, the matching satellite image and the non-matching satellite image form a ternary group.

[0046] Advantages of the present application:

[0047] ​The method of the application transmits local features of a ground image to a satellite image domain, improves the accuracy of cross-view geolocation, and is robust to direction disturbance.

[0048] In addition to the objects, features, and advantages described above, the application has other objects, features, and advantages. The application will be described in further detail below with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings, which form a part of this application, are intended to provide further understanding of the application and are incorporated herein for a description of the application. The drawings are not intended to limit the application in any way.

[0050] Figure 1 is an image input diagram of the application;

[0051] Figure 2 is a feature alignment effect diagram of the application;

[0052] Figure 3 is a network structure diagram of the application. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions, and advantages of the application clearer, the application will be further described below with reference to the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application.

[0054] The method can be divided into three parts. First, local features of a ground image and a satellite image are extracted. Then, the local features of the ground image are transmitted to the satellite image domain to achieve feature alignment. Finally, a triple loss function is used to calculate the difference between the features of the two images and update the network parameters in reverse.

[0055] The overall network structure design of the method is shown in Figure 3

[0056] (1) Local feature extraction. The input of the method is a panoramic ground image and a satellite image. As shown in Figure 1 ​As shown, the VGG16 network is used for local feature extraction from ground and satellite images. The VGG16 network consists of five convolutional layers and four pooling layers. In the first convolutional layer, the inputs are the ground and satellite images, with dimensions of 112 × 616 × 3 and 256 × 256 × 3, respectively. Two convolutional operations are performed on the images, with kernel dimensions of 3 × 3 × 3 × 64 and 3 × 3 × 64 × 64, respectively. That is, the kernel width and height are 3 and 3, respectively, with an input depth (i.e., number of channels) of 3 and an output depth of 64. After the first convolutional layer, a max pooling layer is connected. In the second convolutional layer, the input is the output of the first convolutional layer, with an input depth of 64. Two convolution operations are performed, with kernel dimensions of 3 × 3 × 64 × 128 and 3 × 3 × 128 × 128 respectively, followed by a max-pooling layer. In the third convolutional layer, the input is the output of the second convolutional layer, with an input depth of 128. Three convolution operations are performed, with kernel dimensions of 3 × 3 × 128 × 256, 3 × 3 × 256 × 256, and 3 × 3 × 256 × 256 respectively, followed by a max-pooling layer. In the fourth convolutional layer, the input is the output of the third convolutional layer, and three convolution operations are performed with kernel dimensions of 3 × 3 × 256 × 512, 3 × 3 × 512 × 512, and 3 × 3 × 512 × 512, respectively. A dropout function is applied after each convolution, with a keep prob of 0.8, followed by a max pooling layer. In the fifth convolutional layer, the input is the output of the fourth convolutional layer, and three convolution operations are performed with kernel dimensions of 3 × 3 × 512 × 512, 3 × 3 × 512 × 512, and 3 × 3 × 512 × 512, respectively. A dropout function is applied after each convolution, with a keep prob of 0.8. In all convolutional operations, the stride is set to 1, the activation function is ReLU, and the padding mode is SAME. In all pooling operations, the pooling kernel dimension is 2 × 2. After five convolutional layers and four pooling layers, the final output is a feature map with dimensions of 8 × 8 × 512.

[0057] (2) Feature Transfer and Alignment. The concept of optimal transport is utilized, and the sinkhorn-knopp algorithm is used for feature transfer alignment. The sinkhorn-knopp algorithm is an entropy regularization solution for optimal transport, which can reduce computational cost and improve computational speed. The definition of regularization is as follows:

[0058]

[0059] in This is obtained by regularizing the cost function. The optimal transport problem, after regularization, can be described as:

[0060]

[0061] in This is the regularization coefficient, which affects the strength of regularization's effect on the optimal transport problem. Because... , can be obtained When transferring elements, if the mass distribution is in the minority Above, will make every The value of increases, while the mass is distributed across multiple When, each The value will be relatively small. The former will have a higher cost than the latter.

[0062] Specifically, the Sinkhorn-Knopp algorithm first processes the cost matrix... Applying the exponential kernel, we obtain

[0063]

[0064] Secondly, the Sinkhorn-Knopp algorithm is iteratively normalized. Rows and columns for conversion It is a doubly random matrix. The normalization of its rows and columns is as follows:

[0065]

[0066] in represent One of the elements.

[0067] For the m-th iteration, the Sinkhorn algorithm The output can be recursively represented by Equation 3-4:

[0068]

[0069] When the iteration converges, the characteristic transfer matrix can be obtained:

[0070]

[0071] After obtaining the transfer matrix, feature transfer can be performed between ground images and satellite images, as shown in the following formula:

[0072]

[0073] in , representing the index of the channel, and Representing satellite images and ground images respectively in the 1st The number of features in each channel.

[0074] The Sinkhorn-Knopp algorithm is specifically applied in deep neural networks as follows: After extracting feature maps from ground and satellite images using a VGG16 network, another convolutional layer is first used to reduce the feature dimension along the channel dimension rather than the spatial dimension, preserving the spatial layout information of the extracted features. The final output feature dimension is 8×8×64=4096. Then, bilinear interpolation is used to adjust the feature map of the ground image to match the feature map of the satellite image. Figure One The dimensions are adjusted so that both the ground image feature map and the satellite image feature map are 8x8x64. Then, a convolution operation is performed on the extracted ground feature map with a kernel dimension of 1×1×64×1, a stride of 1, a padding mode of SAME, and an activation function of ReLU. After convolution, an 8×8×1 matrix is ​​obtained. This matrix is ​​then passed through a fully connected layer to obtain a 1×64×64 matrix. Finally, this matrix is ​​subjected to Sinkhorn entropy regularization to obtain the feature transfer matrix. Entropy regularization effectively reduces computational complexity. The ground image feature map matrix (1×8×8×64) is adjusted to a 1×64×64 matrix, where the three dimensions represent the batch, channel, and input dimension (i.e., the number of features in one channel), respectively. Then, a matrix multiplication is performed with the feature transfer matrix (1×64×64) to obtain a 1×64×64 matrix. Its dimension is then adjusted to 1×8×8×64, thus obtaining the satellite domain features after ground image feature transfer. Finally, both the transmitted ground image feature matrix and the satellite image feature matrix are adjusted to a dimension of 1×4096, and then L2 norm regularization is performed along the row coordinates. Finally, the transmitted feature matrix is ​​compared with the feature matrix of the satellite domain image, the loss is calculated, and the transmission matrix is ​​updated via backpropagation. The result is as follows. Figure 2 As shown.

[0075] (3) Triple Loss Function. The method uses the triple loss function, which is widely used to train deep neural networks for image localization and matching tasks. It is defined as: minimizing the distance between the anchor point and positive samples with the same identity, while maximizing the distance between the anchor point and negative samples with different identities. Its goal is to make features with the same label as close as possible in spatial location, while features with different labels as far apart as possible in spatial location. Here, "triple" refers to a triplet: anchor, positive, and negative, i.e., anchor point, positive sample, and negative sample. In the application scenario of this paper, this means ground images, satellite images that match the ground images, and satellite images that do not match. The simplest triple loss function is the max-margin triple loss function:

[0076]

[0077] in and This represents the distances of all positive and negative samples to the selected anchor point, while... It refers to the distance from the edge. Later, scholars proposed the soft-boundary ternary loss function to avoid determining the margin value in the ternary loss function.

[0078]

[0079] in The CVM-NET network uses a soft-boundary ternary loss function for training, but this results in slow convergence. To improve convergence speed, this algorithm employs a weighted soft-boundary ternary loss function, which calculates... Time according to coefficient Scaling :

[0080]

[0081] In the method of the present invention, 12 image pairs are input at one time. Each ground image has one matching satellite image and 11 non-matching satellite images. The ground image, the matching satellite image and the non-matching satellite images constitute a triple.

[0082] The image feature extraction method of this invention can be replaced by other neural network methods such as ResNet50 and Inception_v4.

[0083] This invention transmits local features of ground images to the satellite image domain to achieve feature alignment.

[0084] This invention addresses the problems existing in current UAV aerial image sequence stitching technology and the characteristics of UAV aerial image sequences by proposing an incremental real-time UAV aerial image sequence stitching algorithm.

[0085] To address the shortcomings of existing methods that ignore spatial layout information in ground imagery, a novel cross-view geolocation method is proposed. This method can transfer local features from ground imagery to the satellite imagery domain, achieving feature alignment between the two for more meaningful comparisons, ultimately improving the accuracy and reliability of geolocation.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A ground-to-air geographic positioning method, characterized in that, include: Local feature extraction from ground and satellite images; Local features from ground images are transferred to the satellite image domain to achieve feature alignment; A triple loss function is used to calculate the difference between the two features and to update the network parameters in reverse. The process of transferring local features from the ground image to the satellite image domain to achieve feature alignment includes: Feature transfer alignment is performed using the sinkhorn-knopp algorithm; the definition of regularization is as follows: ， in The optimal transport problem, obtained by regularization of the cost function, can be described as follows: ， in It is the regularization coefficient, which affects the strength of the effect of regularization on the optimal transmission problem; , can be obtained When transferring elements, if the mass distribution is in the minority Above, will make every The value of increases, while the mass is distributed across multiple When, each The value will be relatively small; The Sinkhorn-Knopp algorithm first processes the cost matrix... Applying the exponential kernel, we obtain ， Secondly, the Sinkhorn-Knopp algorithm is iteratively normalized. Transform rows and columns It is a doubly random matrix; the normalization of its rows and columns is as follows: ， ， in represent One of the elements; For the m-th iteration, the Sinkhorn algorithm The output can be recursively represented by Equation 3-4: ， When the iteration converges, the characteristic transfer matrix can be obtained: ， After obtaining the transfer matrix, feature transfer can be performed between ground images and satellite images, as shown in the following formula: ， in , representing the index of the channel, and Representing satellite images and ground images respectively in the 1st The number of features in each channel; After extracting feature maps from ground and satellite images using the VGG16 network, another convolutional layer is first used to reduce the feature dimension along the channel dimension rather than the spatial dimension, preserving the spatial layout information of the extracted features. The final output feature dimension is 8×8×64=4096. Then, bilinear interpolation is used to adjust the feature map of the ground image to the same dimension as the feature map of the satellite image. After adjustment, the dimensions of both the ground and satellite image feature maps are 8x8x64. Then, a convolution operation is performed on the extracted ground feature map with a kernel dimension of 1×1×64×1, a stride of 1, a padding mode of SAME, and an activation function of ReLU. After convolution, an 8×8×1 matrix is ​​obtained. This matrix is ​​then passed through a fully connected layer to obtain a matrix with a dimension of 1×64×64. The matrix is ​​subjected to Sinkhorn entropy regularization to obtain the feature transfer matrix. Entropy regularization can effectively reduce computational complexity. The feature map matrix of the ground image is adjusted from 1×8×8×64 to a matrix of 1×64×64 dimensions, where the three dimensions represent the batch, channel, and input dimensions, respectively. Then, matrix multiplication is performed with the feature transfer matrix of 1×64×64 to obtain a matrix of 1×64×64. Its dimensions are then adjusted to 1×8×8×64, which gives us the satellite domain features after the ground image features are transferred. The feature matrices of the transmitted ground image and satellite image are both adjusted to a dimension of 1×4096, and then L2 norm regularization is performed along the row coordinates. Finally, the transmitted feature matrix is ​​compared with the feature matrix of the satellite domain image, the loss is calculated, and the transmission matrix is ​​updated by backpropagation. The method of using a triple loss function to calculate the difference between the features of the two features and then updating the network parameters in reverse includes: The weighted soft-boundary ternary loss function is adopted, which is used in the calculation of... Time according to coefficient Scaling : ， Input 12 image pairs at once. Each ground image has one matching satellite image and 11 non-matching satellite images. The ground image, the matching satellite image, and the non-matching satellite images constitute a triple.

2. The air-to-ground geographic positioning method according to claim 1, characterized in that, The local feature extraction of ground and satellite images includes: Local feature extraction of terrestrial and satellite images was performed using the VGG16 network. The VGG16 network consists of five convolutional layers and four pooling layers. In the first convolutional layer, the inputs are ground images and satellite images, with dimensions of 112 × 616 × 3 and 256 × 256 × 3, respectively. The images are subjected to two convolutional operations with kernel dimensions of 3 × 3 × 3 × 64 and 3 × 3 × 64 × 64, respectively. That is, the kernel width and height are 3 and 3, respectively, with an input depth of 3 and an output depth of 64, and an input depth of 64 and an output depth of 64. After the first convolutional layer, a max pooling layer is connected. In the second convolutional layer, the input is the output of the first convolutional layer with an input depth of 64. Two convolution operations are performed, with convolutional kernel dimensions of 3 × 3 × 64 × 128 and 3 × 3 × 128 × 128, respectively. Then, a max pooling layer is connected. In the third convolutional layer, the input is the output of the second convolutional layer, with an input depth of 128. Three convolutional operations are performed, with convolutional kernel dimensions of 3 × 3 × 128 × 256, 3 × 3 × 256 × 256, and 3 × 3 × 256 × 256, respectively. Then, a max pooling layer is connected. The fourth convolutional layer takes the output of the third convolutional layer as input and performs three convolution operations with kernel dimensions of 3 × 3 × 256 × 512, 3 × 3 × 512 × 512, and 3 × 3 × 512 × 512, respectively. After each convolution, a dropout function is used, and keep prob is set to 0.

8. Then, a max pooling layer is connected. In the fifth convolutional layer, the input is the output of the fourth convolutional layer, and three convolution operations are performed. The convolutional kernel dimensions are 3 × 3 × 512 × 512, 3 × 3 × 512 × 512, and 3 × 3 × 512 × 512, respectively. After each convolution, the dropout function is used once, and keep prob is set to 0.

8. In all convolution operations, the stride is set to 1, the activation function is ReLU, and the padding mode is SAME. In all pooling operations, the pooling kernel dimension is 2 × 2; after five convolutional layers and four pooling layers, the final output feature map has a dimension of 8 × 8 × 512.

Images