An unmanned aerial vehicle image style transfer method based on a generative adversarial network

Abstract

The application discloses a kind of unmanned aerial vehicle image style migration methods based on cyclic adversarial generative network. The ground image is vertically photographed using unmanned aerial vehicle, and according to the corresponding GPS and yaw angle, it is cut on satellite map, the unmanned aerial vehicle image is saved to unmanned aerial vehicle domain, and the cut satellite image is saved to satellite domain, complete dataset production, train cyclic adversarial generative network using dataset to complete the image style conversion between two domains, and the generator of unmanned aerial vehicle domain to satellite domain in cyclic adversarial generative network is extracted for style migration. The method in the application is more effective when applied to new data photographed in the environment similar to the environment where the unmanned aerial vehicle image is photographed when the training set is produced, thereby narrowing the similarity between the unmanned aerial vehicle map and the satellite map, and thereby facilitating the improvement of the precision between the unmanned aerial vehicle image and the satellite image matching, thereby improving the accuracy of pure vision unmanned aerial vehicle positioning.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing and deep learning, and particularly relates to a UAV image style transfer method based on a generative adversarial network and using satellite images. BACKGROUND

[0002] Most of the current UAVs rely on satellite systems to obtain their latitude and longitude position information, but in the process of using UAVs to perform tasks, it is inevitable to encounter a GNSS (Global Navigation Satellite System) environment, that is, a situation where GPS signals cannot be relied on. In this case, the UAV is difficult to complete self-positioning, and thus cannot perform tasks.

[0003] In the past period of time, UAV cross-view matching has been proposed to solve the problem of UAV self-positioning in a GNSS-free environment by relying solely on vision.

[0004] The mainstream method today is to match the image features extracted from the UAV images by a deep learning model with the satellite image features extracted by the same method by calculating the similarity between them. However, due to the large difference between the UAV view and the satellite view, the matching accuracy based on this method needs to be improved.

[0005] At the same time, with the advent of generative adversarial networks in recent years and the continuous upgrading of their network models, their applications in various fields have been increasing. In the present application, a recurrent generative adversarial network is used to complete the image transfer function, aiming to preprocess the UAV images before inputting them into the positioning network by image conversion, making them closer to the style of satellite images, improving their similarity with corresponding satellite images, and thus reducing the difficulty of model training and improving the matching accuracy. SUMMARY

[0006] The present application aims to provide a UAV image style transfer method based on a generative adversarial network and using satellite images, which is used to process UAV image data before feature extraction and matching, and to perform style transfer on UAV images to narrow the gap between them and satellite images, thereby improving the matching accuracy of UAV visual positioning.

[0007] To achieve the above-mentioned purpose, the technical solution of the present application includes the following steps:

[0008] The main steps of the application are as follows: 1. Using a UAV to vertically shoot ground photos at a certain height from the ground. 2. Cutting satellite images according to the GPS information and yaw angle information stored on the UAV shooting photos. 3. Making a data set including two domains, the UAV shooting photos in the UAV domain and the satellite images in the satellite domain. 4. Training based on the cycle generative adversarial network using the data set. 5. Using the trained generator in the UAV domain to satellite domain in the generative adversarial network to perform style transfer on other shot UAV images.

[0009] Step 1: vertically shooting ground images using a UAV;

[0010] Step 2: cutting on the satellite map according to the GPS information and yaw angle information corresponding to the ground image when the UAV shoots the ground image, to obtain the cut satellite image;

[0011] Step 3: saving the UAV images to the UAV domain, and saving the cut satellite images to the satellite domain, to complete the data set making;

[0012] Step 4: training the cycle generative adversarial network using the data set to complete the image style conversion between the two domains;

[0013] Step 5: extracting the generator in the UAV domain to satellite domain in the cycle generative adversarial network for style transfer of other UAV real-time shooting images.

[0014] The step 1 is specifically: using a DJI UAV to vertically downwardly shoot pictures at a plurality of known heights from the ground, and storing the picture data on the storage card of the UAV to obtain the ground images from the perspective of the UAV.

[0015] In the step 1, a DJI UAV loaded with an sd storage card is used to vertically downwardly shoot a plurality of ground images at a known height from the ground using a gimbal, and the height must be known. In the subsequent step of cutting satellite images to make a data set, the satellite image cutting size needs to be adjusted according to the height. The flight route of the UAV is planned using waypoint flight, and a waypoint action of vertically downwardly shooting is set at each waypoint, and the distance between two waypoints cannot be too large, and the entire area is densely covered as much as possible.

[0016] During the shooting process, high-rise buildings or special facilities that may shield the signal of the UAV should be avoided.

[0017] During the shooting process, the exposure of the gimbal camera should be paid attention to at any time, so as to avoid the overexposure or overdarkness of the shot ground images, which cannot be used as effective training data.

[0018] The step 2 is specifically: reading the GPS and the yaw angle of the picture shooting in the exif, xmp area of the UAV image attribute through the API function, intercepting the satellite image under the same GPS on the satellite map, and rotating the satellite image with the yaw angle.

[0019] In the step 2: after the image shooting is completed, the sd memory card in the UAV is taken out, and the ground image data shot by the UAV is obtained. The GPS information and the yaw angle information of the picture shooting are read in the exif and xmp area of the UAV image attribute through the function in the python-pyexiv2 library.

[0020] According to the shooting area, a satellite large image larger than the shooting coverage area and completely covering the shooting area is intercepted on the satellite map, and the GPS information of the top left corner, the bottom left corner, the top right corner and the bottom right corner of the satellite large image is required.

[0021] Knowing the GPS information of the satellite large image and the GPS information of the UAV image, the satellite image corresponding to the position of the UAV image can be intercepted on the satellite large image.

[0022] In the past, the corresponding image dataset was intercepted on the satellite large image according to the GPS information of the UAV. However, due to the reason that the width of the satellite large image intercepted on most satellite maps is parallel to the latitude line, and the height is parallel to the longitude line, the corresponding satellite image intercepted directly on the satellite large image will also be fixed parallel to the longitude line as the satellite large image, and the buildings or roads on the image are mostly east-west. Figure One

[0023] This is different from the situation that the orientation of the UAV image shot by the UAV is different. Although the images are not one-to-one paired in the model training process, if the UAV images shot and the satellite images directly intercepted without rotation based on the yaw angle are used as training data for training, serious overfitting phenomenon will occur.

[0024] Many buildings on the UAV images with different orientations during shooting are constantly approaching the buildings on the satellite images fixed in the east-west direction at the pixel level during training, so that the UAV images after image translation have serious distortion.

[0025] In order to make the intercepted satellite image and the UAV image have the same center position and the same shooting direction, the satellite large image should be rotated based on the yaw angle of the UAV image before each interception, so that the satellite image with the same shooting direction is intercepted. This operation effectively solves the distortion problem of the translated image mentioned above.

[0026] ​The step 3 is specifically: preparing a data set for training of the cyclic adversarial generation network, center cutting the photographed unmanned aerial vehicle picture, taking a square with the shortest side as the edge length and changing the size to a square image with fixed 256*256 pixels, and similarly, the satellite image is also changed in size to a square image with fixed 256*256 pixels; saving the unmanned aerial vehicle image to the unmanned aerial vehicle domain as the unmanned aerial vehicle true image, saving the cut satellite image to the satellite domain as the satellite true image, and completing the data set preparation.

[0027] In the step 3: the satellite images obtained by cutting are modified in size and integrated into the satellite domain of the data set, and the unmanned aerial vehicle images are center cut and modified in size and integrated into the unmanned aerial vehicle domain of the data set. One satellite image is cut for each unmanned aerial vehicle image photographed, but when the data set is prepared for training, the data is trained in the form of unpaired data. That is, there is no matching relationship between the images of the two domains input into the model for training each time.

[0028] The step 4 is specifically:

[0029] The model of the cyclic adversarial generation network mainly includes four parts: a generator from the unmanned aerial vehicle domain to the satellite domain, a generator from the satellite domain to the unmanned aerial vehicle domain, an unmanned aerial vehicle domain discriminator, and a satellite domain discriminator, which are respectively:

[0030] The generator from the unmanned aerial vehicle domain to the satellite domain is used to receive an unmanned aerial vehicle true image to generate a satellite false image;

[0031] The generator from the satellite domain to the unmanned aerial vehicle domain is used to receive a satellite true image to generate an unmanned aerial vehicle false image;

[0032] The unmanned aerial vehicle domain discriminator is used to receive an unmanned aerial vehicle true image and an unmanned aerial vehicle false image and discriminate the true and false of the input unmanned aerial vehicle true image and the generated unmanned aerial vehicle false image;

[0033] The satellite domain discriminator is used to receive a satellite true image and a satellite false image and discriminate the true and false of the input satellite true image and the generated satellite false image;

[0034] Taking an input unmanned aerial vehicle image as an example, first, the unmanned aerial vehicle image is input into the generator from the unmanned aerial vehicle domain to the satellite domain to generate a satellite false image, and the input unmanned aerial vehicle image and the generated satellite false image are discriminated by the unmanned aerial vehicle domain discriminator and the satellite domain discriminator, respectively, and then the generated satellite false image is input into the generator from the satellite domain to the unmanned aerial vehicle domain to generate an unmanned aerial vehicle reference image for comparison with the input unmanned aerial vehicle true image.

[0035] The training cycle is used to complete the mutual conversion of images between the unmanned aerial vehicle image and the satellite image, specifically, the two discriminators of the unmanned aerial vehicle domain discriminator and the satellite domain discriminator are set according to the following loss, and then the loss is minimized to train and optimize:

[0036] The loss calculated by the discriminator is used for gradient back propagation and weight update of the model, and the specific calculation formula of the loss is as follows:

[0037]

[0038] Wherein, G, F represent the generator for mutual conversion between the two domains, Dx, Dy represent the discriminators of the two domains, And The binary minimax game loss used in the cyclic adversarial network requires the discriminator to more accurately distinguish between the generated pseudo image and the true image, and requires the generator to generate a pseudo image that makes the discriminator distinguish as a true image. In the process of adversarial training, the generator will generate a pseudo image that is difficult to distinguish. The cyclic consistency loss in the cyclic adversarial network, lambda is a pre-set hyperparameter, used to change the influence of this loss on model training, the size of the loss is determined by the overall pixel difference between a true image and a reference image obtained by reversing the image conversion of the true image through a generator and another generator, the greater the pixel difference between the two images, the greater the loss.

[0039] In the training process, the ability of the discriminator to make correct discrimination becomes stronger and stronger, while the generator can generate a pseudo image that makes the discriminator distinguish as a pseudo image as much as possible.

[0040] The step 5 is specifically: after the training is completed, the generator from the unmanned aerial vehicle domain to the satellite domain in the cyclic adversarial network is extracted for style migration of the unmanned aerial vehicle real-time shooting image, specifically, after a real-time vertical shooting image of the unmanned aerial vehicle is input into the generator from the unmanned aerial vehicle domain to the satellite domain after center cutting operation and size changing operation to change the square cutting into a fixed 256*256 pixel square image, the convolution and inverse convolution operation is performed, and a satellite style image based on the unmanned aerial vehicle image is generated and output.

[0041] The beneficial effects of the present application are:

[0042] The method provided in the present application is more effective when applied to new data shot in an environment similar to the environment in which the unmanned aerial vehicle image for making the training set is shot, thereby narrowing the similarity between the unmanned aerial vehicle image and the satellite image, and thereby facilitating the improvement of the precision of the matching between the unmanned aerial vehicle image and the satellite image, thereby improving the accuracy of the pure vision unmanned aerial vehicle positioning. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 Flowchart for the UAV image style transfer method based on the generative adversarial network of the present application;

[0044] Figure 2 Diagram and effect diagram for the method of intercepting a satellite image after rotating a satellite large image based on a yaw angle;

[0045] Figure 3 Flowchart for the training of the generative adversarial network in the present application;

[0046] Figure 4 Effect diagram obtained after inputting the UAV image into the trained generator for style transfer;

[0047] Figure 5 Effect diagram of style transfer by the trained generator based on the dataset produced without the yaw angle rotation operation. Specific embodiments

[0048] To make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application are described in further detail below.

[0049] As Figure 1 shown, the specific examples and implementation processes of the present application are as follows:

[0050] Step 1: A total of 3033 waypoints were planned above the 14 colleges in Hangzhou Xiasha using waypoint flight, and a gimbal was set vertically downward at each waypoint. These waypoints were divided into 14 groups in total, each group covering a region of one college, and the distance between the waypoints was mostly about 20 m. Then, the DJI UAV loaded with an sd memory card was used to vertically shoot multiple ground images at a height of 80 m, 90 m and 100 m from the ground.

[0051] Since the distance between the waypoints is about 20 m, the ground images taken by the UAV have a relatively small interval, effectively covering the entire area photographed. In order to make the dataset produced contain both parts with obvious visual features and parts with unobvious visual features, the trained network generator will have better generalization ability, preventing the generation of images with the same style due to model collapse.

[0052] During the image shooting process, high-rise buildings or special facilities that may shield the UAV signal should be avoided.

[0053] During shooting, pay close attention to the exposure of the gimbal camera to avoid overexposing or underexposing the ground images due to weather changes or sunlight obstruction, which would render the data unusable as training data. Avoid shooting at night when ground images are not clearly visible.

[0054] A total of 9,099 drone images were taken at altitudes of 80m, 90m, and 100m, with 3,033 images taken at each altitude.

[0055] Step 2: After image capture, remove the SD memory card from the drone and extract the ground image data captured by the drone. Use functions from the python-pyexiv2 library to read the EXIF ​​and XMP regions of all 9099 drone images to read the GPS information and yaw angle information at the time of image capture.

[0056] Based on the 14 areas captured, 14 larger satellite images, each larger than the captured area and completely covering it, were extracted from the satellite map. Accurate GPS information for the top left, bottom left, top right, and bottom right corners of these 14 large satellite images was obtained. During this process, the GPS information from the drone imagery was used to locate and confirm the extracted satellite images on the large satellite images. Figure Four The accuracy of corner position information.

[0057] Knowing the GPS information of the large satellite image and the GPS information of the center position of the drone image, you can extract a satellite image from the large satellite image that corresponds to the position of the drone image.

[0058] Previously, data sets created based on images captured by drones were created by directly cropping images from large satellite images using the drone's GPS information. However, because most large satellite images are parallel to latitude lines in width and longitude lines in height, directly cropping the corresponding satellite image from a large satellite image will result in errors similar to those found on the large satellite image. Figure One Generally, the orientation of an image will be fixed and parallel to the meridians, and buildings or roads in the image will mostly run east-west.

[0059] This differs from actual drone photography, where images are taken from various orientations. Although the model training process uses a non-one-to-one pairing of images, using drone images and satellite images directly cropped without yaw angle rotation as training data will result in severe overfitting.

[0060] Many buildings in drone images taken from different directions are gradually brought closer to buildings in satellite images that are fixed to an east-west orientation at the pixel level during training, resulting in severe distortion in the drone images after image escaping.

[0061] In order to make the intercepted satellite image and the unmanned aerial vehicle image in the same center position, the same in the shooting direction, the satellite large image is rotated based on the yaw angle of the unmanned aerial vehicle image before each interception, so as to make the intercepted satellite image in the same shooting direction. This operation effectively solves the distortion problem of the above-mentioned escaped image.

[0062] Taking an unmanned aerial vehicle image as an example, the steps of intercepting the corresponding satellite image are as follows:

[0063] The exif, xmp area of the unmanned aerial vehicle image attribute is read by the function in the python-pyexiv2 library, and the GPS information of the picture shooting is read, including longitude N and latitude S and yaw angle θ. At the same time, a satellite large image covering the shooting area of the unmanned aerial vehicle image is intercepted on the satellite map, and the width and height of the satellite large image are W and H pixels, respectively, and the longitude and latitude of the upper left corner and the longitude and latitude of the lower right corner are T N , T S , B N , B S .

[0064] According to the above known conditions, the pixel position (X, Y) of the center of the unmanned aerial vehicle image in the satellite large image can be directly derived according to the formula, which is as follows:

[0065]

[0066]

[0067] Taking the pixel position (X, Y) as the center point, a rotation matrix Mat is obtained using the function getRotationMartrix2D with the yaw angle θ as the rotation angle, and the rotation matrix is shown in the following formula:

[0068]

[0069] Based on this rotation, the position of each pixel (x, y) of the picture is changed, which is as follows:

[0070]

[0071] After the above work can be directly through the ROI satellite image capture operation, need to note that the different height of the UAV image to use different size of the square frame to intercept, for 80m height of the UAV image, after the acquisition GPS pixel position (X, Y), with the position as the center to intercept a 640*640 pixel size of the square. For 90m height of the UAV image using a 768*768 pixel size of the square to intercept frame. For 100m height of the UAV image using a 896*896 pixel size of the square to intercept frame. Figure 2 For the satellite image based on the yaw angle of the satellite large map is rotated after the method of cutting satellite map and effect diagram.

[0072] Need to note that in the use of affine transformation (warpAffine) function for rotation transformation need to pay attention to the scaling factor needs to be 1, the picture size after affine transformation needs to be larger than the width and height of the original picture W, H, so as to avoid the situation of too little area can be cut after the affine transformation of the whole satellite large map rotation out of the picture window.

[0073] Step 3: the satellite image cutting to the size of the modification operation is modified to 256*256 pixel size after the integration into the satellite domain of the data set, the UAV image shooting image center cutting and size modification operation is modified to 256*256 pixel size after the integration into the UAV domain of the data set.

[0074] The data set is divided into training set and test set. 10 of the 14 universities, a total of 6768 UAV images and corresponding 6768 satellite images are used as training data set. Another 4 universities corresponding to 2331 UAV images and corresponding 2331 satellite images are used as test data set. The training set is used to input the model during training to help the network training, and the test set is used to verify the effect after the model training.

[0075] Each UAV image is intercepted, but when making into data set for training, the form of unpaired data is adopted for training. That is, there is no matching relationship between the two domain images input into the model for training each time.

[0076] Step 4: training cycle of generative adversarial network, to complete the mutual conversion of UAV image and satellite image.

[0077] The cycle GAN mainly consists of four parts, which are the generator from UAV domain to satellite domain, the generator from satellite domain to UAV domain, the discriminator for UAV domain and the discriminator for satellite domain. The network structure of the cycle GAN is shown in Figure 3 .

[0078] The architectures of the two generators are the same. Here, the architecture of the generator is introduced by taking the change of the feature scale of an input model picture as an example. After a 256*256*3 picture is input, it is subjected to several padding, convolution layers, normalization layers and activation functions, and then enters the ResnetBlock stacked 9 times to extract the image features. At this time, the size of the feature map obtained is 64*64*256. Meanwhile, the original picture 256*256*3 is input into the residual module, i.e. a ResnetBlock, to obtain a feature map with a scale of 64*64*256. The two feature maps are added together, and finally subjected to two inverse convolution layers and activation functions, etc. The size is changed to 256*256*3 again, which is the size of the generated pseudo picture.

[0079] The architectures of the two discriminators are the same, which are composed of a small number of convolution layers, normalization layers and LeakyRelu activation function layers. An input image with a size of 256*256*3 can obtain a prediction map with a size of 30*30*1 after being subjected to these architectures. The discriminator obtains a prediction with a size of 30*30, i.e. the input picture is divided into 30*30 parts for discrimination. The final loss is calculated by discriminating the prediction values of the 30*30 parts. For the case of inputting a true picture, the discriminator hopes that the 30*30 prediction scores are closer to 1, and vice versa.

[0080] Taking an input UAV picture as an example, first, the UAV picture is input into the generator from UAV domain to satellite domain to generate a satellite pseudo picture. The UAV domain discriminator and the satellite domain discriminator are used to discriminate the input UAV picture and the generated satellite pseudo picture, respectively. Then, the generated satellite pseudo picture is input into the generator from satellite domain to UAV domain to generate a UAV reference picture for comparison with the input UAV true picture.

[0081] The loss calculated by the discriminator is used for gradient back propagation and weight update of the model. The specific calculation formula of the loss is as follows:

[0082]

[0083] where G and F represent the generators for mutual conversion between the two domains, D represents the discriminator, and x represents the input picture.x D y These represent discriminators for the two domains respectively.

[0084] and This is the binary minimax game loss used in adversarial generative networks. During training, this loss requires the discriminator to more accurately distinguish between generated fake and real images, and the generator to generate fake images that are as likely as possible to be identified as real by the discriminator. During adversarial training, the generator will generate highly realistic fake images. For the cycle consistency loss in the recurrent adversarial generative network, λ is a hyperparameter that can be set by the user to change the degree of influence of this loss on model training. The magnitude of this loss is determined by the overall pixel difference between a real image and a reference image obtained by passing the real image through one generator and then through another generator for inverse image escaping. The greater the pixel difference between the two images, the greater the loss.

[0085] In addition to the losses mentioned above, ontology loss is also used in the actual training process. Taking a drone image as an example, this image is fed into the generator that connects the satellite domain to the drone domain to generate an ontology map of the drone image. The difference between the real image and the ontology map is compared. The more similar the two images are, the smaller the loss is, and vice versa.

[0086] In the final step, step 5, after model training is complete, the UAV-to-satellite domain generator is extracted for style transfer. Six images from the test set are then input into the UAV-to-satellite domain generator for style transfer. The results before and after style transfer are shown below. Figure 4 As shown.

[0087] The drone images captured by the drone are cropped into squares, adjusted to a fixed size that the generator can accept, and then input into the generator to generate images that have undergone style transfer from drone style to satellite style.

[0088] In summary, this application's embodiments utilize ground images captured by a UAV, then extract corresponding satellite images based on the UAV image's GPS and yaw angle information to create a dataset. A recurrent generative adversarial network (RGAN) is then used to achieve image escaping between the two image domains. Finally, a generator extracting the UAV-to-satellite domain is used for style transfer. In creating the dataset, this application builds upon previous methods that used UAV image GPS information and then further extracts corresponding satellite images based on yaw angle information.

[0089] The results obtained from training and testing based on this dataset are shown above. Figure 4 As shown, the semantic information in the image was not destroyed. Comparative experiment. Figure 5is the experimental effect of not rotating the satellite large map based on the yaw angle to directly intercept the satellite map to make the data set. The buildings or other facilities in the image information of the satellite image directly intercepted without rotation are mostly east-west, which makes the buildings or other facilities with different orientations in the UAV photo tend to east-west after style migration, directly changing the semantic information of the UAV map and causing serious overfitting problem. Comparison can show the effect of rotating and intercepting the satellite image based on the yaw angle information in this method.

Claims

1. A method for unmanned image style transfer based on recurrent adversarial generative networks, characterized in that, The method includes the following steps: Step 1: Use a drone to take vertical images of the ground; Step 2: Based on the GPS information and yaw angle information corresponding to the ground image taken by the drone, crop the satellite image on the satellite map to obtain the cropped satellite image; Step 3: Save the UAV images to the UAV domain and the cropped satellite images to the satellite domain to complete the dataset creation; Step 4: Use the dataset to train a recurrent adversarial generative network to complete image style transfer between the two domains; Step 5: Extract the generator from the UAV domain to the satellite domain in the recurrent adversarial generative network for style transfer of other UAV real-time captured images; Step 2 specifically involves: reading the GPS and yaw angle of the image captured in the exif and xmp regions of the UAV image attributes using API functions, extracting satellite images with the same GPS from the satellite map, and rotating them using the yaw angle; Step 3 specifically involves: creating a dataset for training the recurrent adversarial generative network; cropping the captured drone images by centering them, extracting a square with the shortest side as the side length, and resizing it to become a square image with fixed pixels; similarly, the cropped satellite images are also resized to become square images with fixed pixels; saving the drone images in the drone domain as drone real images, and saving the cropped satellite images in the satellite domain as satellite real images, thus completing the dataset creation; Step 4 specifically involves: The described recurrent adversarial generative network comprises four parts: a generator from the UAV domain to the satellite domain, a generator from the satellite domain to the UAV domain, a UAV domain discriminator, and a satellite domain discriminator, which are respectively: A generator that converts a drone domain image to a satellite domain image is used to receive a real drone image and generate a pseudo satellite image. A generator that converts satellite domain data to UAV domain data is used to receive a real satellite image and generate a pseudo UAV image. The drone domain discriminator is used to receive real drone images and fake drone images and to determine whether the input real drone image and the generated fake drone image are real or fake. The satellite domain discriminator is used to receive real satellite images and fake satellite images and to distinguish between the authenticity of the input real satellite images and the generated fake satellite images. The recurrent adversarial generative network is trained by setting the loss for both the UAV domain discriminator and the satellite domain discriminator according to the following settings, and then training and optimizing it with the goal of minimizing the loss: ； Where G and F represent generators for converting between the two domains, and Dx and Dy represent discriminators for the two domains, respectively. and It is the binary minimax game loss used in recurrent adversarial generative networks. To counteract the cycle consistency loss in generative networks, These are pre-set hyperparameters.

2. The unmanned image style transfer method based on recurrent adversarial generative networks as described in claim 1, characterized in that, Step 1 specifically involves: using a drone to take vertically downward images at multiple known altitudes above the ground, storing the captured image data on the drone's memory card, and extracting the ground image from the drone's perspective.

3. The unmanned image style transfer method based on recurrent adversarial generative networks as described in claim 1, characterized in that, Step 5 specifically involves: after training is completed, extracting the style transfer of the UAV-to-satellite domain generator in the recurrent adversarial generative network for real-time UAV-captured images. Specifically, a real-time UAV-captured image is cropped and resized to become a square image with fixed pixels, which is then input into the UAV-to-satellite domain generator. After convolution and deconvolution operations, a satellite-style image based on the UAV image is generated and output.

Images