A remote sensing image generation method and device with controllable sway angle

Abstract

The present invention discloses a method and device for generating remote sensing images with controllable roll angles, the method comprising obtaining a remote sensing information set; using the remote sensing information set to pre-train a preset roll angle classification model to obtain an optimized roll angle classification model; constructing a fixed embedded information set based on the optimized roll angle classification model and the remote sensing information set; constructing an optimized generation model using the optimized roll angle classification model, a preset generation model and the remote sensing information set; and generating controllable generation images using the fixed embedded information set and the optimized generation model. The method of the present invention can controllably generate remote sensing images with different roll angles, effectively increasing the richness of remote sensing image data.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image generation, and in particular relates to a remote sensing image generation method and device with controllable sway angle. Background Art

[0002] Remote sensing image generation refers to the use of computer-generated simulated or synthetic remote sensing images. These methods can be used for a variety of applications, including remote sensing data enhancement, missing data filling, surface change analysis, and simulated sensor observations.

[0003] The roll angle refers to the deviation angle of the remote sensing sensor relative to the vertical direction. The sensor is usually installed on a spacecraft, aircraft or drone to obtain ground information by collecting images. When the sensor's line of sight is not perpendicular to the ground, the image roll angle will exist.

[0004] However, existing remote sensing image generation methods can only generate images with a single sway angle, which limits the application and performance of remote sensing image generation algorithms in various fields. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a remote sensing image generation method and device with controllable side swing angle, which can.

[0006] In order to solve the above technical problems, an embodiment of the present invention discloses a remote sensing image generation method with controllable roll angle, the method comprising:

[0007] S1. Acquire a remote sensing information set; the remote sensing information set includes a plurality of remote sensing information; the remote sensing information includes a remote sensing image, first category information and second category information;

[0008] S2. Pre-training a preset roll angle classification model using the remote sensing information set to obtain an optimized roll angle classification model; the roll angle classification model comprises an input module, a linear module and an output module connected in sequence; the linear module outputs an embedding vector;

[0009] S3, based on the optimized roll angle classification model and the remote sensing information set, construct a fixed embedded information set; the fixed embedded information set includes C fixed embedded information; C is the number of classifications of the roll angle classification model; the fixed embedded information includes a fixed embedded vector and the first category information;

[0010] S4, constructing an optimized generation model using the optimized roll angle classification model, the preset generation model and the remote sensing information set;

[0011] S5. Generate a controllable generated image using the fixed embedding information set and the optimized generation model.

[0012] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the construction of a fixed embedded information set based on the optimized roll angle classification model and the remote sensing information set includes:

[0013] S31, based on the first category information, classify the remote sensing information set to obtain C classified remote sensing information sets;

[0014] S32, respectively perform sampling processing on the C classified remote sensing information sets to obtain C sampled remote sensing information sets;

[0015] S33, using the optimized roll angle classification model, respectively processing the C sampled remote sensing information sets to obtain C embedded information sets; the embedded information sets include a plurality of embedded information; the embedded information includes an embedded vector and first category information;

[0016] S34. Calculate the mean of all the embedding vectors in each of the embedding information sets to obtain C fixed embedding vectors; combine the C fixed embedding vectors with the corresponding first category information respectively to obtain the fixed embedding information; the C fixed embedding information constitute the fixed embedding information set.

[0017] As an optional implementation, in the first aspect of the embodiment of the present invention, the generation model includes a first encoder module, a first summation module, a noise generation module, a second encoder module, an image generator module, an image controller module, a second summation module and a decoder module;

[0018] The input end of the first encoder module is configured as the first input end of the generation model; the output end of the first encoder module is connected to the input end A of the first summing module;

[0019] The output end of the noise generation module is connected to the input end of the second encoder module; the output end of the second encoder module is connected to the input end B of the first summing module and the input end G of the image generator module;

[0020] The output end of the first summing module is connected to the input end of the image controller module; the output end of the image controller module is connected to the input end C of the second summing module;

[0021] The input terminal E of the image generator module is configured as the second input terminal of the generation model; the input terminal F of the image generator module is configured as the third input terminal of the generation model; the output terminal of the image generator module is connected to the input terminal D of the second summation module;

[0022] The output end of the second summation module is connected to the input end of the decoder module; the output end of the decoder module is configured as the output end of the generation model to output the reconstructed image.

[0023] As an optional implementation, in the first aspect of the embodiment of the present invention, the first encoder module is obtained by copying the second encoder module; the image controller module is obtained by copying and reconfiguring the image generator module.

[0024] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the use of the optimized roll angle classification model, the preset generation model and the remote sensing information set to construct an optimized generation model includes:

[0025] S41, connecting the output end of the optimized side swing angle classification model to the second input end and the third input end of the generation model, and connecting the input end of the optimized side swing angle classification model to the first input end of the generation model, to obtain an intermediate generation model;

[0026] S42, constructing a training data set based on the remote sensing information set; the training data set includes a plurality of training data; the training data includes an original image and a generated image;

[0027] S43, using the training data set to minimize the loss function, training the intermediate generation model to obtain the trained intermediate generation model;

[0028] S44, deleting the optimized roll angle classification model from the trained intermediate generation model to obtain the optimized generation model.

[0029] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the loss function is:

[0030]

[0031] Wherein, Loss is the loss value of the loss function, ε(i,j) is the pixel value of the i-th row and j-th column of the generated image ε in the training data; ε θ (i, j) is the reconstructed image ε output by the intermediate generation model θ The pixel value of the i-th row and j-th column of ; M and N are the total number of rows and the total number of columns of the generated image respectively.

[0032] As an optional implementation manner, in the first aspect of the embodiment of the present invention, constructing a training data set based on the remote sensing information set includes:

[0033] S421, acquiring any remote sensing information in the remote sensing information set to obtain original remote sensing information; determining the original image, the first original category, and the second original category, which are the remote sensing image, the first category information, and the second category information of the original remote sensing information respectively;

[0034] S422, acquiring the remote sensing information in the remote sensing information set, wherein any of the first category information is different from the first original category, and the second category information is the same as the second original category, to obtain generated remote sensing information; determining the generated image to be the remote sensing image of the generated remote sensing information;

[0035] S423, combining the original image and the generated image to obtain the training data;

[0036] S424, repeat S times to execute S421 to S423 to obtain S training data; the S training data constitute the training data set; S is a preset sampling number.

[0037] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the generating a controllable generated image by using the fixed embedding information set and the optimized generation model includes:

[0038] S51, obtaining generation category information and initial remote sensing images;

[0039] S52, in the fixed embedding information set, searching for the fixed embedding vector corresponding to the first category information and the generated category information to obtain a constrained embedding vector;

[0040] S53: Using the optimized generation model, the initial remote sensing image and the constraint embedding vector are processed to obtain a controllable generated image.

[0041] A second aspect of an embodiment of the present invention discloses another remote sensing image generation device with controllable side swing angle, the device comprising:

[0042] A memory storing executable program code;

[0043] a processor coupled to the memory;

[0044] The processor calls the executable program code stored in the memory to execute part or all of the steps in the method for generating remote sensing images with controllable roll angle disclosed in the first aspect of the embodiment of the present invention.

[0045] The third aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the remote sensing image generation method with controllable side swing angle disclosed in the first aspect of an embodiment of the present invention.

[0046] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0047] The present invention uses the acquired remote sensing information set to pre-train the preset roll angle classification model to obtain the optimized roll angle classification model; based on the optimized roll angle classification model and the remote sensing information set, a fixed embedded information set is constructed; using the optimized roll angle classification model, the preset generation model and the remote sensing information set, an optimized generation model is constructed; using the fixed embedded information set and the optimized generation model, a controllable generation image is generated. The present invention can controllably generate remote sensing images of different roll angles, effectively increase the richness of remote sensing image data, and provide a basis for further application of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 It is a flow chart of a method for generating a remote sensing image with controllable roll angle disclosed in an embodiment of the present invention.

[0050] Figure 2 It is a structural schematic diagram of a generation model of a remote sensing image generation method with controllable roll angle disclosed in an embodiment of the present invention.

[0051] Figure 3 It is a structural schematic diagram of a remote sensing image generating device with controllable lateral swing angle disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0054] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0055] Embodiment 1

[0056] See also Figure 1 , Figure 1 : is a flow chart of a method for generating a remote sensing image with controllable side swing angle disclosed in an embodiment of the present invention. Figure 1 The described method for generating remote sensing images with controllable roll angle is applied to the field of remote sensing image generation, such as remote sensing image generation for tasks such as environmental monitoring, disaster warning, and map surveying and mapping, and is not limited in the embodiments of the present invention. Figure 1 As shown, the method includes:

[0057] S1. Acquire a remote sensing information set; the remote sensing information set includes a plurality of remote sensing information; the remote sensing information includes a remote sensing image, first category information and second category information.

[0058] It should be noted that the first category information is used to indicate the categories corresponding to the numerical range of the roll angle after being divided into smaller numerical ranges that vary continuously. For example, the roll angle range of 0 to 90° can be divided into 9 categories at intervals of 10°, and the possible value of the first category information is any integer from 1 to 9, which is not limited in the embodiment of the present invention.

[0059] It should be noted that the second category information is used to identify the corresponding detection area of ​​the remote sensing image, and only remote sensing images acquired for the same detection area have the same corresponding second category information. Different detection areas can be numbered with different integers, and the number is the corresponding second category information, which is not limited in the embodiment of the present invention.

[0060] S2. Pre-training a preset roll angle classification model using the remote sensing information set to obtain an optimized roll angle classification model; the roll angle classification model comprises an input module, a linear module and an output module connected in sequence; the linear module outputs an embedding vector.

[0061] It should be noted that the above-mentioned input module is used to sequentially expand the channels, rearrange and normalize the input remote sensing image to obtain an image tensor; for example, a 3-channel remote sensing image with a size of 512×512×3 is transformed into an image tensor of 1×3×512×512 after being processed by the input module, which is not limited in the embodiments of the present invention.

[0062] It should be noted that the above-mentioned linear module is constructed based on the fully connected layer, which is composed of a series of weight matrices and bias vectors, and is used to perform linear transformation on the image tensor to obtain an embedding vector, which is not limited in the embodiment of the present invention.

[0063] It should be noted that the above-mentioned output layer is constructed based on the softmax function, which processes the input embedding vector and outputs a probability vector. Each element value in the probability vector represents the probability value of the corresponding roll angle category. The length of the probability vector is equal to the number of possible values ​​of the first category information, which is not limited in the embodiment of the present invention.

[0064] It should be noted that all remote sensing images in the above remote sensing information set are used as input data for pre-training of the side swing angle classification model.

[0065] Optionally, the pre-training of the above-mentioned sway angle classification model adopts the following cross entropy loss function:

[0066]

[0067] In the formula, CELoss is the cross entropy loss value, p k is the kth element of the probability vector, y k For p k The corresponding label value; where the first category information corresponding to the input remote sensing image is category ii, then the label value y ii The value of the other labels is 0.

[0068] S3. Based on the optimized roll angle classification model and the remote sensing information set, a fixed embedded information set is constructed; the fixed embedded information set includes C fixed embedded information; C is the number of classifications of the roll angle classification model; the fixed embedded information includes a fixed embedded vector and the first category information.

[0069] S4. Constructing an optimized generation model using the optimized roll angle classification model, the preset generation model and the remote sensing information set.

[0070] S5. Generate a controllable generated image using the fixed embedding information set and the optimized generation model.

[0071] In an optional embodiment, the above-mentioned construction of a fixed embedded information set based on the optimized roll angle classification model and the remote sensing information set includes:

[0072] S31. Based on the first category information, classify the remote sensing information set to obtain C classified remote sensing information sets.

[0073] It should be noted that different classified remote sensing information sets correspond to different first category information.

[0074] S32. Perform sampling processing on the C classified remote sensing information sets respectively to obtain C sampled remote sensing information sets.

[0075] It should be noted that the above sampling process is to sample each classified remote sensing information set P times, and each sampling obtains any remote sensing information; the P remote sensing information obtained by sampling each classified remote sensing information set constitutes the corresponding sampled remote sensing information set. The above P can be 1000, which is not limited in the embodiment of the present invention.

[0076] S33. Using the optimized roll angle classification model, the C sampled remote sensing information sets are processed respectively to obtain C embedded information sets; the embedded information sets include a plurality of embedded information; the embedded information includes an embedded vector and first category information.

[0077] It should be noted that, after each remote sensing image in the sampled remote sensing information set is processed by the optimized roll angle classification model, the linear module outputs the corresponding embedding vector, and the embedding vector is combined with the corresponding first category information to obtain the embedded information; the embedded information of all remote sensing images in each sampled remote sensing image set together constitutes the corresponding embedded information set. In addition, the first category information in each embedded information set is the same.

[0078] S34. Calculate the mean of all the embedding vectors in each of the embedding information sets to obtain C fixed embedding vectors; combine the C fixed embedding vectors with the corresponding first category information respectively to obtain the fixed embedding information; the C fixed embedding information constitute the fixed embedding information set.

[0079] In another optional embodiment, Figure 2 As shown, the above generation model includes a first encoder module, a first summing module, a noise generating module, a second encoder module, an image generator module, an image controller module, a second summing module and a decoder module;

[0080] The input end of the first encoder module is configured as the first input end of the generation model; the output end of the first encoder module is connected to the input end A of the first summing module;

[0081] The output end of the noise generation module is connected to the input end of the second encoder module; the output end of the second encoder module is connected to the input end B of the first summing module and the input end G of the image generator module;

[0082] The output end of the first summing module is connected to the input end of the image controller module; the output end of the image controller module is connected to the input end C of the second summing module;

[0083] The input terminal E of the image generator module is configured as the second input terminal of the generation model; the input terminal F of the image generator module is configured as the third input terminal of the generation model; the output terminal of the image generator module is connected to the input terminal D of the second summation module;

[0084] The output end of the second summation module is connected to the input end of the decoder module; the output end of the decoder module is configured as the output end of the generation model to output the reconstructed image.

[0085] It should be noted that the above-mentioned noise generation module is used to generate random noise with the same size as the remote sensing image input to the first input end of the generation model, and the noise conforms to the standard normal distribution, which is not limited in the embodiment of the present invention.

[0086] It should be noted that the first summing module is used to perform a summing operation on the input data of the input terminal A and the input terminal B; and the second summing module is used to perform a summing operation on the input data of the input terminal C and the input terminal D.

[0087] It should be noted that the above-mentioned image generator module and image controller module are both constructed based on U-Net, which is not limited in the embodiments of the present invention. U-Net is a fully convolutional neural network (FCN), which was originally proposed by Ronneberger et al. in 2015 and is widely used in image segmentation tasks, especially in the field of medical image processing. The architecture of U-Net is symmetrical and consists of an encoder and a decoder. The encoder of U-Net extracts the features of the input image by stacking multiple convolutional layers and pooling layers, and gradually reduces the spatial dimension; the decoder of U-Net restores the feature map obtained by the encoder to the size of the original input image through upsampling (deconvolution) operations. Skip connections are used to splice the features of each layer in the encoder of U-Net with the features of the corresponding layer of the decoder of U-Net to obtain richer semantic information. At the same time, by introducing a cross-attention mechanism in the skip connection, feature information from different levels can be effectively fused, thereby improving the accuracy of the decoder in restoring spatial information. The preset image generator module may be a stable diffusion model based on U-Net, which is not limited in the embodiment of the present invention.

[0088] It should be noted that the above-mentioned second encoder module and decoder module are constructed based on VQ-VAE, wherein the second encoder module corresponds to the encoder of VQ-VAE, and the decoder module corresponds to the vector quantizer and decoder of VQ-VAE, which is not limited in the embodiment of the present invention. VQ-VAE, full name Vector Quantized Variational Autoencoder, is a generative model in deep learning, especially good at processing high-dimensional data such as images and audio. It combines autoencoder (Autoencoder) and vector quantization (Vector Quantization) technology to generate discrete representations for continuous data, thus having broad application potential in multiple fields.

[0089] In yet another optional embodiment, the first encoder module is obtained by copying the second encoder module; and the image controller module is obtained by copying and reconfiguring the image generator module.

[0090] It should be noted that in the above-mentioned image generator module, the Q input of each jump-connected cross-attention unit is connected to the output of the layer of the encoder of the U-Net corresponding to the jump connection, and the K input of all cross-attention units is configured as the input E of the image generator module, and the V input of all cross-attention units is configured as the input F of the image generator module; therefore, the data input to the K input of all cross-attention units is the same, and the data input to the V input of all cross-attention units is the same; in addition, after the image generator module is copied to obtain the image controller module, it is also necessary to reconfigure, that is, in the image controller module, the K input and V input of each jump-connected cross-attention unit are no longer configured as the input E and input F of the image controller module, but are also connected to the output of the layer of the encoder of the U-Net corresponding to the jump connection, that is, the input data of the Q input, K input and V input of the same cross-attention unit are the same. In addition, the input of the image controller module, corresponding to the input G of the image generator module, is the input of the encoder of the U-Net.

[0091] In another optional embodiment, the optimized roll angle classification model, the preset generation model and the remote sensing information set are used to construct an optimized generation model, including:

[0092] S41. Connect the output end of the optimized side swing angle classification model to the second input end and the third input end of the generation model, and connect the input end of the optimized side swing angle classification model to the first input end of the generation model to obtain an intermediate generation model.

[0093] S42. Constructing a training data set based on the remote sensing information set; the training data set includes a plurality of training data; the training data includes an original image and a generated image.

[0094] S43. Using the training data set, the intermediate generation model is trained with the goal of minimizing the loss function to obtain the trained intermediate generation model.

[0095] It should be noted that during the training process of the above-mentioned intermediate generation model, the parameters of the optimized roll angle classification model, the first encoder module, the second encoder module, the image generator module and the decoder module remain unchanged, and only the parameters of the image controller need to be updated, thereby effectively reducing the computational complexity of the training process.

[0096] It should be noted that the training of the intermediate generation model may adopt the gradient descent method, which is not limited in the embodiment of the present invention.

[0097] S44, deleting the optimized roll angle classification model from the trained intermediate generation model to obtain the optimized generation model.

[0098] In another optional embodiment, the above loss function is:

[0099]

[0100] Wherein, Loss is the loss value of the loss function, ε(i,j) is the pixel value of the i-th row and j-th column of the generated image ε in the training data; ε θ (i, j) is the reconstructed image ε output by the intermediate generation model θ The pixel value of the i-th row and j-th column of ; M and N are the total number of rows and the total number of columns of the generated image respectively.

[0101] It should be noted that the original image corresponding to the above generated image ε is processed by the intermediate generation model to output the reconstructed image ε θ In addition, the sizes of the original image, generated image, and reconstructed image are exactly the same.

[0102] In yet another optional embodiment, the above-mentioned constructing a training data set based on the remote sensing information set includes:

[0103] S421, acquiring any remote sensing information in the remote sensing information set to obtain original remote sensing information; determining the original image, the first original category, and the second original category, which are the remote sensing image, the first category information, and the second category information of the original remote sensing information respectively;

[0104] S422, acquiring the remote sensing information in the remote sensing information set, wherein any of the first category information is different from the first original category, and the second category information is the same as the second original category, to obtain generated remote sensing information; determining the generated image to be the remote sensing image of the generated remote sensing information;

[0105] S423, combining the original image and the generated image to obtain the training data;

[0106] S424, repeat S times to execute S421 to S423 to obtain S training data; the S training data constitute the training data set; S is a preset sampling number.

[0107] In yet another optional embodiment, the above-mentioned generation of a controllable generated image using the fixed embedded information set and the optimized generation model includes:

[0108] S51, obtaining generation category information and initial remote sensing images;

[0109] S52, in the fixed embedding information set, searching for the fixed embedding vector corresponding to the first category information and the generated category information to obtain a constrained embedding vector;

[0110] S53: Using the optimized generation model, the initial remote sensing image and the constraint embedding vector are processed to obtain a controllable generated image.

[0111] It should be noted that the above-mentioned initial remote sensing image is input into the first input terminal of the optimization generation model, and the constraint embedding vector is simultaneously input into the second input terminal and the third input terminal of the optimization generation model.

[0112] It can be seen that the method for generating remote sensing images with controllable roll angles described in the embodiments of the present invention can controllably generate remote sensing images with different roll angles, thereby effectively increasing the richness of remote sensing image data.

[0113] Embodiment 2

[0114] See also Figure 3 , Figure 3 is a schematic diagram of the structure of another remote sensing image generation device with controllable side swing angle disclosed in an embodiment of the present invention. Figure 3 The described remote sensing image generation device with controllable side swing angle can be applied to the field of remote sensing image generation, such as remote sensing image generation for tasks such as environmental monitoring, disaster warning, and map surveying and mapping, and the embodiments of the present invention do not limit this. Figure 3 As shown, the remote sensing image generation device with controllable side swing angle may include the following parts:

[0115] A memory 201 storing executable program codes;

[0116] a processor 202 coupled to the memory 201;

[0117] The processor 202 calls the executable program code stored in the memory 201 to execute the steps in the remote sensing image generation method with controllable roll angle described in the first embodiment.

[0118] Embodiment 3

[0119] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the method for generating a remote sensing image with controllable roll angle described in the first embodiment.

[0120] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0121] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, a magnetic disk storage, a magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0122] Finally, it should be noted that the method and device for generating remote sensing images with controllable side swing angle disclosed in the embodiment of the present invention disclose only the preferred embodiments of the present invention, which are only used to illustrate the technical scheme of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical schemes described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical schemes from the spirit and scope of the technical schemes of the embodiments of the present invention.

Claims

1. A remote sensing image generation method with controllable roll angle, characterized in that: include: S1. Acquire a remote sensing information set; the remote sensing information set includes a plurality of remote sensing information; the remote sensing information includes a remote sensing image, first category information and second category information; S2. Pre-training a preset roll angle classification model using the remote sensing information set to obtain an optimized roll angle classification model; the roll angle classification model comprises an input module, a linear module and an output module connected in sequence; the linear module outputs an embedding vector; S3. Based on the first category information, classify the remote sensing information set to obtain C classified remote sensing information sets; S4, respectively sampling and processing the C classified remote sensing information sets to obtain C sampled remote sensing information sets; S5, using the optimized roll angle classification model, respectively processing the C sampled remote sensing information sets to obtain C embedded information sets; the embedded information sets include a plurality of embedded information; the embedded information includes an embedded vector and first category information; S6, calculating the mean of all the embedding vectors in each embedding information set to obtain C fixed embedding vectors; combining the C fixed embedding vectors with the corresponding first category information respectively to obtain fixed embedding information; the C fixed embedding information constitutes the fixed embedding information set; S7, constructing an optimized generation model using the optimized roll angle classification model, the preset generation model and the remote sensing information set; The generation model comprises a first encoder module, a first summing module, a noise generating module, a second encoder module, an image generator module, an image controller module, a second summing module and a decoder module; the first encoder module is obtained by copying the second encoder module; the image controller module is obtained by copying and reconfiguring the image generator module; The input end of the first encoder module is configured as the first input end of the generation model; the output end of the first encoder module is connected to the input end A of the first summing module; The output end of the noise generation module is connected to the input end of the second encoder module; the output end of the second encoder module is connected to the input end B of the first summing module and the input end G of the image generator module; The output end of the first summing module is connected to the input end of the image controller module; the output end of the image controller module is connected to the input end C of the second summing module; The input terminal E of the image generator module is configured as the second input terminal of the generation model; the input terminal F of the image generator module is configured as the third input terminal of the generation model; the output terminal of the image generator module is connected to the input terminal D of the second summation module; The output end of the second summing module is connected to the input end of the decoder module; the output end of the decoder module is configured as the output end of the generation model to output the reconstructed image; S8. Generate a controllable generated image using the fixed embedding information set and the optimized generation model.

2. The method for generating remote sensing images with controllable side swing angle according to claim 1, characterized in that: The optimized roll angle classification model, the preset generation model and the remote sensing information set are used to construct an optimized generation model, including: S71, connecting the output end of the optimized side swing angle classification model to the second input end and the third input end of the generation model, and connecting the input end of the optimized side swing angle classification model to the first input end of the generation model, to obtain an intermediate generation model; S72, constructing a training data set based on the remote sensing information set; the training data set includes a plurality of training data; the training data includes an original image and a generated image; S73, using the training data set to minimize the loss function, train the intermediate generation model to obtain the trained intermediate generation model; S74. Delete the optimized roll angle classification model from the trained intermediate generation model to obtain the optimized generation model.

3. The method for generating remote sensing images with controllable side swing angle according to claim 2, characterized in that: The loss function is: Wherein, Loss is the loss value of the loss function, ε(i,j) is the pixel value of the i-th row and j-th column of the generated image ε in the training data; ε θ (i, j) is the reconstructed image ε output by the intermediate generation model θ The pixel value of the i-th row and j-th column of ; M and N are the total number of rows and the total number of columns of the generated image respectively.

4. The method for generating remote sensing images with controllable side swing angle according to claim 2, characterized in that: The step of constructing a training data set based on the remote sensing information set includes: S721, acquiring any remote sensing information in the remote sensing information set to obtain original remote sensing information; Determine the original image, the first original category, and the second original category, which are respectively the remote sensing image, the first category information, and the second category information of the original remote sensing information; S722, acquiring the remote sensing information in the remote sensing information set, wherein any of the first category information is different from the first original category, and the second category information is the same as the second original category, to obtain generated remote sensing information; Determine the generated image as the remote sensing image for generating remote sensing information; S723, combining the original image and the generated image to obtain the training data; S724, repeat S times to execute S721 to S723 to obtain S training data; the S training data constitute the training data set; S is a preset sampling number.

5. The method for generating remote sensing images with controllable roll angle according to claim 1, characterized in that: The step of generating a controllable generated image by using the fixed embedded information set and the optimized generation model includes: S81, obtaining generation category information and initial remote sensing images; S82, in the fixed embedding information set, searching for the fixed embedding vector corresponding to the first category information and the generated category information to obtain a constrained embedding vector; S83. Using the optimized generation model, the initial remote sensing image and the constraint embedding vector are processed to obtain a controllable generated image.

6. A remote sensing image generation device with controllable side swing angle, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the remote sensing image generation method with controllable roll angle as described in any one of claims 1 to 5.

7. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the method for generating remote sensing images with controllable roll angle as described in any one of claims 1 to 5.

Images