Remote sensing image processing method and device, electronic equipment and medium

Abstract

The present disclosure provides a remote sensing image processing method and device, electronic equipment and medium, and the specific implementation scheme comprises: obtaining a to-be-processed image and a reference style image; performing de-stylization processing on the reference style image to obtain a restored image; determining a style mapping relationship between the restored image and the reference image according to the reference style image and the restored image; performing style migration on the to-be-processed image according to the style mapping relationship to obtain a target image, and the target image has the image style of the reference style image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a remote sensing image processing method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0002] Currently, due to differences in imaging conditions such as imaging time, lighting, shooting angle, and land cover type, significant differences in color and brightness often exist between images. Therefore, remote sensing images need to be processed to facilitate their subsequent production and use. To facilitate the subsequent production and use of remote sensing images, processing is typically required. Summary of the Invention

[0003] This disclosure provides a remote sensing image processing method, apparatus, electronic device, storage medium, and computer program product.

[0004] According to one aspect of this disclosure, a remote sensing image processing method is provided, comprising: acquiring an image to be processed and a reference style image; performing destylating processing on the reference style image to obtain a restored image; determining a style mapping relationship between the restored image and the reference image based on the reference style image and the restored image; and performing style transfer on the image to be processed according to the style mapping relationship to obtain a target image, wherein the target image has the image style of the reference style image.

[0005] According to another aspect of this disclosure, a remote sensing image processing apparatus is provided, comprising: an acquisition module for acquiring an image to be processed and a reference style image; a processing module for performing destylating processing on the reference style image to obtain a restored image; a determination module for determining a style mapping relationship between the restored image and the reference image based on the reference style image and the restored image; and a style transfer module for performing style transfer on the image to be processed according to the style mapping relationship to obtain a target image, wherein the target image has the image style of the reference style image.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform a remote sensing image processing method as described above.

[0007] According to another aspect of this disclosure, a computer-readable storage medium is provided having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the remote sensing image processing method as described above.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the remote sensing image processing method as described above. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0010] Figure 1 This is a flowchart of a remote sensing image processing method according to an embodiment of the present disclosure;

[0011] Figure 2 This is a block diagram of a remote sensing image processing apparatus according to embodiments of the present disclosure;

[0012] Figure 3 This is a block diagram of an electronic device suitable for implementing a remote sensing image processing method according to embodiments of the present disclosure. Detailed Implementation

[0013] The technical solutions of the present disclosure will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure are within the scope of protection of the present disclosure.

[0014] It should be noted that the sequence numbers of each operation in the following methods are for descriptive purposes only and should not be construed as indicating the execution order of the operations. Unless explicitly stated otherwise, the method does not need to be executed exactly in the order shown. Furthermore, in the description of this disclosure, the terms "first," "second," etc. (if present) are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0015] The technical solutions disclosed herein involve data collection, storage, use, processing, transmission, provision, disclosure, and application, all of which comply with the relevant laws and regulations of the relevant countries and regions and do not violate public order and good morals. Furthermore, in the technical solutions disclosed herein, authorization or consent from the data owner is obtained before acquiring or collecting any relevant data.

[0016] Figure 1 This is a flowchart of a remote sensing image processing method according to an embodiment of the present disclosure.

[0017] like Figure 1As shown, the remote sensing image processing method 100 includes operations S110 to S140.

[0018] In operation S110, acquire the image to be processed and the reference style image.

[0019] In operation S120, the reference style image is destylated to obtain the restored image.

[0020] In operation S130, the style mapping relationship between the restored image and the reference image is determined based on the reference style image and the restored image.

[0021] In operation S140, style transfer is performed on the image to be processed according to the style mapping relationship to obtain the target image, which has the image style of the reference style image.

[0022] The image to be processed is a remote sensing image undergoing style transfer. The image includes red, green, and blue band data. It contains at least one object, such as a building or body of water.

[0023] The reference style image is a remote sensing image with a reference style. The reference style may include, for example, the brightness, contrast, and color of the reference style image. The reference style image includes at least one reference object, which may be the same as, partially the same as, or completely different from at least one object contained in the image to be processed. For example, the at least one reference object may include grassland, woodland, etc. The reference style image includes red band data, green band data, and blue band data. The reference style image and the image to be processed typically have the same bit depth, for example, both 8 bits / channel. In this embodiment of the disclosure, the reference style image can be selected according to actual needs, and no limitation is made here.

[0024] In embodiments of this disclosure, style transfer of the image to be processed can be performed based on the reference style of a reference style image, so that the image style of the image to be processed is consistent with the image style of the reference style image. Specifically, by performing destylating processing on the reference style image, a restored image corresponding to the reference style image is recovered from the reference style image. This restored image is a remote sensing image with the same content as the reference style image but without the application of the reference style. Then, based on the reference style image and the restored image, a style mapping relationship is determined between them, which is used to characterize the mapping from the restored image to the reference style image. Furthermore, style transfer of the image to be processed is performed according to this style mapping relationship to obtain a target image, which is a remote sensing image with the same content as the image to be processed and has the image style of the reference style image. Based on the technical solution of this disclosure, both rapid style transfer and good style transfer effect can be achieved.

[0025] According to embodiments of this disclosure, a destylating network can be used to destylate a reference style image to obtain a restored image. This destylating network is obtained by training an initial destylating network using multiple sample images. The trained destylating network is able to recover the original style-removed image from a styled reference image. The training process of this destylating network is described in detail below.

[0026] First, obtain multiple sample image pairs.

[0027] According to embodiments of this disclosure, each sample image pair includes an original sample image and a reference sample image corresponding to the original sample image. The original sample image is a content image without any style applied, while the reference sample image is an image obtained by applying a sample style to the original sample image. The specific sample style can be set according to actual conditions, and this disclosure does not limit it. Both the original sample image and the reference sample image include red band data, green band data, and blue band data. Both the original sample image and the reference sample image include at least one sample object, such as a building or road.

[0028] Next, for each pair of sample images, the reference sample image in the pair is destylated using the initial destylization network to obtain the restored image.

[0029] According to embodiments of this disclosure, the initial destylating network can, for example, employ a deep residual convolutional autoencoder network. This deep residual convolutional autoencoder network includes an encoder, a residual module, and a decoder. The encoder includes multiple downsampling convolutional layers for performing feature extraction and spatial downsampling on the input reference sample image to obtain a downsampling feature map. The residual module is located between the encoder and the decoder and includes multiple stacked residual blocks. Each residual block includes a first convolutional layer, a first batch normalization layer, a ReLU activation layer, a second convolutional layer, a second batch normalization layer, and skip connections connected in sequence. The first and second convolutional layers can, for example, both be 3x3 convolutional layers, and the skip connections are used to element-wise add the input of the residual block to the output of the second batch normalization layer. Each residual block is used to implement nonlinear feature transformation. The decoder includes multiple upsampling convolutional layers for receiving the feature map output by the residual module and restoring the spatial resolution of the image through progressive upsampling, ultimately outputting a restored image corresponding to the reference sample image.

[0030] Next, based on the pixel value differences between each original sample image and the corresponding restored image, the parameters of the initial destylization network are adjusted to obtain the destylization network.

[0031] According to embodiments of this disclosure, sample loss can be determined based on the difference between the pixel value of each pixel in each recovered image and the pixel value of the corresponding pixel in the corresponding original sample image. The parameters of the initial destylating network are then adjusted based on the sample loss until the network converges, thereby obtaining the destylating network. This destylating network can be used to destylate a reference style image with an applied style, that is, to recover an original image with the same content but without any applied style from the reference style image with an applied style.

[0032] According to an embodiment of this disclosure, in the above operation S130, determining the style mapping relationship between the restored image and the reference image based on the reference style image and the restored image includes the following steps.

[0033] First, determine the weight corresponding to each pixel in the restored image.

[0034] As mentioned earlier, the restored image is obtained by destylating the reference style image using a destylating network. Since the destylating network may have varying restoration accuracy for different pixels, the restored pixel values ​​in the restored image have different levels of confidence. To avoid interference from unreliable pixels in the restored image on style parameter estimation during the subsequent construction of the objective function, which would affect the accuracy of the style mapping relationship between the reference style image and the restored image, it is necessary to quantify the confidence level of the restored pixel value for each pixel in the restored image.

[0035] In this embodiment, the weight corresponding to each pixel in the restored image can be used to characterize the reliability of the restored pixel value. For example, for a pixel in the restored image, the larger the weight corresponding to the pixel, the higher the reliability of the restored pixel value; conversely, the smaller the weight corresponding to the pixel, the lower the reliability of the restored pixel value. By introducing this weight to evaluate the reliability of the restored pixel values ​​of each pixel in the restored image, this embodiment reduces the influence of unreliable pixels when constructing the objective function, thereby improving the accuracy and robustness of style parameter estimation, and ultimately improving the accuracy and reliability of style mapping relationships.

[0036] According to embodiments of this disclosure, the weight corresponding to each pixel in the restored image is calculated based on the uncertainty value of that pixel. The weight is negatively correlated with the uncertainty value. The process of obtaining the weight corresponding to each pixel in the restored image is described below.

[0037] First, determine the uncertainty value corresponding to each pixel in the restored image.

[0038] In this embodiment of the disclosure, the uncertainty value corresponding to each pixel in the recovered image includes a first uncertainty value and a second uncertainty value.

[0039] The first uncertainty value reflects the category bias variance of the semantic category to which the pixel belongs. Category bias variance refers to the statistical variance of the color deviation of all pixels belonging to the same semantic category in multiple original sample images, used to measure the dispersion of color fluctuations within that semantic category. A larger category bias variance indicates a more dispersed color distribution of that semantic category across multiple original sample images, leading to higher uncertainty in the destylating network's recovery of pixels of that category. This results in lower reliability of the destylating network's recovery of pixels of that category, and pixels of that category should be assigned lower weights to reduce their interference with style parameter estimation. Conversely, a smaller category bias variance indicates higher reliability of the destylating network's recovery of pixels of that category, and pixels of that category should be assigned higher weights.

[0040] The second uncertainty value reflects the degree of confidence the destylating network has in the pixel restoration results. Specifically, the destylating network, which retains the Dropout layer, can perform multiple inferences on the reference style image. If the restored pixel values ​​obtained from multiple inferences for a certain pixel location differ significantly (e.g., large variance), it indicates that the destylating network has a low degree of confidence in the restoration results for that pixel location. Consequently, the second uncertainty value for that pixel location is high, and that pixel location should be assigned a lower weight. Conversely, if the restored pixel values ​​obtained from multiple inferences for that pixel location differ significantly (e.g., small variance), it indicates that the destylating network has a high degree of confidence in the restoration results for that pixel location. Consequently, the second uncertainty value for that pixel location is small, and that pixel location should be assigned a higher weight.

[0041] The process of determining the uncertainty value corresponding to each pixel in the restored image is explained below.

[0042] First, obtain the preset semantic prior data.

[0043] According to embodiments of this disclosure, semantic prior data is prior information obtained through semantic segmentation and statistical analysis of multiple original sample images. This semantic prior data includes the class bias variances corresponding to multiple semantic categories in the multiple original sample images.

[0044] According to embodiments of this disclosure, semantic prior data is obtained as follows: First, the semantic segmentation result corresponding to each original sample image is obtained, and based on the semantic segmentation result corresponding to each original sample image, pixels belonging to the same semantic category in multiple original sample images are determined. Then, based on the pixel values ​​of pixels belonging to the same semantic category in multiple original sample images, the mean category color corresponding to each semantic category is determined. Next, based on the pixel values ​​of pixels belonging to the same semantic category in multiple original sample images and the mean category color corresponding to each semantic category, the category bias variance corresponding to each semantic category in multiple original sample images is determined.

[0045] For example, for each semantic category, the mean of the category color and the variance of the category bias can be calculated using the following formula.

[0046]

[0047]

[0048] In the above formula, cls represents any semantic category among multiple original sample images, and S cls It is a set of pixels that belong to the same semantic category cls in multiple original sample images. For pixel set S cls The number of pixels in S, where c represents the set of pixels S. cls Any pixel in Let c be the pixel value of the corresponding original sample image. This represents the mean color of the category corresponding to the semantic category cls. This represents the category bias variance corresponding to the semantic category cls. This represents the deviation between the pixel value of pixel c in the corresponding original sample image and the mean color of its semantic category.

[0049] Next, for each pixel in the restored image, based on the target semantic category to which the corresponding pixel in the reference style image belongs, the target category deviation variance corresponding to the target semantic category is queried from the semantic prior data, and the target category deviation variance is used as the first uncertainty value of the pixel.

[0050] Understandably, although the reference-style image and the reconstructed image have different styles, their semantic content is the same. Therefore, the same pixel locations in both images have the same semantic category. Furthermore, compared to the reconstructed image, the reference-style image, as the input image to the destylating network, has semantic content that has not undergone network reconstruction. Therefore, it can more accurately and stably reflect the semantic information of the original scene, avoiding semantic segmentation errors caused by suboptimal reconstruction by the destylating network. Based on these reasons, for each pixel in the reconstructed image, its semantic category can be determined based on the target semantic category of its corresponding pixel in the reference-style image. Then, based on this semantic category, a first uncertainty value corresponding to that pixel can be determined from the semantic prior data.

[0051] According to embodiments of this disclosure, semantic segmentation results can be obtained by performing semantic segmentation on a reference style image. Based on the semantic segmentation results, the target semantic category to which each pixel in the reference style image belongs can be determined. Then, for each pixel in the reconstructed image, based on the target semantic category to which the corresponding pixel in the reference style image belongs, the target category deviation variance corresponding to the target semantic category is queried from the semantic prior data, and the target category deviation variance is used as the first uncertainty value corresponding to the pixel.

[0052] Next, multiple restored images are obtained by inferring multiple times from the reference style image. For each pixel position in the restored image, the second uncertainty value corresponding to the pixel position is determined according to the restored pixel value corresponding to the pixel position in the multiple restored images.

[0053] In this embodiment, the MC Dropout method can be used to estimate the second uncertainty value corresponding to each pixel location in the restored image. This method retains the Dropout layer during the model inference phase, obtains multiple outputs corresponding to the same input through multiple forward propagations, and then calculates the statistical distribution (e.g., variance) of the output results to measure the model's uncertainty. If the model's restoration result for a certain pixel location in the input image differs significantly across different inference iterations (e.g., large variance), it indicates that the model's restoration result for that pixel location is not sufficiently certain. Accordingly, the second uncertainty value corresponding to that pixel location is high, and the restoration result corresponding to that pixel location should be assigned a lower weight.

[0054] Specifically, during the inference phase, a destylating network that retains the Dropout layer performs T forward propagation inferences on the reference style image, resulting in T reconstructed images, where T is an integer greater than 1. For example, T might be 10. Since the Dropout layer randomly discards different neurons during each forward propagation, the destylating network generates T slightly different reconstructed images. For each pixel location in the reconstructed image, the corresponding reconstructed pixel value in each of the T reconstructed images is obtained, resulting in T reconstructed pixel values. Then, the variance of these T reconstructed pixel values ​​is calculated, and this variance is used as the second uncertainty value corresponding to that pixel location.

[0055] Understandably, if the restoration result of a certain pixel location in the reference style image by the destylization network differs greatly in T inferences (e.g., the variance is large), it indicates that the restoration result of the destylization network for that pixel location is not certain enough. Accordingly, the second uncertainty value corresponding to that pixel location in the restored image is high, and that pixel location in the restored image should be assigned a lower weight.

[0056] Next, for each pixel in the restored image, the uncertainty value corresponding to that pixel is obtained based on the first uncertainty value and the second uncertainty value corresponding to that pixel.

[0057] According to embodiments of this disclosure, for each pixel in the recovered image, the first uncertainty value and the second uncertainty value corresponding to the pixel are added or multiplied to obtain the uncertainty value corresponding to the pixel.

[0058] Next, the weight of each pixel is determined based on the uncertainty value of each pixel in the restored image.

[0059] As mentioned earlier, the weight of each pixel in the restored image is negatively correlated with the uncertainty value of that pixel. In one example, the weight of each pixel in the restored image can be obtained using the following formula.

[0060]

[0061] In the above formula, w n This represents the weight corresponding to the nth pixel in the recovered image. This represents the uncertainty value corresponding to the nth pixel in the restored image, where n is a positive integer.

[0062] Next, for each band of the reference style image, based on the preset style transfer model, the mapped pixel values ​​of all pixels in the reference style image in that band are determined according to the restored pixel values ​​of all pixels in the restored image in each band.

[0063] According to embodiments of this disclosure, a style transfer model is used to characterize the sub-style mapping relationship between a reference style image and a reconstructed image in various bands. For each band of the reference style image, based on the style transfer model, and according to the reconstructed pixel values ​​of each pixel in the reconstructed image in the three bands (including R-band, G-band, and B-band), the mapped pixel value of the corresponding pixel in the reference style image in that band can be determined.

[0064] For example, the style transfer model can be represented by the following formula for each band.

[0065]

[0066] In the above formula, x i,n This represents the restored pixel value of the nth pixel in the i-th band of the restored image (i=1,2,3, corresponding to the R-band, G-band, and B-band respectively), y r,n This represents the mapped pixel value of the nth pixel in the reference style image in the r-band. , , and These are the style parameters corresponding to the r-band. n=1,2,…,N, where N is the total number of pixels in the reference style image or the restored image.

[0067] Next, for each band of the reference style image, an objective function is constructed based on the reference pixel values ​​of all pixels in the reference style image in that band, the mapped pixel values ​​of all pixels in the reference style image in that band, and the weights corresponding to each pixel in the recovered image.

[0068] According to embodiments of this disclosure, for each band of a reference style image, the objective function is a function determined with the objective of minimizing the difference between the reference pixel value and the mapped pixel value of each pixel in the reference style image in that band.

[0069] For example, the objective function can be expressed using the following formula.

[0070]

[0071] In the above formula, This represents the objective function corresponding to the r-band, where N is the total number of pixels in the reference style image, and w n This represents the weight corresponding to the nth pixel in the recovered image. y represents the reference pixel value of the nth pixel in the reference style image in the r band. r,n This represents the mapped pixel value of the nth pixel in the reference style image in band r, y r,nIt is based on the style transfer model corresponding to the r-band, and is calculated based on the restored pixel values ​​of the corresponding pixels in each band in the restored image, where n is a positive integer less than or equal to N.

[0072] Next, for each band of the reference style image, the style parameters corresponding to that band are determined according to the objective function, and the sub-style mapping relationship corresponding to that band is constructed based on the style parameters corresponding to that band.

[0073] According to embodiments of this disclosure, by solving the objective function, the style parameters corresponding to each band can be obtained, including those described in the above embodiments. , , and Then, the style parameters corresponding to each band are applied to the style transfer model of the corresponding band to obtain the sub-style mapping relationship for each band.

[0074] Next, the style mapping relationship is obtained based on the sub-style mapping relationship corresponding to each band of the reference style image.

[0075] According to embodiments of this disclosure, the sub-style mapping relationships corresponding to each band of a reference style image can be used as style mapping relationships. These style mapping relationships are used to convert the image to be processed into an output image with a reference style.

[0076] According to an embodiment of this disclosure, in the above operation S140, performing style transfer on the image to be processed according to the style mapping relationship to obtain the target image includes: for each band in the image to be processed, using the sub-style mapping relationship corresponding to that band, determining the target pixel value of the pixel in the corresponding band of the target image based on the original pixel value of each pixel in each band of the image to be processed. Traversing all pixels in the image to be processed, the target image is obtained.

[0077] Figure 2 This is a block diagram of a remote sensing image processing apparatus according to an embodiment of the present disclosure.

[0078] like Figure 2 As shown, the remote sensing image processing device 200 includes: an acquisition module 210, a processing module 220, a determination module 230, and a style transfer module 240.

[0079] The acquisition module 210 is used to acquire the image to be processed and the reference style image.

[0080] The processing module 220 is used to perform destylization processing on the reference style image to obtain the restored image.

[0081] The determination module 230 is used to determine the style mapping relationship between the restored image and the reference image based on the reference style image and the restored image.

[0082] The style transfer module 240 is used to perform style transfer on the image to be processed according to the style mapping relationship to obtain the target image, which has the image style of the reference style image.

[0083] It should be noted that the implementation methods, technical problems solved, functions achieved, and technical effects of each module in the device embodiment are the same as or similar to the implementation methods, technical problems solved, functions achieved, and technical effects of each corresponding step in the method embodiment, and will not be repeated here.

[0084] Figure 3 A block diagram schematically illustrates an electronic device suitable for implementing a remote sensing image processing method according to an embodiment of the present disclosure.

[0085] like Figure 3 As shown, an electronic device 300 according to an embodiment of this disclosure includes a processor 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage portion 308 into a random access memory (RAM) 303. The processor 301 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 301 may also include onboard memory for caching purposes. The processor 301 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0086] RAM 303 stores various programs and data required for the operation of electronic device 300. Processor 301, ROM 302, and RAM 303 are interconnected via bus 304. Processor 301 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 302 and / or RAM 303. It should be noted that the programs may also be stored in one or more memories other than ROM 302 and RAM 303. Processor 301 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0087] According to embodiments of this disclosure, the electronic device 300 may further include an input / output (I / O) interface 302, and an input / output (I / O) interface 305 is also connected to the bus 304. The electronic device 300 may also include one or more of the following components connected to the I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 310 as needed so that computer programs read from it can be installed into the storage section 308 as needed.

[0088] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the remote sensing image processing method according to the embodiments of this disclosure.

[0089] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 302 and / or RAM 303 and / or one or more memories other than ROM 302 and RAM 303 described above.

[0090] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the remote sensing image processing method provided in the embodiments of this disclosure.

[0091] When the computer program is executed by the processor 301, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0092] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via communication section 309, and / or installed from removable medium 311. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0093] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by processor 301, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0094] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, Python, C, or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] Those skilled in the art will understand that the features described in the various embodiments of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0097] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A remote sensing image processing method, comprising: Acquire the image to be processed and the reference style image; The reference style image is destylated to obtain the restored image; Based on the reference style image and the restored image, determine the style mapping relationship between the restored image and the reference image; The image to be processed is style-transferred according to the style mapping relationship to obtain a target image, which has the image style of the reference style image.

2. The method according to claim 1, wherein, The process of destylating the reference style image to obtain the restored image includes: The reference style image is destylated using a destylating network to obtain the restored image, wherein the destylating network is trained on an initial destylating network using multiple sample images.

3. The method according to claim 2, wherein, The destylization network was trained in the following way: Multiple sample image pairs are obtained, each sample image pair including an original sample image and a reference sample image corresponding to the original sample image. The original sample image is a content image without any style applied, and the reference sample image is an image obtained by applying a sample style to the original sample image. For each pair of sample images, the reference sample image in the pair is destylated using an initial destylization network to obtain the restored image; Based on the pixel value differences between each original sample image and its corresponding restored image, the parameters of the initial destylization network are adjusted to obtain the destylization network.

4. The method according to claim 3, wherein, The step of determining the style mapping relationship between the restored image and the reference image based on the reference style image and the restored image includes: Determine the weight corresponding to each pixel in the restored image, the weight being used to characterize the reliability of the restored pixel value of the corresponding pixel in the restored image; For each band of the reference style image, based on a preset style transfer model, the mapped pixel value of all pixels in the reference style image in that band is determined according to the recovered pixel values ​​of all pixels in the recovered image in each band. For each band of the reference style image, an objective function is constructed based on the reference pixel values ​​of all pixels in the reference style image in that band, the mapped pixel values ​​of all pixels in the reference style image in that band, and the weights corresponding to each pixel in the restored image. For each band of the reference style image, the style parameters corresponding to the band are determined according to the objective function, and a sub-style mapping relationship corresponding to the band is constructed based on the style parameters corresponding to the band. The style mapping relationship is obtained based on the sub-style mapping relationship corresponding to each band of the reference style image.

5. The method according to claim 4, wherein, Determining the weight corresponding to each pixel in the restored image includes: Determine the uncertainty value corresponding to each pixel in the restored image; The weight of each pixel is determined based on the uncertainty value of each pixel in the restored image.

6. The method according to claim 5, wherein, The uncertainty value of each pixel in the restored image is negatively correlated with the weight of that pixel.

7. The method according to claim 5, wherein, The determination of the uncertainty value corresponding to each pixel in the restored image includes: Obtain preset semantic prior data, which includes the category bias variance corresponding to each of the multiple semantic categories in multiple original sample images. The category bias variance refers to the statistical variance of the color deviation of all pixels belonging to the same semantic category in multiple original sample images. For each pixel in the restored image, based on the target semantic category to which the corresponding pixel in the reference style image belongs, the target category deviation variance corresponding to the target semantic category is queried from the semantic prior data, and the target category deviation variance is used as the first uncertainty value of the pixel. Multiple restored images are generated by multiple inferences from the reference style image. Based on the restored pixel value corresponding to each pixel position in the restored image in the multiple restored images, a second uncertainty value corresponding to the pixel position is determined. For each pixel in the restored image, the uncertainty value corresponding to that pixel is obtained based on the first uncertainty value and the second uncertainty value corresponding to that pixel.

8. A remote sensing image processing apparatus, comprising: The acquisition module is used to acquire the image to be processed and the reference style image; The processing module is used to perform destylization processing on the reference style image to obtain the restored image; The determining module is used to determine the style mapping relationship between the restored image and the reference image based on the reference style image and the restored image; The style transfer module is used to perform style transfer on the image to be processed according to the style mapping relationship to obtain a target image, wherein the target image has the image style of the reference style image.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the remote sensing image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the remote sensing image processing method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the remote sensing image processing method according to any one of claims 1 to 7.

Images