Remote sensing image loading method and device, electronic equipment and medium

Abstract

The invention provides a remote sensing image loading method and device, electronic equipment and a medium, and the specific implementation scheme comprises the steps: receiving a tile obtaining request of a user terminal, the tile obtaining request comprising tile parameters of a to-be-loaded tile; initial tile data are obtained according to the tile parameters, and the resolution of the initial tile data is a first resolution; in response to determining that a scaling hierarchy corresponding to the initial tile data reaches a preset hierarchy according to the tile parameters, super-division processing is performed on the initial tile data to obtain super-division tile data, the resolution of the super-division tile data is a second resolution, and the second resolution is greater than the first resolution; performing resampling processing on the super-divided tile data to obtain target tile data; and returning the target tile data to the user terminal.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image loading, and particularly relates to a remote sensing image loading method and device, electronic equipment, a storage medium and a computer program product. BACKGROUND

[0002] Remote sensing images, as important geographic information data, need to be frequently loaded and used in various geographic information systems. In related technologies, a tile map service is usually used to realize loading and display of remote sensing images. This method needs to pre-cut remote sensing image data into tile images with fixed sizes, and provide the tile images to a client in the form of a Web service. However, because the spatial resolution of a tile is limited by original image data and slice level settings, some areas may be affected in display effect due to lack of details at a high zoom level. SUMMARY

[0003] The present application provides a remote sensing image loading method and device, electronic equipment, a storage medium and a computer program product.

[0004] According to one aspect of the present application, a remote sensing image loading method is provided, including: receiving a tile acquisition request of a user terminal, the tile acquisition request including tile parameters of a tile to be loaded; acquiring initial tile data according to the tile parameters, the resolution of the initial tile data being a first resolution; in response to determining that a zoom level corresponding to the initial tile data reaches a preset level according to the tile parameters, performing super-resolution processing on the initial tile data to obtain super-resolution tile data, the resolution of the super-resolution tile data being a second resolution, the second resolution being greater than the first resolution; performing resampling processing on the super-resolution tile data to obtain target tile data; and returning the target tile data to the user terminal.

[0005] According to another aspect of the present application, a remote sensing image loading device is provided, including: a receiving module configured to receive a tile acquisition request of a user terminal, the tile acquisition request including tile parameters of a tile to be loaded; an acquiring module configured to acquire initial tile data according to the tile parameters, the resolution of the initial tile data being a first resolution; a super-resolution module configured to, in response to determining that a zoom level corresponding to the initial tile data reaches a preset level according to the tile parameters, perform super-resolution processing on the initial tile data to obtain super-resolution tile data, the resolution of the super-resolution tile data being a second resolution, the second resolution being greater than the first resolution; a resampling module configured to perform resampling processing on the super-resolution tile data to obtain target tile data; and a returning module configured to return the target tile data to the user terminal.

[0006] According to another aspect of the present application, there is provided an electronic device comprising: one or more processors; a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the remote sensing image loading method as described above.

[0007] According to another aspect of the present application, there is provided a computer readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the remote sensing image loading method as described above.

[0008] According to another aspect of the present application, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the remote sensing image loading method as described above. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0010] Figure 1 is a schematic diagram of a system architecture according to an embodiment of the present application which can apply the remote sensing image loading method;

[0011] Figure 2 is a flowchart of the remote sensing image loading method according to an embodiment of the present application;

[0012] Figure 3 is a flowchart of the second super-resolution model obtaining method according to an embodiment of the present application;

[0013] Figure 4A is a schematic diagram of the structure of the initial super-resolution model according to an embodiment of the present application;

[0014] Figure 4B is a schematic diagram of the structure of the first feature extraction sub-module and the second feature extraction sub-module according to an embodiment of the present application;

[0015] Figure 4C is a schematic diagram of the structure of the spatial attention sub-module according to an embodiment of the present application;

[0016] Figure 5 is a block diagram of the remote sensing image loading device according to an embodiment of the present application;

[0017] Figure 6 is a block diagram of the electronic device suitable for implementing the remote sensing image loading method according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments and the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0019] It should be noted that the serial numbers of the operations in the following methods are only used to represent the operations for description, and should not be regarded as representing the execution sequence of the operations. Unless explicitly stated, the method does not need to be executed in the order shown. In addition, in the description of the present application, the terms "first", "second", etc. (if any) are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0020] In the technical solutions of the present application, the collection, storage, use, processing, transmission, provision, disclosure and application of the data (for example, including but not limited to user personal information) involved all comply with the relevant legal regulations of the relevant countries and regions, and do not violate public order and good customs. In addition, in the technical solutions of the present application, the authorization or consent of the data owner is obtained before obtaining or collecting relevant data.

[0021] Figure 1 is a system architecture schematic diagram according to the remote sensing image loading method of the embodiments of the present application. It should be noted that, Figure 1 The shown is only an example of a system architecture that can apply the embodiments of the present application, to help those skilled in the art understand the technical content of the present application, but does not mean that the embodiments of the present application cannot be used for other devices, systems, environments or scenarios.

[0022] As Figure 1 shown, the system architecture 100 according to the embodiments can include a user terminal 101, a server 102 and a network 103.

[0023] The user terminal 101 may, for example, be various electronic devices provided with an interactive interface, including but not limited to a smart phone, a notebook computer, a tablet computer, a VR / AR device, a vehicle-mounted terminal, etc. The user terminal 101 may, for example, be installed with various client applications, including but not limited to geographic information system applications, map navigation applications or web browser applications, etc. The user can query the required information or data through human-computer interaction with the user terminal 101.

[0024] The server 102 can be a background management server or the like that provides support for the running of a client application in the user terminal 101, for example. The server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms, and the like.

[0025] The network 103 is a medium for providing a communication link between the user terminal 101 and the server 102. The network 103 can include various connection types, such as wired, wireless communication links, or fiber optic cables, and the like.

[0026] The user terminal 101 can obtain tile parameters corresponding to a tile to be loaded, for example, through human-computer interaction, and send a tile acquisition request to the server 102, the tile acquisition request carrying the tile parameters of the tile to be loaded. After receiving the tile acquisition request sent by the user terminal 101, the server 102 obtains initial tile data according to the tile parameters carried in the tile acquisition request, the resolution of the initial tile data being a first resolution. Then, the server 102 determines whether the zoom level corresponding to the initial tile data reaches a preset level according to the tile parameters, and performs super-resolution processing on the initial tile data to obtain super-resolution tile data in the case where the zoom level corresponding to the initial tile data reaches the preset level, the resolution of the super-resolution tile data being a second resolution, the second resolution being greater than the first resolution. Next, the server 102 performs resampling processing on the super-resolution tile data to obtain target tile data, and further returns the target tile data to the user terminal 101 for display on the user terminal 101.

[0027] In some embodiments, the server 102 can also input a sample image into an initial super-resolution model, perform super-resolution processing on the sample image using the initial super-resolution model to obtain a super-resolution image. Then, the server 102 performs parameter adjustment on the initial super-resolution model according to a reference image corresponding to the super-resolution image and the sample image to obtain a first super-resolution model. After that, the server 102 performs reparameterization processing on the first super-resolution model to generate a second super-resolution model. The server 102 can perform super-resolution processing on the initial tile data using the second super-resolution model to obtain super-resolution tile data.

[0028] It should be noted that the remote sensing image loading method provided by the embodiment of the present application can be executed by the server 102. Correspondingly, the remote sensing image loading device provided by the embodiment of the present application can be arranged in the server 102. The remote sensing image loading method provided by the embodiment of the present application can also be executed by a server or a server cluster different from the server 102 and capable of communicating with the user terminal 101 and / or the server 102. Correspondingly, the remote sensing image loading device provided by the embodiment of the present application can also be arranged in a server or a server cluster different from the server 102 and capable of communicating with the user terminal 101 and / or the server 102.

[0029] It should be understood that Figure 1 The number of the user terminal 101, the server 102 and the network 103 in the system 100 is only illustrative. According to actual needs, there can be any number of user terminals 101, servers 102 and networks 103.

[0030] Figure 2 FIG. 2 is a flowchart of a remote sensing image loading method according to an embodiment of the present application.

[0031] As shown in FIG. 2, the remote sensing image loading method 200 includes operations S210-S250. Figure 2

[0032] In operation S210, a tile acquisition request of a user terminal is received, and the tile acquisition request includes tile parameters of a tile to be loaded.

[0033] According to the embodiment of the present application, the tile parameters are used to identify and locate the tile data corresponding to the tile to be loaded. Here, the tile data refers to the grid slice data. The tile parameters may, for example, include parameters such as the zoom level, the row number and the column number corresponding to the tile to be loaded.

[0034] In operation S220, the initial tile data is acquired according to the tile parameters.

[0035] According to the embodiment of the present application, the corresponding tile data is read according to the tile parameters to obtain the initial tile data, and the resolution of the initial tile data is the first resolution.

[0036] In operation S230, in response to determining that the zoom level corresponding to the initial tile data reaches a preset level according to the tile parameters, the initial tile data is super-resolution processed to obtain super-resolution tile data.

[0037] In the embodiment of the present application, the super-resolution processing refers to image super-resolution processing, which generates a high-resolution image (SR image) by performing super-resolution processing on a low-resolution image (LR image).

[0038] ​It can be understood that in the tile map service, higher zoom level usually requires higher data resolution. However, since the resolution of the tile data stored in the server is usually fixed, when expressing a region with rich ground details, the tile data of the corresponding level may not be sufficient in resolution, which affects the clarity and detail performance of the remote sensing image, and thus reduces the display effect of the remote sensing image.

[0039] In view of this, in the embodiments of the present application, whether the tile data meets the data accuracy requirement of the corresponding zoom level is determined by judging whether the zoom level corresponding to the tile data reaches the preset level, and then it is determined whether the tile data needs to be super-resolution processed. In this way, image data matching the zoom level can be obtained at any zoom level, so as to ensure that the display effect of the remote sensing image is adapted to the corresponding zoom level.

[0040] For example, if it is determined according to the tile parameters that the zoom level corresponding to the initial tile data does not reach the preset level, it means that the initial tile data can meet the data accuracy requirement of the corresponding zoom level, and thus the initial tile data can be directly returned to the user terminal for display. If it is determined according to the tile parameters that the zoom level corresponding to the initial tile data reaches the preset level, it means that the initial tile data cannot meet the data accuracy requirement of the corresponding zoom level, and thus the initial tile data needs to be super-resolution processed to obtain super-resolution tile data. The resolution of the super-resolution tile data is a second resolution, and the second resolution is greater than the first resolution. In this way, the target tile data returned subsequently can be matched with the corresponding zoom level, and thus the image display effect is ensured.

[0041] It should be noted that in the embodiments of the present application, the preset tile level can be set according to actual needs, which is not limited herein.

[0042] In operation S240, the super-resolution tile data is resampled to obtain target tile data.

[0043] In the embodiments of the present application, the super-resolution tile data is resampled to process the super-resolution tile data into a standard tile size to adapt to the scheduling specification of the map engine.

[0044] In operation S250, the target tile data is returned to the user terminal.

[0045] After obtaining the target tile data, the server can return the target tile data to the user terminal to display the target tile data on the user terminal.

[0046] According to the technical scheme of the present application, by taking the comparison result of the current tile data corresponding scaling level and the preset level as the trigger condition of the super-resolution processing, not only the accurate control of the tile super-resolution processing is realized, but also the adaptive matching of the image display effect and the scaling level is realized.

[0047] According to the embodiment of the present application, in the operation S230, the initial tile data can be super-resolution processed by using the second super-resolution model to obtain the super-resolution tile data. The second super-resolution model is obtained by re-parameterizing the first super-resolution model, and the parameter amount of the second super-resolution model is less than that of the first super-resolution model. The first super-resolution model is obtained by model training of the initial super-resolution model by using the sample image pair. The process of obtaining the second super-resolution model is described below.

[0048] Figure 3 The flow chart of the second super-resolution model obtaining method according to the embodiment of the present application is shown in FIG. 3.

[0049] As shown in FIG. 3, the second super-resolution model obtaining method includes operations S301-S307. Figure 3

[0050] In operation S301, a sample image pair is obtained, which includes a sample image and a reference image corresponding to the sample image.

[0051] The sample image can be a low-resolution remote sensing image. The reference image is a high-resolution remote sensing image corresponding to the sample image. In the embodiment of the present application, the sample image can be obtained by reducing the resolution of the reference image (for example, including but not limited to down-sampling processing).

[0052] In the embodiment of the present application, the dimension of the sample image can be represented as [C, H, W], wherein C represents the channel number of the sample image, H represents the height of the sample image, and W represents the width of the sample image, and C, H, and W are integers greater than 1. For example, the channel number C of the sample image can be 3.

[0053] In operation S302, the shallow feature extraction network in the initial super-resolution model is used to perform shallow feature extraction on the sample image to obtain an initial feature map.

[0054] Figure 4A The structure diagram of the initial super-resolution model is shown in FIG. 4.

[0055] As shown in FIG. 4, the initial super-resolution model includes a shallow feature extraction network, a deep feature extraction network, and a reconstruction network. Figure 4A

[0056] ​​The shallow feature extraction network may be, for example, a network structure composed of one or more convolutional layers, which is configured to perform shallow feature extraction on an input image to obtain an initial feature map.

[0057] The deep feature extraction network may include a plurality of processing modules configured to perform multi-level feature extraction and fusion processing on the initial feature map output by the shallow feature extraction network. For example, as shown in FIG. 1, the deep feature extraction network may include, for example, a first feature processing module, a second feature processing module, a third feature processing module, a first channel attention module, a fourth feature processing module, a fifth feature processing module, and a second channel attention module, which are connected in cascade. The first feature processing module, the second feature processing module, the third feature processing module, the fourth feature processing module, and the fifth feature processing module are composed of the same sub-modules, which are respectively configured to perform deep feature extraction on respective input feature maps. The first channel attention module and the second channel attention module have the same structure, and are respectively configured to perform processing on respective input feature maps based on a channel attention mechanism. Figure 4A

[0058] In some embodiments, the first channel attention module and the second channel attention module may each be implemented by, for example, a Squeeze-and-Excitation module (SE Block). The input of the first channel attention module is the feature map output by the third feature processing module, and the input of the second channel attention module is the feature map output by the fifth feature processing module.

[0059] In some embodiments, the first feature processing module, the second feature processing module, the third feature processing module, the fourth feature processing module, and the fifth feature processing module each include a first feature extraction sub-module, a second feature extraction sub-module, and a spatial attention sub-module connected in cascade. The first feature extraction sub-module and the second feature extraction sub-module have the same structure, and are respectively configured to perform local feature extraction on respective input feature maps. The spatial attention sub-module is configured to perform processing on the feature map output by the second feature extraction sub-module based on a spatial attention mechanism.

[0060] In the inference stage, the first feature extraction sub-module and the second feature extraction sub-module included in each of the first feature processing module, the second feature processing module, the third feature processing module, the fourth feature processing module, and the fifth feature processing module may be respectively re-parameterized as a convolutional layer with a convolutional kernel size of k*k. For example, as shown in FIG. 2, the first feature extraction sub-module and the second feature extraction sub-module included in the first feature processing module may be respectively re-parameterized as a convolutional layer with a convolutional kernel size of 3*3. Figure 4A ​As shown, the first feature extraction submodule in the first feature processing module can be re-parameterized as a first convolutional layer with a convolution kernel size of k*k (k is 3 for example). The second feature extraction submodule in the first feature processing module can be re-parameterized as a second convolutional layer with a convolution kernel size of k*k (k is 3 for example). The first convolutional layer has the same performance as the first feature extraction submodule, and the second convolutional layer has the same performance as the second feature extraction submodule, but the parameter amount of the first convolutional layer is less than that of the first feature extraction submodule, and the parameter amount of the second convolutional layer is less than that of the second feature extraction submodule.

[0061] In some embodiments, for each of the first feature processing module, the second feature processing module, the third feature processing module, the fourth feature processing module, and the fifth feature processing module, in addition to including the cascaded first feature extraction submodule, the second feature extraction submodule, and the spatial attention submodule, other submodules can also be included, for example, a first activation submodule and a second activation submodule are also included. The first activation submodule is used to activate the feature map output by the first feature extraction submodule to increase the nonlinearity of the features. The second activation submodule is used to activate the feature map output by the second feature extraction submodule to increase the nonlinearity of the features. The activation function used by the first activation submodule and the second activation submodule can be, for example, a Gelu activation function, etc.

[0062] In some embodiments, as shown in FIG. 1, the first feature processing module, the second feature processing module, the third feature processing module, the fourth feature processing module, and the fifth feature processing module can be cascaded to form a feature processing network. Figure 4B As shown, the first feature extraction submodule and the second feature extraction submodule each include a first channel transformation unit, a scaling unit, a spatial convolution unit, a second channel transformation unit, and a residual connection unit.

[0063] The first channel transformation unit is used to expand the number of channels of the input feature map of the first feature extraction submodule (or the second feature extraction submodule). For example, the first channel transformation unit can include at least one convolutional layer with a convolution kernel size of 1*1.

[0064] The scaling unit is used to perform channel-by-channel scaling processing on the feature map output by the first channel transformation unit. For example, the scaling unit can be implemented by a scaling function with learnable weight parameters. In the training phase, the learnable weight parameters contained in the scaling unit are randomly initialized together with the convolution kernel, and are automatically learned through back propagation. In the inference phase, the scaling unit is fused into the first convolutional layer or the second convolutional layer introduced earlier through re-parameterization processing.

[0065] The spatial convolution unit is used to extract spatial features from the feature map output by the scaling unit. In one example, the spatial convolution unit can include a convolutional layer with a convolution kernel size of n*n (n is 3 for example) and an identity mapping.

[0066] The second channel change unit is configured to compress the channel number of the feature map output by the spatial convolution unit, so that the channel number of the compressed feature map is the same as the channel number of the input feature map of the first feature extraction sub-module (or the second feature extraction sub-module). For example, the second channel change unit can include at least one convolution layer with a convolution kernel size of 1*1.

[0067] The residual connection unit is configured to splice the input feature map of the first feature extraction sub-module (or the second feature extraction sub-module) and the feature map output by the second channel change unit. For example, the residual connection unit can use the operation in 4B to splice the two feature maps.

[0068] In some embodiments, the spatial attention sub-module includes a local importance estimation unit, a gating signal generation unit, and a fusion unit.

[0069] The local importance estimation unit is configured to generate a spatial attention weight map according to the feature map output by the second feature extraction sub-module. For example, as Figure 4CAs shown, the local importance estimation unit includes, for example, a third convolutional layer, a Softpool layer, a fourth convolutional layer, a fifth convolutional layer, a first Sigmoid activation layer, and an up-sampling layer. The third convolutional layer can be a 1*1 convolutional layer, which acts as a projection layer for feature transformation of the feature map output by the second feature extraction sub-module, so as to provide a more suitable feature space for the Softpool layer to calculate the local importance. The Softpool layer is configured to perform a Softmax weighting-based pooling operation in a local region (e.g., a 3*3 window) of the feature map output by the third convolutional layer, and output a down-sampled, low-resolution initial importance map. The initial importance map is an initial importance matrix, which is configured to roughly represent the relative importance of each local region in the feature map output by the third convolutional layer in its context. Each element in the initial importance map corresponds to a local region (e.g., a 3*3 window) in the feature map output by the third convolutional layer, and the value of each element represents the relative importance of the local region. The greater the value of each element, the higher the importance of the corresponding local region. The fourth convolutional layer can be a 3*3 convolutional layer, for example, which is configured to perform convolutional processing on the initial importance map output by the Softpool layer to obtain a first intermediate importance map. When the initial importance map is processed by the fourth convolutional layer, the convolutional operation fuses more context information, so that the initial importance map can be further refined and calibrated, thereby making the importance judgment of the local region more robust and accurate. The fifth convolutional layer can be a 3*3 convolutional layer, for example, which is configured to perform convolutional processing on the first intermediate importance map to achieve more fine-grained context fusion of the first intermediate importance map, thereby further consolidating and refining the importance information, and generating a more robust and discriminative low-resolution importance representation, i.e., a second intermediate importance map. The first Sigmoid activation layer can be implemented by a Sigmoid function, for example, which is configured to map the value of each element in the second intermediate importance map output by the fifth convolutional layer to the interval (0, 1) to obtain an initial spatial attention weight map. The up-sampling layer is configured to up-sample the initial spatial attention weight map output by the first Sigmoid activation layer back to the same spatial size as the feature map output by the second feature extraction sub-module, for example, by bilinear interpolation, to obtain a spatial attention weight map. The spatial attention weight map is a spatial attention weight matrix, and the greater the value of an element in the spatial attention weight matrix, the higher the importance of the feature at the corresponding spatial position in the feature map output by the second feature extraction sub-module for the subsequent super-resolution reconstruction task. The spatial attention weight map can be used to perform weighted processing on the feature map output by the second feature extraction sub-module, so as to enhance the important details in the feature map and suppress redundant information, thereby facilitating subsequent efficient and high-quality super-resolution reconstruction.

[0070] The gated signal generation unit performs average pooling on the feature map output by the second feature extraction submodule along the channel dimension to obtain a single-channel feature map. Based on this single-channel feature map, a gated weight matrix is ​​generated. This gated matrix is ​​used to refine and calibrate the spatial attention weight map output by the local importance estimation unit, compensating for potential information loss and localization ambiguity caused by downsampling / upsampling during the generation of the spatial attention weight map by the local importance estimation unit. In one example, the gated signal generation unit may include an average pooling layer and a second Sigmoid activation layer. The average pooling layer performs average pooling on the feature map output by the second feature extraction submodule along the channel dimension to obtain a single-channel feature map. The second Sigmoid activation layer can be implemented using a Sigmoid function, which maps the value of each element in the single-channel feature map to the (0,1) interval to obtain the gated matrix.

[0071] The fusion unit is used to weight the feature map output by the second feature extraction submodule using a spatial attention weight map and a gating matrix to achieve an efficient and effective attention mechanism. For example, the fusion unit can use... Figure 4C In Symbols are used to represent them.

[0072] Return to reference Figure 4A As shown, the reconstruction network is used to perform feature reconstruction processing on the feature map output by the deep feature extraction network to obtain the final super-resolution image. In some embodiments, the reconstruction network may include convolutional layers and depth-to-space layers. The convolutional layers may be, for example, 1*1 or 3*3 convolutional layers. The convolutional layers are used to transform the number of channels of the feature map output by the deep feature extraction network into the target number of channels (e.g., C_out*r). 2 Where C_out is the number of channels in the output super-resolution image, and r is the super-resolution magnification factor. The Depth-to-Space layer rearranges the information of the feature map output by the convolutional layer in the channel dimension to the spatial dimension through data rearrangement operation, thereby converting the feature map with a high number of channels and low spatial resolution into a low number of channels and high spatial resolution output, thus obtaining the super-resolution image.

[0073] Returning to operation S302, in this embodiment of the invention, the sample image is input into a shallow feature extraction network so that shallow features are extracted from the sample image using the shallow feature extraction network to obtain an initial feature map. The dimensions of the initial feature map can be represented as [C0, H, W], where C0 is an integer greater than C.

[0074] In operation S303, the deep feature extraction network in the initial super-resolution model is used to perform feature enhancement processing on the initial feature map to obtain the target fusion enhanced feature map.

[0075] The network structure in Figure 4A to Figure 4C is taken as an example to illustrate the acquisition process of the target fusion enhanced feature map.

[0076] Firstly, the initial feature map is subjected to deep feature extraction using the first feature processing module to obtain a first deep feature map.

[0077] The acquisition process of the first deep feature map is introduced below.

[0078] Please refer to Figure 4A , after the initial feature map is input into the first feature processing module in the deep feature extraction network, the initial feature map is subjected to local feature extraction using a first feature extraction submodule in the first feature processing module to obtain an initial deep feature map.

[0079] Exemplarily, please refer to Figure 4B , after the initial feature map is input into the first feature extraction submodule, the initial feature map is subjected to channel expansion using a first channel transformation unit, and subjected to per-channel scaling processing using a scaling unit to obtain a scaled feature map. Then, the scaled feature map is subjected to spatial feature extraction using a spatial convolution unit to obtain a convolution feature map. Then, the convolution feature map is subjected to channel compression using a second channel transformation unit to obtain a channel-compressed convolution feature map. The channel-compressed convolution feature map has the same number of channels as the initial feature map. Then, the initial feature map and the channel-compressed convolution feature map are spliced using a residual connection unit to obtain the initial deep feature map. The initial deep feature map has the same dimension as the initial feature map, both being [C0, H, W].

[0080] Then, the initial deep feature map is subjected to local feature extraction using a second feature extraction submodule in the first feature processing module to obtain an intermediate deep feature map.

[0081] In the embodiment of the present application, the process of local feature extraction of the initial deep feature map using the second feature extraction submodule is similar to the process of local feature extraction of the initial feature map using the first feature extraction submodule, and thus is not described again here for the sake of brevity.

[0082] In the embodiment of the present application, the dimension of the intermediate deep feature map is consistent with that of the initial deep feature map, both being [C0, H, W].

[0083] Then, the intermediate deep feature map is processed based on a spatial attention mechanism using a spatial attention submodule in the first feature processing module to obtain the first deep feature map.

[0084] The following describes the process of using the spatial attention submodule to process intermediate depth feature maps based on the spatial attention mechanism.

[0085] First, the spatial attention weight map is determined based on the intermediate depth feature map using the local importance estimation unit in the spatial attention submodule.

[0086] For example, please refer to Figure 4C As shown, for example, the third convolutional layer in the local importance estimation unit can be used to perform feature transformation on the intermediate depth feature map to obtain the transformed intermediate depth feature map. Next, the Softpool layer in the local importance estimation unit performs a Softmax-weighted pooling operation on the transformed intermediate depth feature map to obtain the initial importance map. Then, the fourth convolutional layer in the local importance estimation unit performs convolution processing on the initial importance map to fuse a wider range of contextual information, thereby further refining and calibrating the initial importance map to obtain the first intermediate importance map. Next, the fifth convolutional layer in the local importance estimation unit performs convolution processing on the first intermediate importance map to achieve more refined contextual fusion, thereby further consolidating and refining the importance information, and generating a more robust and discriminative low-resolution importance representation, namely the second intermediate importance map. Finally, the first Sigmoid activation layer in the local importance estimation unit maps the value of each element in the second intermediate importance map output by the fifth convolutional layer to the (0,1) interval to obtain the initial spatial attention weight map. Next, the initial spatial attention weight map output by the first Sigmoid activation layer is upsampled back to the same spatial size as the intermediate depth feature map using the upsampling layer in the local importance estimation unit, thus obtaining the spatial attention weight map.

[0087] In this embodiment of the invention, the spatial attention weight map is a matrix with the same spatial size as the intermediate depth feature map. The value of each element in the spatial attention weight map represents the importance of the feature at the corresponding spatial location in the intermediate depth feature map to the subsequent super-resolution reconstruction task. A larger value for an element indicates higher importance of the feature at the corresponding spatial location in the intermediate depth feature map to the subsequent super-resolution reconstruction task, and it will be assigned a higher weight in subsequent processing. The spatial attention weight map can be used to weight the intermediate depth feature map to enhance important details and suppress redundant information, thereby facilitating efficient and high-quality super-resolution reconstruction.

[0088] Next, the gating signal generation unit in the spatial attention submodule is used to perform average pooling on the intermediate depth feature map in the channel dimension to obtain a single-channel feature map, and a gating matrix is ​​generated based on the single-channel feature map.

[0089] Exemplarily, the intermediate deep feature map can be subjected to average pooling processing in the channel dimension by using an average pooling layer in the gate signal generation unit to obtain a single-channel feature map. Then, each element in the single-channel feature map is mapped to the interval (0, 1) by using a second Sigmoid activation layer in the gate signal generation unit to obtain a gating matrix. The gating matrix is used to finely calibrate the spatial attention weight map to make up for the loss of detailed information and the positioning ambiguity that may be caused by down-sampling / up-sampling in the process of generating the spatial attention weight map by the local importance estimation unit, which can effectively prevent the edge blurring of the spatial attention weight map and the false activation of unimportant regions. In this way, the accuracy and precision of the spatial attention weight map can be improved, and the accuracy and precision of the super-resolution reconstruction result can be improved, so that the details in the super-resolution image are clearer and the texture is more natural.

[0090] Then, the intermediate deep feature map is subjected to weighted processing according to the spatial attention weight map and the gating matrix by using a fusion unit in the spatial attention sub-module to obtain a first deep feature map.

[0091] For example, the spatial attention weight map and the gating matrix can be multiplied element by element by using the fusion unit to obtain a calibrated spatial attention weight map, and the calibrated spatial attention weight map and the intermediate deep feature map can be multiplied element by element to obtain the first deep feature map. The dimensions of the first deep feature map and the intermediate deep feature map remain the same, both being [C0, H, W].

[0092] At this point, the depth feature extraction on the initial feature map by using the first feature processing module is completed, and the first deep feature map is obtained.

[0093] Next, the first deep feature map is split into two parts in the channel dimension according to a first preset ratio to obtain a first sub deep feature map and a second sub deep feature map.

[0094] In the embodiments of the present application, the first preset ratio can be set according to actual conditions, which is not limited here. The dimension of the first sub deep feature map can be represented as [C1, H, W] for example, and the dimension of the second sub deep feature map can be represented as [C0-C1, H, W] for example, wherein C1 represents the first C1 channels of the first deep feature map, and C1 is an integer greater than 0 and less than or equal to C0.

[0095] In the embodiments of the present application, the first sub deep feature map will be processed in the next level for deeper processing, and the second sub deep feature map will be reserved for subsequent feature splicing. By splitting the first deep feature map in the channel dimension, the number of channels of the feature map participating in subsequent calculation can be reduced, so that the computational load of subsequent processing can be reduced while the performance of the deep feature extraction network is maintained (guaranteed by subsequent feature splicing), and the processing efficiency is significantly improved.

[0096] Next, the first sub-depth feature map is subjected to depth feature extraction using a second feature processing module to obtain a second depth feature map.

[0097] Similar to the processing procedure of the first feature processing module, the first sub-depth feature map can be subjected to depth feature extraction using a second feature processing module in the embodiment of the application to obtain a second depth feature map. The dimension of the second depth feature map is consistent with that of the first sub-depth feature map, which can be expressed as [C1, H, W] for example.

[0098] Next, the second depth feature map is split into two parts in the channel dimension according to a second preset ratio to obtain a third sub-depth feature map and a fourth sub-depth feature map.

[0099] In the embodiment of the application, the second preset ratio can be set according to actual conditions, which is not limited here. The dimension of the third sub-depth feature map can be expressed as [C2, H, W] for example, and the dimension of the fourth sub-depth feature map can be expressed as [C1-C2, H, W] for example, where C2 represents the first C2 channels of the second depth feature map, and C2 is an integer greater than 0 and less than or equal to C1.

[0100] In the embodiment of the application, the third sub-depth feature map will be subjected to deeper processing in the next level, and the fourth sub-depth feature map is reserved for subsequent feature splicing. Similarly, by splitting the second depth feature map in the channel dimension, the number of channels of the feature map participating in subsequent calculation can be reduced, so that the calculation amount of subsequent processing can be reduced while the performance of the depth feature extraction network is maintained (guaranteed by subsequent feature splicing), and the processing efficiency is significantly improved.

[0101] Next, the third sub-depth feature map is subjected to depth feature extraction using a third feature processing module to obtain a third depth feature map.

[0102] Similarly, the third sub-depth feature map is subjected to depth feature extraction using a third feature processing module in the embodiment of the application to obtain a third depth feature map. The dimension of the third depth feature map is consistent with that of the third sub-depth feature map, which can be expressed as [C2, H, W] for example.

[0103] Next, the third depth feature map is processed based on a channel attention mechanism using a first channel attention module to obtain a weighted depth feature map.

[0104] It can be understood that, since the second super-resolution model obtained by the embodiment of the present application is applied to the scene of real-time super-resolution processing of tile data, in order to avoid the problem of “seam” appearing when the tile data processed by super-resolution is displayed on the user terminal, thereby affecting the display effect of the remote sensing image, the embodiment of the present application introduces a first channel attention module (for example, SE Block) before feature fusion, so as to perform global importance weighting on the third depth feature map output by the third feature processing module based on the channel attention mechanism, and generate a weighted depth feature map. The dimension of the weighted depth feature map is consistent with the dimension of the third depth feature map, which can be represented as [C2, H, W], for example.

[0105] In the embodiment of the present application, by using the first channel attention module to perform global importance weighting on the third depth feature map, the generated weighted depth feature map is aligned with the fourth sub-depth feature map to be spliced in the channel dimension, so as to effectively suppress the semantic discontinuity or splicing artifacts that may occur in the subsequent feature splicing process, thereby facilitating the improvement of the smoothness and consistency of multi-scale feature fusion.

[0106] Next, the weighted depth feature map and the fourth sub-depth feature map are spliced to obtain a first fusion feature map, and a fourth feature processing module is used to extract depth features from the first fusion feature map to obtain a first fusion enhanced feature map.

[0107] In the embodiment of the present application, by splicing the previously retained fourth sub-depth feature map and the weighted depth feature map, the fusion of fine-grained spatial information and rich semantic information is realized and the first fusion feature map is generated. The dimension of the first fusion feature map can be represented as [C1, H, W]. Then, the fourth feature processing module is used to refine the first fusion feature map to complete the depth fusion of cross-level features and generate the first fusion enhanced feature map. The dimension of the first fusion enhanced feature map is consistent with the dimension of the first fusion feature map, which can be represented as [C1, H, W]. The first fusion enhanced feature map not only retains accurate spatial detail information, but also has sufficient high-level semantic information, which can be fused with shallower features again in the subsequent process, thereby providing high-quality and high-expression feature input for the final feature reconstruction process.

[0108] It should be noted that the process of using the fourth feature processing module to extract depth features from the first fusion feature map is similar to the process of using the first feature processing module to extract depth features from the initial feature map described above, which will not be described here.

[0109] Next, the first fusion enhanced feature map and the second sub-depth feature map are spliced to obtain a second fusion feature map, and a fifth feature processing module is used to extract depth features from the second fusion feature map to obtain a second fusion enhanced feature map.

[0110] In the embodiment of the present application, the first fusion enhanced feature map and the previously reserved second sub-depth feature map are spliced to obtain a second fusion feature map. The dimension of the second fusion feature map can be represented as [C0, H, W]. Then, the fifth feature processing module is used to further refine the second fusion feature map, so as to realize the secondary depth fusion of global semantic information and local detail information, and obtain a second fusion enhanced feature map with the most delicate spatial detail information and the most rich high-level semantic information. The dimension of the second fusion enhanced feature map is consistent with that of the second fusion feature map, which can be represented as [C0, H, W].

[0111] It should be noted that the process of extracting depth features from the second fusion feature map by using the fifth feature processing module is similar to the process of extracting depth features from the initial feature map by using the first feature processing module described above, which will not be repeated here.

[0112] Next, the second channel attention module is used to process the second fusion enhanced feature map based on the channel attention mechanism to obtain a target fusion enhanced feature map.

[0113] In the embodiment of the present application, before the feature reconstruction (i.e., the reconstruction network processing), the second channel attention module (for example, SE Block) can be introduced to perform global importance weighting on the second fusion enhanced feature map based on the channel attention mechanism to generate a target fusion enhanced feature map. The dimension of the target fusion enhanced feature map is consistent with that of the second fusion enhanced feature map, which can be represented as [C0, H, W], for example. By using the second channel attention module to perform global importance calibration at the channel level on the second fusion enhanced feature map, the embodiment of the present application can further suppress useless or noise channels and enhance key channels with rich information. This provides a feature with better coordination and uniform scale for the subsequent feature reconstruction process, thereby avoiding the joint problem caused by discontinuous features.

[0114] At this point, the feature enhancement processing of the initial feature map by using the depth feature extraction network is completed, and a target fusion enhanced feature map is obtained.

[0115] In operation S304, the reconstruction network in the initial super-resolution model is used to perform feature reconstruction processing on the target fusion enhanced feature map to obtain a super-resolution image.

[0116] For example, the convolution layer in the reconstruction network described above can be used to transform the channel number of the target fusion enhanced feature map into a target channel number (for example, C_out*r 2where C_out is the number of channels of the output super-resolution image, and r is the upscaling factor of the super-resolution, e.g., 2). Then, the Depth-to-Space layer in the reconstruction network is used to rearrange the information of the feature map output by the convolution layer in the channel dimension to the spatial dimension, so as to realize the conversion of the high-channel, low-spatial-resolution feature map to the low-channel, high-spatial-resolution output, and obtain the super-resolution image.

[0117] In operation S305, a sample loss is obtained according to the difference between the super-resolution image and the reference image.

[0118] In the embodiments of the present application, the sample loss can be determined by using methods including but not limited to mean square error, cross entropy and L1 loss function, and the specific method can be selected according to actual needs, which is not limited here.

[0119] In operation S306, the model parameters of the initial super-resolution model are adjusted according to the sample loss to obtain a first super-resolution model.

[0120] In one example, the parameters of the shallow feature extraction network, the deep feature extraction network and the reconstruction network in the initial super-resolution model can be adjusted according to the sample loss to obtain the first super-resolution model.

[0121] In operation S307, the first super-resolution model is re-parameterized to obtain a second super-resolution model.

[0122] In the embodiments of the present application, in the inference stage, the second super-resolution model is obtained by re-parameterizing the to-be-processed modules in the first super-resolution model. The second super-resolution model has comparable performance with the first super-resolution model, but the parameter amount of the second super-resolution model is less than that of the first super-resolution model. The second super-resolution model can be used to perform super-resolution processing on the initial tile data to generate super-resolution tile data. Here, the to-be-processed modules include the first feature extraction sub-module and the second feature extraction sub-module included in the first feature processing module, the second feature processing module, the third feature processing module, the fourth feature processing module and the fifth feature processing module of the model after parameter adjustment.

[0123] For example, in the inference stage, the first feature extraction sub-module and the second feature extraction sub-module in the first feature processing module, the second feature processing module, the third feature processing module, the fourth feature processing module and the fifth feature processing module can be re-parameterized into the first convolution layer and the second convolution layer as shown in the following table. Figure 4A In the embodiments of the present application, by re-parameterizing these modules, the expression ability of the second super-resolution model can be ensured without increasing the time consumption in the inference, and the performance and processing efficiency of the model are balanced. The second super-resolution model can meet the real-time super-resolution processing requirements of the tile data.

[0124] Figure 5 is a block diagram of a remote sensing image loading device according to an embodiment of the present application.

[0125] As shown in Figure 5 , the remote sensing image loading device 500 includes a receiving module 510, an obtaining module 520, a super-resolution module 530, a resampling module 540 and a returning module 550.

[0126] The receiving module 510 is configured to receive a tile request of a user terminal, the tile request including tile parameters of a tile to be loaded.

[0127] The obtaining module 520 is configured to obtain initial tile data according to the tile parameters, the initial tile data having a first spatial resolution.

[0128] The super-resolution module 530 is configured to perform super-resolution processing on the initial tile data to obtain super-resolution tile data in response to determining that a zoom level corresponding to the initial tile data reaches a preset level according to the tile parameters, the super-resolution tile data having a second spatial resolution, the second spatial resolution being greater than the first spatial resolution.

[0129] The resampling module 540 is configured to perform resampling processing on the super-resolution tile data to obtain target tile data.

[0130] The returning module 550 is configured to return the target tile data to the user terminal.

[0131] It should be noted that the implementation of each module and the like in the device part embodiments, the technical problems solved, the functions implemented, and the technical effects achieved are the same as or similar to those of the implementation of each corresponding step in the method part embodiments, and will not be described here.

[0132] Figure 6 A block diagram of an electronic device suitable for implementing the remote sensing image loading method according to an embodiment of the present application is schematically shown.

[0133] As shown in Figure 6 , the electronic device 600 according to an embodiment of the present application includes a processor 601, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may, for example, include a general-purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chipset, and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), and the like. The processor 601 can also include an on-board memory for cache use. The processor 601 can include a single processing unit or a plurality of processing units for performing different actions of the method processes according to embodiments of the present application.

[0134] In the RAM 603, various programs and data required for the operation of the electronic device 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other via the bus 604. The processor 601 performs various operations of the method flow according to the embodiments of the present application by executing the programs in the ROM 602 and / or the RAM 603. It should be noted that the programs can also be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 can also perform various operations of the method flow according to the embodiments of the present application by executing the programs stored in the one or more memories.

[0135] According to the embodiments of the present application, the electronic device 600 can further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 can further include one or more of the following components connected to the I / O interface 605: an input part 606 including a keyboard, a mouse, etc.; an output part 607 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 608 including a hard disk, etc.; and a communication part 609 including a network interface card such as a LAN card, a modem, etc. The communication part 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as necessary. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 610 as necessary, so that a computer program read out therefrom is installed in the storage part 608 as necessary.

[0136] The present application also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the remote sensing image loading method according to the embodiments of the present application is realized.

[0137] According to an embodiment of the present application, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, can include but not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, the computer readable storage medium can include one or more memories of the ROM 602 and / or the RAM 603 described above and / or one or more memories other than the ROM 602 and the RAM 603.

[0138] An embodiment of the present application also includes a computer program product, which includes a computer program containing program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the remote sensing image loading method provided by the embodiment of the present application.

[0139] The above functions defined in the system / device of the embodiment of the present application are executed when the computer program is executed by the processor 601. According to an embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0140] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, downloaded and installed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from the detachable medium 611. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any suitable combination of the foregoing.

[0141] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the detachable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present application are executed. According to an embodiment of the present application, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0142] According to embodiments of the present application, program code for implementing the computer programs provided by embodiments of the present application can be written in any combination of one or more programming languages, and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming language can include, but is not limited to, Java, C++, python, "C" language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0143] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0144] Those skilled in the art will appreciate that the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways, even if such combinations or integrations are not expressly noted in the present application. In particular, the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways without departing from the spirit and scope of the present application. All such combinations and / or integrations are within the scope of the present application.

[0145] The embodiments of the present application have been described above. However, these embodiments are merely for the purpose of illustration and are not intended to limit the scope of the present application. Although the respective embodiments are described above separately, this does not mean that the measures in the respective embodiments cannot be used advantageously in combination. Numerous alternatives and modifications of embodiments of the present application can be made by those skilled in the art without departing from the scope of the present application, and all such alternatives and modifications are to be included within the scope of the present application.

Claims

1. A remote sensing image loading method, comprising: receiving a tile acquisition request of a user terminal, the tile acquisition request comprising tile parameters of a tile to be loaded; acquiring initial tile data according to the tile parameters, the initial tile data having a first resolution; in response to determining that a zoom level corresponding to the initial tile data reaches a preset level according to the tile parameters, performing super-resolution processing on the initial tile data to obtain super-resolution tile data, the super-resolution tile data having a second resolution, the second resolution being greater than the first resolution; performing resampling processing on the super-resolution tile data to obtain target tile data; returning the target tile data to the user terminal.

2. The method of claim 1, wherein, The super-resolution processing on the initial tile data to obtain super-resolution tile data comprises: performing super-resolution processing on the initial tile data using a second super-resolution model to obtain super-resolution tile data; wherein the second super-resolution model is obtained by reparameterizing a first super-resolution model, and the first super-resolution model is obtained by model training on an initial super-resolution model using a sample image pair.

3. The method of claim 2, wherein, The initial super-resolution model comprises a shallow feature extraction network, a deep feature extraction network, and a reconstruction network; and the first super-resolution model is trained in the following manner: acquiring a sample image pair comprising a sample image and a reference image corresponding to the sample image, wherein the sample image is obtained by performing resolution reduction processing on the reference image; performing shallow feature extraction on the sample image using the shallow feature extraction network to obtain an initial feature map; performing feature enhancement processing on the initial feature map using the deep feature extraction network to obtain a target fusion enhanced feature map; performing feature reconstruction processing on the target fusion enhanced feature map using the reconstruction network to obtain a super-resolution image; obtaining a sample loss according to the difference between the super-resolution image and the reference image; adjusting model parameters of the initial super-resolution model according to the sample loss to obtain the first super-resolution model.

4. The method of claim 3, wherein, The deep feature extraction network comprises a first feature processing module, a second feature processing module, a third feature processing module, a first channel attention module, a fourth feature processing module, a fifth feature processing module, and a second channel attention module; The feature enhancement processing on the initial feature map using the deep feature extraction network to obtain a target fusion enhanced feature map comprises: performing deep feature extraction on the initial feature map using the first feature processing module to obtain a first deep feature map, and splitting the first deep feature map into two parts in the channel dimension according to a first preset ratio to obtain a first sub deep feature map and a second sub deep feature map; performing deep feature extraction on the first sub deep feature map using the second feature processing module to obtain a second deep feature map, and splitting the second deep feature map into two parts in the channel dimension according to a second preset ratio to obtain a third sub deep feature map and a fourth sub deep feature map; performing deep feature extraction on the third sub-depth feature map using the third feature processing module to obtain a third deep feature map; performing processing on the third deep feature map based on a channel attention mechanism using the first channel attention module to obtain a weighted deep feature map; performing splicing on the weighted deep feature map and the fourth sub-depth feature map to obtain a first fusion feature map, and performing deep feature extraction on the first fusion feature map using the fourth feature processing module to obtain a first fusion enhanced feature map; performing splicing on the first fusion enhanced feature map and the second sub-depth feature map to obtain a second fusion feature map, and performing deep feature extraction on the second fusion feature map using the fifth feature processing module to obtain a second fusion enhanced feature map; performing processing on the second fusion enhanced feature map based on a channel attention mechanism using a second channel attention module to obtain the target fusion enhanced feature map.

5. The method of claim 4, wherein, The first feature processing module comprises a first feature extraction sub-module, a second feature extraction sub-module, and a spatial attention sub-module; The first feature processing module comprises a first feature extraction sub-module, a second feature extraction sub-module, and a spatial attention sub-module; performing local feature extraction on the initial feature map using the first feature extraction sub-module to obtain an initial deep feature map; performing local feature extraction on the initial deep feature map using the second feature extraction sub-module to obtain an intermediate deep feature map; performing processing on the intermediate deep feature map based on a spatial attention mechanism using the spatial attention sub-module to obtain the first deep feature map.

6. The method of claim 5, wherein, The spatial attention sub-module comprises a local importance estimation unit, a gating signal generation unit, and a fusion unit; The spatial attention sub-module comprises a local importance estimation unit, a gating signal generation unit, and a fusion unit; performing local feature extraction on the initial deep feature map using the second feature extraction sub-module to obtain an intermediate deep feature map; performing processing on the intermediate deep feature map based on a spatial attention mechanism using the spatial attention sub-module to obtain the first deep feature map. The first feature extraction sub-module comprises a first channel transformation unit, a scaling unit, a spatial convolution unit, a second channel transformation unit, and a residual connection unit; 7. The method of claim 5, wherein, The first feature extraction sub-module comprises a first channel transformation unit, a scaling unit, a spatial convolution unit, a second channel transformation unit, and a residual connection unit; performing channel expansion on the initial feature map using the first channel transformation unit, and performing per-channel scaling processing on the initial feature map after channel expansion using the scaling unit to obtain a scaled feature map; performing spatial feature extraction on the scaled feature map using the spatial convolution unit to obtain a convolution feature map; performing spatial feature extraction on the scaled feature map using the spatial convolution unit to obtain a convolution feature map; The second channel transformation unit is used for channel compression on the convolution feature map, to obtain a channel-compressed convolution feature map, the channel number of the channel-compressed convolution feature map being the same as the channel number of the initial feature map; The residual connection unit is used for splicing the initial feature map and the channel-compressed convolution feature map, to obtain the initial deep feature map.

8. The method of claim 5, wherein, The second super-resolution model is obtained by performing re-parameterization processing on a to-be-processed module in the first super-resolution model, and the parameter quantity of the second super-resolution model is less than the parameter quantity of the first super-resolution model. The to-be-processed module includes first feature processing modules, second feature processing modules, third feature processing modules, fourth feature processing modules, and fifth feature processing modules each containing first feature extraction sub-modules and second feature extraction sub-modules in a model parameter-adjusted deep feature extraction network.

9. A remote sensing image loading device, comprising: A receiving module configured to receive a tile acquisition request of a user terminal, the tile acquisition request including tile parameters of a to-be-loaded tile; An obtaining module configured to obtain initial tile data according to the tile parameters, the initial tile data having a first resolution; A super-resolution module configured to perform super-resolution processing on the initial tile data to obtain super-resolution tile data in response to determining that a zoom level corresponding to the initial tile data reaches a preset level according to the tile parameters, the super-resolution tile data having a second resolution, the second resolution being greater than the first resolution; A resampling module configured to perform resampling processing on the super-resolution tile data to obtain target tile data; A returning module configured to return the target tile data to the user terminal.

10. An electronic device, comprising: one or more processors; a memory configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1-8.

11. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1-8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images