Shearlet prior-based fine-grained representation remote sensing image interpretation method and system

Abstract

The application discloses a fine-grained representation remote sensing image interpretation method and system based on Shearlet prior knowledge, adopts Shearlet transformation to extract high-frequency sub-band information of preprocessed remote sensing image data; uses Laplace pyramid and shearlet filter to obtain high-frequency components and multi-direction high-frequency sub-bands after directional splitting; through a spatial attention mechanism, sub-bands of the same level are adaptively fused to generate a detail-enhanced feature map; high-frequency components extracted through Shearlet transformation are prior to multi-scale and multi-level weighted fusion of deep residual blocks obtained through a deep residual network; a trained detail-enhanced backbone and different detection heads of multiple tasks are used to output accurate remote sensing image analysis results; through combination of Shearlet transformation and an advanced deep learning architecture, the accuracy and efficiency of remote sensing image interpretation are significantly improved, meanwhile, good model generalization ability and practicability are maintained, and it is expected that the application will promote the development of remote sensing image processing in a more intelligent and automatic direction.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning and image analysis, and specifically relates to a fine-grained representation remote sensing image interpretation method and system based on Shearlet prior knowledge. BACKGROUND

[0002] Remote sensing technology has been an important tool for Earth observation since the mid-20th century, especially in the application of satellite and aerial images. These technologies can provide quantitative information about the Earth's surface and atmosphere, which is crucial for climate change research, environmental protection, agricultural monitoring, and urban planning. With the development of remote sensing technology, the resolution and acquisition frequency of images have significantly improved, but at the same time, it has brought challenges such as large data volume and high information complexity.

[0003] Despite the development of remote sensing technology, which brings more data and higher image quality, existing image analysis methods often struggle to effectively process these data. For example, classification methods based on traditional feature extraction rely on manually selected features, which not only consume time and effort but also have limited effectiveness due to the representativeness and comprehensiveness of the selected features. In addition, traditional methods often fail to achieve high recognition accuracy when dealing with high-resolution remote sensing images due to the lack of algorithmic ability to handle the complexity of the images.

[0004] In recent years, deep learning, especially convolutional neural networks (CNN), has been proven to be excellent in image recognition and classification tasks. However, when applied to remote sensing image analysis, these models are often limited by their lack of ability to capture high-frequency details. The unique properties of remote sensing images, such as large-scale changes, complex textures, and structural features, make it difficult for standard CNN models to be directly applicable.

[0005] Shearlet transform provides an effective multi-scale, multi-directional analysis method, especially suitable for processing image data with anisotropic characteristics. By accurately capturing high-frequency features such as edges and cracks in images, Shearlet transform can provide more detailed image analysis than traditional wavelet transform. In addition, the nonlinear decomposition ability of Shearlet transform allows it to reveal hidden, non-explicit information levels in images, which is particularly valuable for analyzing complex remote sensing data.

[0006] 5. Combining the high-frequency feature decomposition ability of Shearlet transform with the strong learning ability of neural networks can greatly improve the accuracy and efficiency of remote sensing image analysis. Through this integration, not only can the network's perception of details be enhanced, but also the model's ability to analyze complex scenes can be improved while maintaining the integrity of the deep semantic of the image. This new method is expected to drive the development of remote sensing image processing from traditional methods to more intelligent and automated directions. SUMMARY

[0007] The technical problem to be solved by the present application is to provide a fine-grained representation remote sensing image interpretation method and system based on Shearlet prior knowledge, which integrates spatial attention into the learning process of the neural network, optimizes the network's ability to analyze complex remote sensing data, and solves the technical problem of missing fine-grained structure information in traditional deep learning methods with the deepening of convolutional layers.

[0008] The present application adopts the following technical solutions:

[0009] A fine-grained representation remote sensing image interpretation method based on Shearlet prior knowledge, comprising the following steps:

[0010] S1, performing a preprocessing operation on the original remote sensing image data;

[0011] S2, using Shearlet transform to extract multi-level and multi-direction high-frequency sub-band information from the remote sensing image data preprocessed in step S1; using Laplacian pyramid and shearlet filter to obtain high-frequency components and multi-direction high-frequency sub-bands after splitting them by direction, respectively;

[0012] S3, constructing a high-frequency component prior based on Shearlet transform, and adaptively fusing sub-bands of the same level through a spatial attention mechanism to generate a detail-enhanced feature map;

[0013] S4, constructing a detail-enhanced backbone network based on ResNet, and performing multi-scale and multi-level weighted fusion on the high-frequency component prior extracted by Shearlet transform in step S3 and the deep residual blocks obtained by the deep residual network;

[0014] S5, training the detail-enhanced backbone network based on ResNet obtained in step S4 using a supervised learning method, and repeatedly iterating using the remote sensing image data labeled in step S1 until the downstream task model performs stably on the validation set;

[0015] S6, using the detail-enhanced backbone trained in step S5 and multiple task detection heads to output remote sensing image analysis results.

[0016] Preferably, in step S1, the preprocessing operation includes noise removal, contrast adjustment and image cropping.

[0017] Preferably, step S2 is specifically:

[0018] S201, inputting the preprocessed remote sensing image and performing color space conversion on the image from RGB to HSI;

[0019] S202, applying a Laplacian pyramid method to the I component of the remote sensing image to perform high and low frequency information layering, and extracting the highest frequency component of the image;

[0020] S203, for the extracted high frequency component, using a Shearlet filter to obtain a multi-directional highest frequency sub-band;

[0021] S204, down-sampling the remaining low frequency information to half, applying the Laplacian pyramid method again to decompose the high and low frequency information, and obtaining a multi-directional sub-band.

[0022] Preferably, in step S202, the Shearlet transform details are as follows:

[0023]

[0024] wherein, and represent a spatial vector and a translation vector, respectively, is a scaling parameter, denotes a shear factor, is an expansion matrix, is a shear matrix, is a transform domain.

[0025] Preferably, steps S201 to S204 are repeated three times until the highest frequency sub-band, the sub-high frequency sub-band and the higher frequency sub-band information are obtained.

[0026] Preferably, step S3 is specifically:

[0027] S301, by dimension rearrangement of the high frequency sub-band extracted for three times, the input format CxHxW of the deep learning model is prepared, wherein C is the channel number, H is the height, and W is the width; each channel C receives all directional sub-bands corresponding to one extraction, representing directional information at a specific scale;

[0028] S302, using a spatial attention module based on Shearlet transform to fuse the multi-directional sub-bands;

[0029] S303, the spatial attention module uses a convolution layer to enhance the learning ability of the model to spatial texture information by using a ReLU activation function; then a second convolution layer deepens feature fusion by using a Sigmoid activation function, generates a compressed feature map focusing on key information of the image, and the final output of the Shearlet attention feature map has a dimension of 1xHxW.

[0030] Preferably, step S4 is specifically:

[0031] S401, construct ResNet50 as a basic network, ResNet50 has five CNN residual blocks in total, which are the zeroth layer, the first layer, the second layer, the third layer and the fourth layer respectively;

[0032] S402, adjust the feature map size obtained in step S3, so that the highest frequency feature map and the second layer CNN residual block are the same size, the second highest frequency feature map and the third layer CNN residual block are the same size, and the higher frequency feature map and the fourth layer CNN residual block are the same size;

[0033] S403, multiply the high-frequency prior feature map adjusted in step S402 with the CNN deep residual block feature output by the ResNet backbone network in a multi-scale pixel-by-pixel manner, to realize effective integration of the two types of features.

[0034] Preferably, step S5 is specifically:

[0035] The neural network is trained in a supervised learning manner, and the labeled remote sensing data is used for repeated iteration; after the neural network is trained, the performance of the neural network is evaluated.

[0036] Preferably, step S6 is specifically:

[0037] S601, using the trained detail enhancement backbone network to capture discriminative features in remote sensing images;

[0038] S602, integrating a plurality of task detection heads with the detail enhancement backbone network to obtain a complete remote sensing multi-task processing model;

[0039] S603, model integration, the multi-task remote sensing image interpretation model obtained in step S602 is outputted.

[0040] In a second aspect, the embodiments of the present application provide

[0041] A preprocessing module performs a preprocessing operation on original remote sensing image data;

[0042] A Shearlet transform module adopts Shearlet transform to extract multi-level and multi-direction high-frequency sub-band information from the preprocessed remote sensing image data; a Laplacian pyramid and a shearlet filter are used to obtain high-frequency components and multi-direction high-frequency sub-bands after directional splitting of the high-frequency components respectively;

[0043] A spatial attention module constructs a high-frequency component prior based on Shearlet transform, adaptively fuses sub-bands of the same level through a spatial attention mechanism, and generates a detail-enhanced feature map;

[0044] The detail enhancement backbone network module constructs a detail enhancement backbone network based on ResNet, and performs multi-scale and multi-level weighted fusion on the high-frequency components extracted by the Shearlet transform and the deep residual blocks obtained by the deep residual network.

[0045] The training and output module trains the detail enhancement backbone network based on ResNet obtained by the supervised learning method, repeatedly iterates using the labeled remote sensing image data until the downstream task model performs stably on the validation set, and outputs the remote sensing image analysis result using the trained detail enhancement backbone and different detection heads of multiple tasks.

[0046] In a third aspect, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned Shearlet-prior-based fine-grained representation remote sensing image interpretation method when executing the computer program.

[0047] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium including a computer program, and the computer program implements the steps of the above-mentioned Shearlet-prior-based fine-grained representation remote sensing image interpretation method when executed by a processor.

[0048] Compared with the prior art, the present application has at least the following beneficial effects:

[0049] A Shearlet-prior-based fine-grained representation remote sensing image interpretation method, the hearlet transform as a multi-scale and multi-directional geometric analysis tool can effectively capture the detail scale and directional features in the image, especially the edge details and other high-frequency parts. The output of this transform is input into the neural network as multi-scale and multi-level prior knowledge, which greatly improves the performance of the network in remote sensing image analysis tasks, especially in semantic segmentation, target detection and tracking applications; by adopting the Shearlet prior knowledge, the high-frequency detail features in the image can be effectively captured, which is often not achieved in traditional wavelet transform methods. The multi-scale and multi-directional capability of Shearlet transform is particularly suitable for processing remote sensing images with complex texture and edges, thereby significantly improving the accuracy and efficiency of image interpretation.

[0050] Further, the preprocessing operation includes noise removal, contrast adjustment and image cropping, aiming to improve the quality of the original image data and provide clearer and more consistent input data for subsequent high-frequency feature extraction and analysis. This step reduces the error and complexity in subsequent processing, ensuring the stability and reliability of the system.

[0051] Further, through the Shearlet transform, the high-frequency sub-band information of the preprocessed image is extracted, which not only improves the resolution of the image and the visibility of the edge information, but also reveals the micro and detailed features in the image. This method is especially suitable for analyzing high-resolution remote sensing data, thereby improving the accuracy of classification and identification.

[0052] Further, a high-frequency component prior based on the Shearlet transform is constructed, and the spatial attention mechanism is used to adaptively fuse sub-bands of the same level, so that the key areas in the image can be focused on more effectively, the response of the model to important parts of the image is enhanced, and the interpretation quality of the remote sensing image is improved.

[0053] Further, the construction of the detail enhancement backbone network realizes the effective integration of high-frequency prior features and deep features by combining with the deep residual network. This strategy optimizes the learning ability of the network, especially when dealing with complex remote sensing scenes, the integrity of the deep semantic features of the image can be maintained while the detailed features are emphasized.

[0054] Further, the supervised learning training of the detail enhancement backbone network enables the model to perform stably on the validation set after iterative training, which guarantees the reliability and generalization ability of the model in practical applications.

[0055] Further, using the trained detail enhancement backbone and multi-task detection head, the model can output accurate remote sensing image analysis results, which is suitable for various complex application scenarios such as land cover classification and target detection, and shows its wide application potential and practical value.

[0056] It can be understood that the beneficial effects of the above-mentioned second aspect can be referred to the related description in the above-mentioned first aspect, which will not be repeated here.

[0057] In summary, by combining the Shearlet transform and advanced deep learning architecture, the present application significantly improves the accuracy and efficiency of remote sensing image interpretation, while maintaining good model generalization ability and practicality, and is expected to promote the development of remote sensing image processing towards a more intelligent and automated direction.

[0058] The technical solutions of the present application will be further described in detail below with the aid of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 A Shearlet prior knowledge-based fine-grained representation remote sensing image method flowchart is provided for the embodiments of the present application.

[0060] Figure 2A schematic diagram of using a Laplacian pyramid to split high and low frequency information and obtain multi-directional high frequency subbands in Shearlet transform provided for an embodiment of the present application;

[0061] Figure 3 A flowchart of adaptively fusing high frequency multi-directional subbands under the same scale in Shearlet transform provided for an embodiment of the present application;

[0062] Figure 4 A schematic diagram of a fine-grained enhanced backbone ResNet provided for an embodiment of the present application.

[0063] Figure 5 A schematic diagram of a computer device provided for an embodiment of the present application;

[0064] Figure 6 A block diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of the present application.

[0066] In the description of the present application, it should be understood that the terms "comprise" and "include" indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or sets thereof.

[0067] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, unless otherwise clearly indicated by the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0068] It should be further understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.

[0069] It should be understood that, although the terms first, second, third, etc. can be employed in describing the preset ranges, etc. in the embodiments of the present application, the preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from each other. For example, the first preset range can also be referred to as the second preset range, and similarly, the second preset range can also be referred to as the first preset range without departing from the scope of the embodiments of the present application.

[0070] Depending on the context, the word "if" as used herein can be interpreted to mean "when" or "while" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted to mean "when it is determined" or "in response to determining" or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)".

[0071] Various structural diagrams according to the disclosed embodiments of the present application are shown in the accompanying drawings. These diagrams are not drawn to scale, in which certain details are exaggerated for the purpose of clarity and some details may be omitted. The shapes of various regions, layers and their relative sizes and positional relationships shown in the drawings are only exemplary, and in actuality, they can deviate due to manufacturing tolerances or technical limitations, and a person skilled in the art can additionally design regions / layers with different shapes, sizes and relative positions according to actual needs.

[0072] The present application provides a fine-grained representation remote sensing image interpretation method based on Shearlet prior knowledge, aiming to improve the accuracy of remote sensing image analysis; by applying Shearlet transform to extract high-frequency features from remote sensing images, the present application can highlight the fine-grained structure information in the image, which is usually missed in traditional deep learning methods as the convolution layer deepens; the high-frequency feature emphasis mechanism of the present method integrates into the learning process of the neural network in the form of spatial attention, thereby optimizing the network's ability to analyze complex remote sensing data; in technical implementation, Shearlet transform, as a multi-scale and multi-directional geometric analysis tool, can effectively capture the detail scale and directional features in the image, especially the edge details and other high-frequency parts. Inputting the output of this transform as multi-scale, multi-level prior knowledge into the neural network greatly improves the network's performance in remote sensing image analysis tasks, especially in semantic segmentation, target detection and tracking applications.

[0073] The fine-grained representation remote sensing image interpretation method based on Shearlet prior knowledge of the present application comprises the following steps:

[0074] S1. Receive raw remote sensing image data and perform a series of preprocessing operations on it, including noise removal, contrast adjustment and image cropping, to ensure the quality and consistency of the data.

[0075] S2. Shearlet transform is used to extract multi-level high-frequency subband information from the preprocessed remote sensing image.

[0076] The high-frequency components and their directional splitting into multi-directional high-frequency subbands were obtained using the Laplace pyramid and shearlet filters, respectively.

[0077] S201. Input the preprocessed remote sensing image and perform color space conversion on the image, converting it from RGB to HSI;

[0078] S202. Apply the Laplace pyramid method to divide the I component of the remote sensing image into high and low frequency information layers and extract the highest frequency component of the image.

[0079] The details of the Shearlet transform are as follows:

[0080]

[0081] in, and They represent spatial vectors and translation vectors, respectively. It's a scaling parameter. This represents the shearing factor. The shearing transformation is performed using the expansion matrix defined below. and shearing matrix To adjust the scale and direction:

[0082]

[0083] S203. For the extracted high-frequency components, use a Shearlet filter to obtain the highest frequency subband in multiple directions;

[0084] The specific process is as follows: The spatial domain Shearlet filter is determined by the scale. Cut and cone area Parameterization; shear parameters for each cone region In scale The range above Extended to ,in =0.5; This configuration effectively divides the frequency domain into different directional segments for processing.

[0085] In the frequency domain, the support for Shearlet filters is similar to that of trapezoids.

[0086] Cone parameters is 0, 1 or 2, determining the directionality of the filter in the frequency domain: for a low frequency filter, emphasizing horizontal features (horizontal direction) while focusing on vertical structures (vertical direction).

[0087] When and taking values in {-1, 0, 1}, the configurations from 1 to 7 are (0, 0, 0) (i.e. low pass component), (0, -1, h), (0, 0, h), (0, 1, h), (0, 1, v), (0, 0, v) and (0, -1, v), respectively, as shown in Figure 3 This structured and detailed approach enhances the ability of the Shearlet transform to optimize feature extraction and noise management in complex image processing tasks by effectively exploiting scale, shear and cone parameters.

[0088] S204, downsample the remaining low frequency information to half of the previous, and apply the Laplacian pyramid method again to decompose the high and low frequency information, and obtain the multi-directional sub-band of the next high frequency.

[0089] The above process is performed three times until the highest frequency sub-band, the next high frequency sub-band and the higher frequency sub-band information are obtained.

[0090] S3, construct a high frequency prior based on Shearlet transform, after multi-level sub-band extraction, use a spatial attention mechanism to adaptively fuse sub-bands of the same level;

[0091] S301, through dimension rearrangement of the high frequency sub-band extracted by 3 times of recursion, prepare for the input format CxHxW of the deep learning model, wherein C is the number of channels, H is the height, and W is the width; each channel C receives all directional sub-bands of the corresponding extraction, which represents the directional information at a specific scale;

[0092] S302, use a spatial attention module based on Shearlet transform to fuse the multi-directional sub-bands;

[0093] Through this method, the model can generate a single-channel attention feature map, which highlights the most important feature area in the image, which is of discriminative significance for analyzing remote sensing images.

[0094] S303, the spatial attention module uses a convolution layer (Conv1) to enhance the learning ability of the model to spatial texture information through a ReLU activation function; then the second convolution layer (Conv2) deepens the feature fusion through a Sigmoid activation function, generating a compressed feature map focusing on the key information of the image. The final output of the Shearlet attention feature map has a dimension of 1xHxW.

[0095] S4, construct a ResNet-based detail enhancement backbone, and perform multi-scale and multi-level weighted fusion of the high-frequency prior extracted by the Shearlet transform and the deep residual block obtained by the network, so as to enhance the perception depth and complexity of the neural network without sacrificing the deep semantic of the image;

[0096] S401, construct ResNet50 as a basic network to perform multi-scale feature extraction and obtain multi-scale information of different receptive fields;

[0097] ResNet50 has a total of five CNN residual blocks, namely: the zeroth layer, the first layer, the second layer, the third layer, and the fourth layer.

[0098] S402, adjust the size of the feature map obtained in step S303, so that the highest frequency feature map and the second layer CNN residual block are the same size, the second highest frequency feature map and the third layer CNN residual block are the same size, and the higher frequency feature map and the fourth layer CNN residual block are the same size;

[0099] S403, multiply the high-frequency prior feature map adjusted in step S402 with the CNN deep residual block feature output by the ResNet backbone network in a multi-scale and pixel-by-pixel manner to realize effective integration of the two types of features.

[0100] S5, the combined network is trained in a supervised learning manner, and labeled remote sensing data is used for repeated iteration until the model performs stably on the validation set; after the model training is completed, a comprehensive performance evaluation is performed, including the calculation of key indicators in each downstream task;

[0101] S501, train the neural network, and the network is trained in a supervised learning manner and iterated repeatedly using labeled remote sensing data.

[0102] This process continues until the model performs to the predetermined stability standard on the validation set.

[0103] S502, once the neural network training is completed, the next stage is entered;

[0104] In this stage, the model stops iteration and is ready for the next performance evaluation;

[0105] S503, after the model training is completed, a comprehensive performance evaluation is performed on the network.

[0106] Including calculating the key performance indicators of the model in each downstream task, such as accuracy, recall rate, and F1 score, etc., to ensure that the model meets the needs of practical applications.

[0107] S6, using the trained detail enhancement backbone and multiple task-specific probes, the model outputs accurate remote sensing image analysis results, supporting specific application scenarios such as land cover classification and object detection.

[0108] S601, using the trained detail enhancement backbone network, which is specifically designed to capture discriminative features in remote sensing images;

[0109] This enhanced feature capture capability is achieved by combining Shearlet transform with deep residual learning, ensuring that the model can effectively handle high-complexity remote sensing data.

[0110] S602, integrate multiple specially designed task probes with the detail enhancement backbone network;

[0111] These probes are optimized for different application scenarios, such as land cover classification and object detection, allowing the model to demonstrate its powerful analysis capabilities in multiple fields.

[0112] S603, the model integrates all the above functions and outputs accurate remote sensing image analysis results.

[0113] Those skilled in the art can understand that various aspects of the present application can be implemented as systems, methods or program products. Therefore, various aspects of the present application can be specifically implemented as the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or hardware and software combined implementation, which can be collectively referred to as "circuit", "module" or "platform".

[0114] In another embodiment of the present application, a remote sensing image interpretation system based on Shearlet prior knowledge is provided, which can be used to implement the above-mentioned remote sensing image interpretation method based on Shearlet prior knowledge. Specifically, the remote sensing image interpretation system based on Shearlet prior knowledge includes a preprocessing module, a Shearlet transform module, a spatial attention module, a detail enhancement backbone network module, and a training and output module.

[0115] The preprocessing module performs preprocessing operations on the original remote sensing image data.

[0116] The Shearlet transform module uses Shearlet transform to extract multi-level, multi-direction high-frequency sub-band information from the preprocessed remote sensing image data; uses Laplacian pyramid and shearlet filter to obtain high-frequency components and multi-direction high-frequency sub-bands after direction splitting, respectively.

[0117] The spatial attention module constructs a high-frequency component prior based on a Shearlet transform, adaptively fuses subbands of the same level through a spatial attention mechanism, and generates a feature map with enhanced details.

[0118] The detail enhancement backbone network module constructs a detail enhancement backbone network based on a ResNet, performs multi-scale and multi-level weighted fusion between the high-frequency component prior extracted by the Shearlet transform and deep residual blocks obtained by the deep residual network, and outputs a feature map with enhanced details.

[0119] The training and output module trains the detail enhancement backbone network based on the ResNet by using a supervised learning method, repeatedly iterates using labeled remote sensing image data until a downstream task model performs stably on a validation set, and outputs remote sensing image analysis results by using the trained detail enhancement backbone and different detection heads of multiple tasks.

[0120] In another embodiment of the present application, a terminal device is provided, which comprises a processor and a memory, the memory is used to store a computer program, the computer program comprises program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), ready-to-program gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to realize corresponding method processes or corresponding functions. The processor in the embodiments of the present application can be used for the operation of the fine-grained representation remote sensing image interpretation method based on the Shearlet prior knowledge, which comprises:

[0121] The original remote sensing image data is preprocessed, the preprocessed remote sensing image data is extracted in multiple levels and multiple directions by using a shearlet transform, a Laplacian pyramid and a shearlet filter are used to obtain high-frequency components and multi-directional high-frequency subbands after directional splitting of the high-frequency components respectively, a high-frequency component prior based on the shearlet transform is constructed, spatial attention mechanisms are used to adaptively fuse subbands in the same level to generate a feature map with enhanced details, a ResNet-based detail enhancement backbone network is constructed, high-frequency component priors extracted by the shearlet transform and deep residual blocks obtained by the ResNet are weighted and fused in multiple scales and multiple levels, the ResNet-based detail enhancement backbone network trained by using a supervised learning method is used for repeated iteration by using labeled remote sensing image data until a downstream task model performs stably on a validation set, and the trained detail enhancement backbone and different detection heads of multiple tasks are used to output remote sensing image analysis results.

[0122] In another embodiment of the present application, the present application further provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in a terminal device, used for storing programs and data. It can be understood that the computer readable storage medium herein can include an internal storage medium in the terminal device, and of course can also include an expansion storage medium supported by the terminal device, and can be any tangible medium containing or storing programs, which can be used by or in combination with an instruction execution system, device or apparatus. The computer readable storage medium provides a storage space, which stores an operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that more specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection with one or more conductive wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0123] Computer readable storage media further includes data signals transported through a carrier wave and a communications medium, including or excluding wired storage media.

[0124] The program code can be executed by using any combination of one or more programming languages including an object oriented programming language such as Java, C++ or the like, and a conventional procedural programming language such as a "C" language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0125] The one or more instructions stored in the computer readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the method for interpreting remote sensing image based on Shearlet prior knowledge in the above embodiments; the one or more instructions stored in the computer readable storage medium are loaded and executed by the processor to implement the following steps:

[0126] The original remote sensing image data is preprocessed; the preprocessed remote sensing image data is subjected to multi-level and multi-directional high-frequency sub-band information extraction by Shearlet transform; the Laplacian pyramid and the shearlet filter are used to obtain the high-frequency components and the multi-directional high-frequency sub-bands after directional splitting, respectively; a high-frequency component prior based on Shearlet transform is constructed, the sub-bands at the same level are adaptively fused through a spatial attention mechanism to generate a feature map with enhanced details; a detail enhancement backbone network based on ResNet is constructed, the high-frequency component prior extracted by Shearlet transform is weighted and fused with deep residual blocks obtained by the depth residual network in a multi-scale and multi-level manner; the detail enhancement backbone network based on ResNet obtained by supervised learning is trained, and the labeled remote sensing image data is used for repeated iteration until the downstream task model performs stably on the validation set; the trained detail enhancement backbone and different detection heads of multiple tasks are used to output remote sensing image analysis results.

[0127] Please refer to Figure 5 , the terminal device is a computer device, the computer device 60 of the embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61, and the computer program 63 realizes the Shearlet prior-based fine-grained representation remote sensing image interpretation method in the embodiment when executed by the processor 61, to avoid repetition, which will not be described here. Alternatively, the computer program 63 is executed by the processor 61 to realize the functions of each model / unit in the Shearlet prior-based fine-grained representation remote sensing image interpretation system of the embodiment, to avoid repetition, which will not be described here.

[0128] The computer device 60 can be a desktop computer, a notebook computer, a palm computer, and a cloud server, etc. The computer device 60 can include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art can understand that Figure 5 is only an example of the computer device 60 and does not constitute a limitation on the computer device 60, and can include more or fewer components than shown, or combine certain components, or different components, for example, the computer device can also include an input / output device, a network access device, a bus, etc.

[0129] The processor 61 can be a central processing unit (CPU), and can also be other general-purpose processors, central processing units, graphics processing units, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, quantum computing-based data processing logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0130] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or a memory of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 60.

[0131] Further, the memory 62 can include both an internal storage unit and an external storage device of the computer device 60. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0132] Any reference to storage, databases or other media used to store data in the embodiments provided herein is intended to include at least one of volatile and non-volatile storage. Non-volatile storage can include, for example, optical, floppy disks, hard disks, or solid state drives. Volatile storage can include, for example, random access memory (RAM). A basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within the electronic device, such as during startup, can typically be stored in non-volatile memory. By way of illustration, and not limitation, a basic input / output system based on the BIOS, can include a BIOS, a unified extensible firmware interface (UEFI), or the like, including open firmware, firmware option ROM, flash BIOS, or the like. RAM typically contains data and / or program modules that are immediately accessible to and / or being operated on by the processing unit(s) and can include, for example, operating system, application programs, other program modules, and program data.

[0133] The database referred to in the embodiments provided herein can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, and the like, but is not limited thereto. The processor referred to in the embodiments provided herein can be a general processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like, but is not limited thereto.

[0134] Referring to Figure 6 , the terminal device 600 is an electronic device, which is manifested in the form of a general computing device. The components of the electronic device can include, but are not limited to, at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components including the storage unit 620 and the processing unit 610, a display unit 640, and the like.

[0135] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present application described in the method part of the present specification. For example, the processing unit 610 can perform the steps as shown in Figure 1 .

[0136] The storage unit 620 can include a readable medium in the form of volatile storage such as random access memory (RAM) 6201 and / or cache memory 6202, and also can include a non-volatile storage such as read only memory (ROM) 6203.

[0137] The storage unit 620 also can include a program / utility 6204 having a set of programs / modules 6205, including an operating system, one or more application programs, other program modules, and program data, each of which can implement aspects of a network environment, as each of these example or some combination thereof.

[0138] The bus 630 can represent one or more of several types of bus structures, including a storage bus or bus controller, a peripheral bus, a graphics bus, a processor or local bus using any of a variety of bus architectures.

[0139] The electronic device 600 also can communicate with one or more external devices 700 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices that enable a user to interact with the electronic device 600; and / or one or more devices that enable the electronic device 600 to communicate with one or more other computing devices. Such communication can occur via an input / output (I / O) interface 650. Still yet, the electronic device 600 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network such as the Internet, via a network adapter 660. The network adapter 660 can be communicatively coupled to the other components of the electronic device 600 via the bus 630. It should be appreciated that the electronic device 600 can be a part of one or more networks, such as virtual networks, which further can include more than one network.

[0140] For the purpose of clarity, technical solutions and advantages of embodiments of the present application will be further described below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0141] Resnet-50 is selected as the baseline experiment for multiple tasks such as semantic segmentation, target detection and minimum target tracking, the downstream task processing head is respectively selected: deeplabv3, faster-rcnn and siamrpn++, the shearlet layer is transmitted into three times respectively, and the experimental results are as follows:

[0142]

[0143]

[0144]

[0145] In summary, the fine-grained representation remote sensing image interpretation method and system based on shearlet prior knowledge can prove the effectiveness of the prior knowledge in multiple downstream tasks in the remote sensing field by embedding the prior knowledge into the deep residual network in layers, and the effect of multiple tasks and multiple indicators is obviously improved.

[0146] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0147] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in a certain embodiment can be referred to the related description of other embodiments.

[0148] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0149] In the embodiments of the present application, it should be understood that the disclosed apparatus / terminal and method can be implemented in other manners. For example, the embodiments of the apparatus / terminal described above are merely schematic, and the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electric, mechanical or in other forms.

[0150] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0151] In addition, each function unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or software function unit.

[0152] The integrated module / unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the computer-readable medium can include or exclude contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0153] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0154] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0155] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate a computer implemented process, so that the instructions executed on the computer or other programmable data processing devices provide a process for implementing the functions specified in the flowchart Figure 1 one flow or a plurality of flows and / or the functions specified in the block Figure 1 one flow or a plurality of flows and / or the functions specified in the block

[0156] The above is only to illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical scheme falls within the protection scope of the claims of the present application.

Claims

1. A method for fine-grained representation of remote sensing image interpretation based on Shearlet prior knowledge, characterized in that, The method comprises the following steps: S1, preprocessing operation is performed on original remote sensing image data; S2, multi-level and multi-directional high-frequency sub-band information extraction is performed on the remote sensing image data preprocessed in step S1 by using Shearlet transform; high-frequency components and multi-directional high-frequency sub-bands obtained by splitting the high-frequency components in the direction are obtained by using Laplacian pyramid and shearlet filter respectively; S3, a high-frequency component prior based on Shearlet transform is constructed, and the same level sub-bands are adaptively fused through a spatial attention mechanism to generate a feature map with enhanced details, specifically: S301, the high-frequency sub-bands extracted by three recursive extractions are dimensionally rearranged to prepare for the input format CxHxW of the deep learning model, wherein C is the number of channels, H is the height, and W is the width; each channel C receives all directional sub-bands extracted at a corresponding time, representing directional information at a specific scale; S302, the multi-directional sub-bands are fused by using a spatial attention module based on Shearlet transform; S303, the spatial attention module uses a convolution layer to enhance the learning ability of the model to spatial texture information by using a ReLU activation function; then a second convolution layer deepens feature fusion by using a Sigmoid activation function to generate a compressed feature map focusing on key information of the image, and the final output Shearlet attention feature map has a dimension of 1xHxW; S4, a detail enhancement backbone network based on ResNet is constructed, and the high-frequency component prior extracted by Shearlet transform in step S3 is multi-scale and multi-level weighted fused with deep residual blocks obtained by the deep residual network; S5, the detail enhancement backbone network based on ResNet obtained in step S4 is trained by using a supervised learning method, and the remote sensing image data labeled in step S1 is repeatedly iterated until the downstream task model performs stably on the validation set; S6, the detail enhancement backbone trained in step S5 and different detection heads of multiple tasks are used to output remote sensing image analysis results.

2. The method according to claim 1, wherein the Shearlet prior-based fine-grained representation remote sensing image interpretation method is characterized in that, In step S1, the preprocessing operation includes noise removal, contrast adjustment and image cropping. 3.The Shearlet-prior-based fine-grained representation remote sensing image interpretation method according to claim 1, characterized in that, Step S2 specifically includes: S201, inputting the preprocessed remote sensing image and converting the image from RGB to HSI in color space; S202, applying a Laplacian pyramid method to layer high and low frequency information of the I component of the remote sensing image, and extracting the highest frequency component of the image; S203, for the extracted high-frequency component, a shearlet filter is used to obtain multi-directional highest frequency sub-bands; S204, the remaining low-frequency information is down-sampled to half of the previous size, and the Laplacian pyramid method is applied again to decompose high and low frequency information and obtain multi-directional sub-bands.

4. The method according to claim 3, wherein the Shearlet prior-based fine-grained representation remote sensing image interpretation method is characterized in that, In step S202, the details of the Shearlet transform are as follows: wherein, and represent a spatial vector and a translation vector, respectively, is a scaling parameter, denotes a shear factor, is a dilation matrix, is a shear matrix, is a transform domain.

5. The method according to claim 1, wherein the Shearlet prior-based fine-grained representation remote sensing image interpretation method is characterized in that, Step S5 specifically includes: The neural network is trained by using a supervised learning method, and the labeled remote sensing data is repeatedly iterated; after the neural network is trained, the performance of the neural network is evaluated.

6. The method according to claim 1, wherein the Shearlet prior-based fine-grained representation remote sensing image interpretation method is characterized in that, Step S6 specifically includes: S601, using the trained detail enhancement backbone network to capture discriminative features in the remote sensing image; S602, integrate the plurality of task detection heads with the detail enhancement backbone network to obtain a complete remote sensing multi-task processing model; S603, model integration, integrate the multi-task remote sensing image interpretation model obtained in step S602 to output a remote sensing image analysis result.

7. A fine-grained remote sensing image interpretation system based on Shearlet prior knowledge, characterized in that, Comprise: a preprocessing module, which performs preprocessing operations on original remote sensing image data; a Shearlet transform module, which uses Shearlet transform to extract multi-level and multi-direction high-frequency sub-band information from the preprocessed remote sensing image data; uses Laplacian pyramid and shearlet filter to obtain high-frequency components and multi-direction high-frequency sub-bands after directional splitting, respectively; a spatial attention module, which constructs a high-frequency component prior based on Shearlet transform, adaptively fuses sub-bands of the same level through a spatial attention mechanism, and generates a detail-enhanced feature map, specifically: dimensional rearrangement is performed on the high-frequency sub-bands extracted by 3 times of recursion to prepare for the input format CxHxW of the deep learning model, wherein C is the number of channels, H is the height, and W is the width; each channel C receives all directional sub-bands extracted at a corresponding time, representing directional information at a specific scale; a spatial attention module based on Shearlet transform is used to fuse the multi-directional sub-bands; the spatial attention module uses a convolution layer to enhance the learning ability of the model for spatial texture information through a ReLU activation function; then a second convolution layer deepens feature fusion through a Sigmoid activation function to generate a compressed feature map focusing on key image information, and the final output Shearlet attention feature map has a dimension of 1xHxW; a detail enhancement backbone network module, which constructs a detail enhancement backbone network based on ResNet, and performs multi-scale and multi-level weighted fusion of the high-frequency component prior extracted by Shearlet transform and the deep residual block obtained by the deep residual network; a training and output module, which trains the detail enhancement backbone network based on ResNet obtained by using supervised learning, iterates repeatedly using labeled remote sensing image data, until the downstream task model performs stably on the validation set, uses the trained detail enhancement backbone and different detection heads of multiple tasks to output a remote sensing image analysis result.

Images