Remote sensing image target detection method based on manifold learning and electronic equipment

Abstract

The invention relates to a manifold learning-based remote sensing image target detection method and electronic equipment, and the method comprises the steps: carrying out the feature extraction of an input remote sensing image through a feature extraction backbone network, and obtaining feature maps of different levels; constructing a feature fusion network, and performing fusion operation on the feature maps of different levels to obtain a fused feature map; performing dimension reduction operation on the different levels of feature maps by using a manifold learning algorithm to obtain a plurality of dimension-reduced feature maps; fusing the feature map after dimension reduction with the fused feature map to obtain a fused feature map with enhanced information; performing target detection on the fused feature map by using a feature detection head, and calculating classification loss and position prediction loss; repeatedly executing the steps, and training the detection model; and detecting the remote sensing image by using the detection model. According to the method, the capability of classifying and positioning the interested target in the remote sensing image by the model is improved, the target positioning accuracy of the remote sensing image is improved, and the model detection capability is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image feature fusion, and in particular to a remote sensing image target detection method and electronic equipment based on manifold learning. Background Art

[0002] With the continuous development of remote sensing technology and the continuous improvement of satellite resolution, the number and resolution of remote sensing images are also increasing, and the number of remote sensing images acquired is growing exponentially. Remote sensing image target detection refers to the process of automatically detecting targets of interest from remote sensing images acquired from satellites or aerial vehicles. This technology is widely used in many fields, such as agriculture, forestry, urban planning, natural resource management, and environmental monitoring. Remote sensing images have large widths, complex imaging backgrounds, and a large number of small targets. Therefore, how to efficiently detect targets from large numbers of remote sensing images has become an important research direction. Currently, commonly used remote sensing image target detection techniques include traditional feature engineering-based methods and deep learning-based methods.

[0003] Traditional feature engineering methods require manual extraction of features from images and classification of these features using machine learning algorithms. While this approach can achieve some results, it requires manual design of feature extraction rules and is difficult to process with complex remote sensing image data.

[0004] Deep learning-based methods use deep convolutional neural networks (CNNs) to automatically learn features from images and achieve end-to-end object detection. This approach is highly accurate and robust, and does not require manual feature extraction, making it widely used in remote sensing image object detection.

[0005] Traditional object detection algorithms struggle to meet the demands of efficient processing and accurate recognition for large-scale remote sensing imagery. Therefore, deep learning-based object detection algorithms for remote sensing images have become a research hotspot. Simultaneously, with the continuous advancement of deep learning technology, deep learning-based methods are also being continuously improved and optimized, such as by introducing techniques like attention mechanisms and multi-task learning to enhance detection accuracy. Feature fusion networks, as a method for improving object detection accuracy, play a crucial role in remote sensing image object detection. They leverage the feature extraction capabilities of convolutional neural networks (CNNs) to fuse feature maps from multiple layers and scales, thereby obtaining richer and more accurate object information. This fusion can be achieved through various approaches, such as cascade structures, feature pyramids, and attention mechanisms. Feature fusion networks can fuse features at multiple levels, enabling the network to obtain more comprehensive information. By constructing feature fusion networks, traditional feature fusion methods can be applied to deep learning models. These methods typically employ simple weighted summation or concatenation to merge feature maps from different layers into a single, comprehensive feature map.

[0006] Traditional feature fusion methods mainly use simple weighted summation or splicing to merge feature maps from different feature layers into a comprehensive feature map. However, this simple feature fusion method has some shortcomings. First, feature maps from different feature layers have different resolutions and semantic information, and simple fusion methods cannot make good use of their differences and complementarity. Secondly, in order to achieve feature fusion operations, the above methods have successively downsampled feature maps of different sizes and adjusted channel dimensions, which has caused the loss of a lot of important information. In addition, whether it is channel addition or dimensional splicing operations, most feature fusion methods are based on linear transformations, while the important target feature relationships in target detection tasks are often nonlinear, which further leads to poor fusion effects. The above problems have hindered the existing remote sensing image target detection tasks from achieving better performance. Summary of the Invention

[0007] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a remote sensing image target detection method and electronic equipment based on manifold learning, so as to help the target detection model capture the nonlinear relationship of high-dimensional feature data in the feature fusion stage, help the model better understand the distribution of data, learn more abstract feature representations, enhance the information dimension of the original prediction feature map, and thus improve the generalization ability of target detection tasks.

[0008] To achieve the above-mentioned object of the invention, the present invention provides a remote sensing image target detection method based on manifold learning, comprising the following steps:

[0009] Step S1: extract features from the input remote sensing image through the feature extraction backbone network to obtain feature maps at different levels;

[0010] Step S2: construct a feature fusion network, perform fusion operations on feature maps of different levels, and obtain a fused feature map;

[0011] Step S3: using a manifold learning algorithm to perform dimensionality reduction operations on the feature maps of different levels to obtain multiple feature maps after dimensionality reduction;

[0012] Step S4: Fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map with enhanced information;

[0013] Step S5: using a feature detection head to perform target detection on the fused feature map, and calculating the classification loss and position prediction loss;

[0014] Step S6: Repeat steps S1 to S5 to train the detection model;

[0015] Step S7: Detect the remote sensing image using the detection model obtained in step S6.

[0016] According to a technical solution of the present invention, step S1 specifically includes:

[0017] Step S11, constructing a feature extraction backbone network;

[0018] Step S12: using the feature extraction backbone network to receive input remote sensing images and perform preprocessing operations on the remote sensing images.

[0019] The pre-processing operation includes at least image cropping, image flipping, projection transformation, mosaic enhancement and image filling;

[0020] Step S13: performing feature extraction on the remote sensing image to obtain feature maps at different levels.

[0021] According to a technical solution of the present invention, in step S2, it specifically includes:

[0022] Step S21: construct a feature fusion network;

[0023] Step S22: using the feature fusion network to receive the multi-level remote sensing image feature maps from the feature extraction backbone network, performing a fusion operation on the remote sensing image feature maps of different levels to obtain a fused feature map of the remote sensing image target;

[0024] Among them, the feature maps of different levels include at least shallow feature maps, middle feature maps and deep feature maps, and the fusion method includes at least upsampling, horizontal connection and dimensional splicing.

[0025] According to a technical solution of the present invention, in step S22, the following steps are specifically included:

[0026] Step S221: traverse the shallow feature map, the middle feature map, and the deep feature map, and use 1×1 convolution to make the number of channels of the feature map consistent;

[0027] Step S222: perform a size upsampling operation on the feature map with the same number of channels, adjust the size of the shallow feature map to the size of the middle feature map, and then perform dimension splicing or channel addition to obtain the original fused feature map.

[0028] According to a technical solution of the present invention, in step S3, the following steps are specifically included:

[0029] The manifold learning method is used to reduce the dimensionality of the feature maps of different levels extracted by the feature extraction backbone network, including shallow feature maps, middle feature maps and deep feature maps.

[0030] The manifold learning method includes high-dimensional space feature representation, distance estimation, adjacency matrix construction and low-dimensional mapping.

[0031] According to a technical solution of the present invention, in step S31, it specifically includes:

[0032] Step S311: First, vectorize the multi-level feature graph from the feature extraction backbone network and define F∈R H×W×C To represent a certain level of feature map from the feature extraction backbone network, the size is H×W and the number of channels is C;

[0033] F=[x1,x2,…,x i ,…,x N ]

[0034] Among them, [x1,x2,…,x i ,…,x N ] represents the sequence data obtained after the feature map F is stretched, N = H × W, x i ∈R N*C Represents the data at a certain position of the feature map, i∈[1,2,…,N];

[0035] Step S312: Use distance measurement method to determine the sample x in the sequence i The k nearest neighbor nodes of , and then construct the adjacency matrix:

[0036]

[0037] in w ji is the weight information, Calculate xi Node and x ji The distance between nodes, and finally obtain the weight matrix W of high-dimensional features = {w1,w2,…,w N}∈R k*N , and get K distances x i The nearest neighboring nodes of a node, obtain the adjacency matrix;

[0038] Step S313: Use the weight matrix W of the high-dimensional features in step S312 to transform the low-dimensional space target matrix to ensure that the local linear properties of the low-dimensional space are the same as those of the original input space. Define the low-dimensional space target matrix Y = {y1, y2, ..., y N}∈R d*N , Each sample of Y is represented as a mapping from high-dimensional space to low-dimensional space, the sample length is d, and the corresponding local linear property constraint of the low-dimensional target matrix Y is expressed as:

[0039]

[0040] in, Indicates that the sum of the values of all samples in the same dimension is 0. After the weight matrix transformation, the required P-dimensional data is obtained, and the matrix Y consisting of the P eigenvectors corresponding to the minimum matrix Y under the constraint conditions is obtained. P ={y1,y2,…,y P} T , get the feature map Y after dimensionality reduction P ∈R P*d ;

[0041] Step S314: Use the shape reshaping operation to reduce the dimension of the feature map Y obtained in step S313. P ={y1,y2,…,y P} T Process it to normalize it to the original feature map F∈R H×W×C domain to obtain the feature map size that meets the subsequent fusion requirements.

[0042] According to a technical solution of the present invention, in step S4, it specifically includes:

[0043] Step S41: performing feature normalization on the multiple feature maps obtained in step S3 after dimensionality reduction;

[0044] Step S42: Fusing the normalized dimensionality reduction feature map with the fusion feature map of the remote sensing image target obtained in step S2 to obtain a fused feature map.

[0045] Wherein, the fusion method includes an addition operation, a feature concatenation operation or a dual fusion operation;

[0046] Step S43: Filter the fused feature map obtained in step S42 through the attention mechanism to obtain the optimal channel information representation.

[0047] According to a technical solution of the present invention, in step S41, the dimension reduction vectors that do not meet the image pixel threshold range are normalized to the range of 0 to 255 to obtain data representation within the appropriate channel range.

[0048] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the above-mentioned one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device performs a remote sensing image target detection method based on manifold learning as described in any one of the above-mentioned technical solutions.

[0049] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, a remote sensing image target detection method based on manifold learning as described in any of the above technical solutions is implemented.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention proposes a remote sensing image target detection method and electronic device based on manifold learning. When training a remote sensing image target detection model based on a convolutional neural network, the input remote sensing image features are extracted through a feature extraction backbone network to obtain feature maps at different hierarchical positions, thereby improving the model's ability to classify and locate targets of interest in the remote sensing image. The remote sensing image target detection method based on manifold learning is then used to enhance the fusion quality of shallow feature maps, mid-layer feature maps, and deep feature maps generated in the feature fusion network, helping the target detection model to capture the nonlinear relationship of high-dimensional feature data in the feature fusion stage, helping the model to better understand the distribution of data, learn more abstract feature representations, enhance the information dimension of the original prediction feature map, and thus improve the generalization ability of target detection tasks.

[0052] Furthermore, the remote sensing image target detection model can obtain multi-scale information of the target during the detection process, and obtain richer and more accurate target information, thereby improving the model's ability to classify and locate targets of interest in remote sensing images, and improving the accuracy of remote sensing image target positioning, which is of great significance for rotating frame target detection in high-resolution remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0054] Figure 1 Schematically showing a flow chart of a method for remote sensing image target detection based on manifold learning according to one embodiment of the present invention;

[0055] Figure 2 A schematic diagram illustrating the structure of a target detection model constructed by a remote sensing image target detection method based on manifold learning according to an embodiment of the present invention;

[0056] Figure 3 The figure schematically shows the overall flow chart of a feature fusion target detection algorithm based on manifold learning according to one embodiment of the present invention. DETAILED DESCRIPTION

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0058] like Figures 1 to 3 As shown, a remote sensing image target detection method based on manifold learning of the present invention includes the following steps:

[0059] Step S1: extract features from the input remote sensing image through the feature extraction backbone network to obtain feature maps at different levels;

[0060] Step S2: construct a feature fusion network, perform fusion operations on feature maps of different levels, and obtain a fused feature map;

[0061] Step S3: using a manifold learning algorithm to perform dimensionality reduction operations on the feature maps of different levels to obtain multiple feature maps after dimensionality reduction;

[0062] Step S4: Fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map with enhanced information;

[0063] Step S5: using a feature detection head to perform target detection on the fused feature map, and calculating the classification loss and position prediction loss;

[0064] Step S6: Repeat steps S1 to S5 to train the detection model;

[0065] Step S7: Detect the remote sensing image using the detection model obtained in step S6.

[0066] In this embodiment, when training a remote sensing image target detection model based on a convolutional neural network, the input remote sensing image features are extracted through a feature extraction backbone network to obtain feature maps at different levels, thereby improving the model's ability to classify and locate targets of interest in remote sensing images. The remote sensing image target detection method based on manifold learning is then used to enhance the fusion quality of shallow feature maps, mid-level feature maps, and deep feature maps generated in the feature fusion network, helping the target detection model to capture the nonlinear relationship of high-dimensional feature data in the feature fusion stage, helping the model to better understand the distribution of data, learn more abstract feature representations, enhance the information dimension of the original prediction feature map, and thus improve the generalization ability of target detection tasks.

[0067] Furthermore, the remote sensing image target detection model can obtain multi-scale information of the target during the detection process, and obtain richer and more accurate target information, thereby improving the model's ability to classify and locate targets of interest in remote sensing images, and improving the accuracy of remote sensing image target positioning, which is of great significance for rotating frame target detection in high-resolution remote sensing images.

[0068] In one embodiment of the present invention, preferably, the method specifically includes:

[0069] Step S11, constructing a feature extraction backbone network;

[0070] Step S12: using the feature extraction backbone network to receive input remote sensing images and perform preprocessing operations on the remote sensing images.

[0071] The pre-processing operation includes at least image cropping, image flipping, projection transformation, mosaic enhancement and image filling;

[0072] Step S13: performing feature extraction on the remote sensing image to obtain feature maps at different levels.

[0073] In this embodiment, before performing feature extraction operations on the remote sensing image, the remote sensing image and the target label corresponding to the remote sensing image are first obtained, and appropriate data preprocessing operations are performed on them. The preprocessing process includes at least random flipping, rotation, scaling and cropping, which is beneficial to enhancing the robustness, universality and generalization ability of the model algorithm.

[0074] In one embodiment of the present invention, preferably, step S2 specifically includes:

[0075] Step S21: construct a feature fusion network;

[0076] Step S22: using the feature fusion network to receive the multi-level remote sensing image feature maps from the feature extraction backbone network, performing a fusion operation on the remote sensing image feature maps of different levels to obtain a fused feature map of the remote sensing image target;

[0077] Among them, the feature maps of different levels include at least shallow feature maps, middle feature maps and deep feature maps, and the fusion method includes at least upsampling, horizontal connection and dimensional splicing.

[0078] In this embodiment, when training a remote sensing image target detection model based on a convolutional neural network, constructing a feature fusion network through a feature pyramid can enable the model to obtain multi-scale information of the target, thereby improving the model's ability to classify and locate targets of interest in remote sensing images.

[0079] In one embodiment of the present invention, preferably, step S3 specifically includes:

[0080] Step S31: Using the manifold learning method, the feature maps of different levels extracted by the feature extraction backbone network are reduced in dimension, including shallow feature maps, middle feature maps, and deep feature maps.

[0081] The manifold learning method includes high-dimensional space feature representation, distance estimation, adjacency matrix construction and low-dimensional mapping;

[0082] Step S32: Use the shallow feature map, the middle feature map, and the deep feature map after dimensionality reduction to participate in subsequent feature fusion operations.

[0083] In this embodiment, manifold learning is used to perform dimensionality reduction processing on the shallow, middle and deep feature maps generated in the feature fusion network, and then the multiple feature maps obtained after dimensionality reduction processing based on manifold learning are fused with the original feature maps, thereby enhancing the fusion quality of the shallow feature maps, middle feature maps and deep feature maps generated in the feature fusion network, helping the target detection model to capture the nonlinear relationship of high-dimensional feature data in the feature fusion stage, helping the model to better understand the distribution of data, learn more abstract feature representations, enhance the information dimension of the original prediction feature map, and thus improve the generalization ability of the target detection task.

[0084] In one embodiment of the present invention, preferably, step S4 specifically includes:

[0085] Step S41: performing feature normalization on the multiple feature maps obtained in step S3 after dimensionality reduction;

[0086] Step S42: Fusing the normalized dimensionality reduction feature map with the fusion feature map of the remote sensing image target obtained in step S2 to obtain a fused feature map.

[0087] Wherein, the fusion method includes an addition operation, a feature concatenation operation or a dual fusion operation;

[0088] Step S43: Filter the fused feature map obtained in step S42 through the attention mechanism to obtain the optimal channel information representation.

[0089] like Figure 2 and Figure 3 As shown, in one embodiment of the present invention, preferably, in step S22, the traditional feature fusion method is retained to facilitate the subsequent manifold learning-assisted feature fusion process so that more valuable information is retained or enhanced, specifically including:

[0090] Step S221: traverse the shallow feature map, middle feature map, and deep feature map obtained in step S21, and use 1*1 convolution to make the number of feature map channels consistent;

[0091] Step S222: perform a size upsampling operation on the feature map with the same number of channels obtained in step S221, adjust the shallow feature size to the size of the middle feature map, and then perform dimension splicing or channel addition to obtain the original fused feature map.

[0092] In one embodiment of the present invention, preferably, in step S31, pixel-level data traversal processing is performed on the input feature map information, and a feature vector of 1*1*number of channels is extracted according to the pixel cycle to obtain a corresponding number of feature points. Then, dimensionality reduction processing is performed on the obtained feature data to obtain a feature vector. The dimensionality reduction method based on manifold learning includes but is not limited to isometric mapping, local linear embedding or Laplace eigenmapping, as follows:

[0093] Step S311: First, vectorize the multi-level feature graph from the feature extraction backbone network and define F∈R H×W×C To represent a certain level of feature map from the feature extraction backbone network, the size is H×W and the number of channels is C;

[0094] F=[x1,x2,…,x i ,…,x N ]

[0095] Among them, [x1,x2,…,x i ,…,x N ] represents the sequence data obtained after the feature map F is stretched, N = H × W, x i ∈R N*CRepresents the data at a certain position of the feature map, i∈[1,2,…,N];

[0096] Step S312: Use distance measurement method to determine the sample x in the sequence i The k nearest neighbor nodes of , and then construct the adjacency matrix:

[0097]

[0098] in ω ji is the weight information, Calculate x i Node and x ji The distance between nodes, and finally obtain the weight matrix W of high-dimensional features = {w1,w2,…,w N}∈R k*N , and get K distances x i The nearest neighbor nodes of a node are obtained to obtain the adjacency matrix.

[0099] Step S313: Use the weight matrix W of the high-dimensional features in step S312 to transform the low-dimensional space target matrix to ensure that the local linear properties of the low-dimensional space are the same as those of the original input space. Define the low-dimensional space target matrix Y = {y1, y2, ..., y N}∈R d*N ,Each sample of Y is represented as a mapping from high-dimensional space to low-dimensional space, the sample length is d, and the corresponding local linear property constraint of the low-dimensional target matrix Y is expressed as;

[0100]

[0101] in Indicates that the sum of the values of all samples in the same dimension is 0. After the weight matrix transformation, the required P-dimensional data can be obtained, that is, the matrix Y consisting of the P eigenvectors corresponding to the minimum matrix Y under the constraint conditions is obtained. P ={y1,y2,…,y P} T That’s it. In this way, we get the feature map Y after dimensionality reduction P ∈R P*d .

[0102] Step S314: further reshape the sequence Y obtained after the dimension reduction in step S313 using the reshape operation P ={y1,y2,…,y P} T Process it to normalize it to the original feature map F∈R H×W×C domain to obtain the feature map size that meets the subsequent fusion requirements.

[0103] In one embodiment of the present invention, preferably, the step S41 specifically includes:

[0104] In step S411 and step S3, the dimension reduction vectors that do not meet the image pixel threshold range are normalized to obtain data representation within a suitable channel range, that is, normalized to the range of 0 to 255.

[0105] like Figure 3 As shown, the target detection model training flow chart constructed by the remote sensing image target detection method based on manifold learning, first the remote sensing image is read and preprocessed, and after the backbone feature extraction network is completed, different from the traditional feature fusion network construction, this application adopts the feature fusion network construction based on manifold learning; receiving multi-layer feature maps; feature dimensionality reduction based on manifold learning; feature reshaping to adapt to subsequent fusion size; feature map production after dimensionality reduction; feature map data normalization; traditional fusion feature map generation; fusion of dimensionality reduction feature map based on manifold learning and traditional feature map; feature map decoupling by feature detection head; classification and position regression prediction; loss budget; model training is completed.

[0106] like Figure 3 As shown, a remote sensing image target detection method based on manifold learning according to an embodiment of the present invention may include the following specific steps: step S100, acquiring remote sensing image data, and performing appropriate preprocessing operations, including image cropping, image flipping, projection transformation, etc.; step S200, building a deep neural network model, and extracting target features through a backbone feature extraction network; step S300, building a feature fusion network, and fusing the feature maps obtained by the backbone feature extraction network; step S400, using manifold learning to perform dimensionality reduction operations on shallow, middle, and deep feature maps to improve the image quality. The shapes of feature maps at different levels are changed to facilitate subsequent fusion; step S500, the feature map obtained by dimensionality reduction using manifold learning is spliced with the feature map after normal fusion to promote the information flow of the fusion process; step S600, the final feature map obtained after the fusion of the feature detection head is constructed for decoupling, and the classification loss and position prediction loss are calculated; step S700, it is determined whether the training is completed, if so, step S900 is executed, if not, steps S100 to S600 are executed again; step S800, the remote sensing image is detected using the detection model obtained in step S700.

[0107] Among them, the gap between the model prediction results and the actual annotations is used as the judgment basis to control whether the training is completed.

[0108] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the above-mentioned one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device performs a remote sensing image target detection method based on manifold learning as any one of the above-mentioned technical solutions.

[0109] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, a remote sensing image target detection method based on manifold learning as described in any of the above technical solutions is implemented.

[0110] Computer-readable storage media may include any medium capable of storing or transmitting information. Examples of computer-readable storage media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. The code segments may be downloaded via a computer network such as the Internet, an intranet, and the like.

[0111] The present invention provides a remote sensing image target detection method and electronic device based on manifold learning. The remote sensing image target detection method based on manifold learning comprises: step S1, extracting features from an input remote sensing image through a feature extraction backbone network to obtain feature maps of different levels; step S2, constructing a feature fusion network, fusing feature maps of different levels to obtain fused feature maps; step S3, using a manifold learning algorithm to perform dimensionality reduction operations on the feature maps of different levels to obtain multiple feature maps after dimensionality reduction; step S4, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing information; step S5, using a feature detection head to perform dimensionality reduction operations on the feature maps of different levels to obtain a plurality of feature maps after dimensionality reduction; step S6, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing information; step S7, using a feature detection head to perform dimensionality reduction operations on the feature maps of different levels to obtain a plurality of feature maps after dimensionality reduction; step S8, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing information; step S9, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing information; step S10, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing information; step S11, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing information; step S12, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing information; step S13, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing information; step S14, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing information; step S15, fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map after enhancing The fused feature map is used for target detection, and the classification loss and position prediction loss are calculated; step S6, repeating steps S1 to S5 to train the detection model; step S7, using the detection model obtained in step S6 to detect the remote sensing image, the remote sensing image target detection method based on manifold learning enhances the fusion quality of the shallow feature map, the middle feature map, and the deep feature map generated in the feature fusion network, helps the target detection model to capture the nonlinear relationship of high-dimensional feature data in the feature fusion stage, helps the model to better understand the distribution of the data, learns more abstract feature representations, enhances the information dimension of the original prediction feature map, and thus improves the generalization ability of the target detection task.

[0112] Furthermore, the remote sensing image target detection model can obtain multi-scale information of the target during the detection process, and obtain richer and more accurate target information, thereby improving the model's ability to classify and locate targets of interest in remote sensing images, and improving the accuracy of remote sensing image target positioning, which is of great significance for rotating frame target detection in high-resolution remote sensing images.

[0113] Furthermore, it should be noted that the present invention may be provided as a method, apparatus, or computer program product. Thus, embodiments of the present invention may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code.

[0114] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0116] It should also be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal device comprising the element.

[0117] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be noted that although the preferred embodiment of the present invention has been described, it is clear that those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered as within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A remote sensing image target detection method based on manifold learning, characterized in that: The following steps are involved: Step S1: extract features from the input remote sensing image through the feature extraction backbone network to obtain feature maps at different levels; Step S2: construct a feature fusion network, perform fusion operations on feature maps of different levels, and obtain a fused feature map; Step S3: using a manifold learning algorithm to perform dimensionality reduction operations on the feature maps of different levels to obtain multiple feature maps after dimensionality reduction; Step S4: Fusing the feature map after dimensionality reduction with the fused feature map to obtain a fused feature map with enhanced information; Step S5: Use the feature detection head to perform target detection on the fused feature map, and calculate the classification loss and position prediction loss; Step S6: Repeat steps S1 to S5 to train the detection model; Step S7: Detect the remote sensing image using the detection model obtained in step S6.

2. The method for remote sensing image target detection based on manifold learning according to claim 1, characterized in that: The step S1 specifically includes: Step S11, constructing a feature extraction backbone network; Step S12: using the feature extraction backbone network to receive input remote sensing images and perform preprocessing operations on the remote sensing images. The pre-processing operation includes at least image cropping, image flipping, projection transformation, mosaic enhancement and image filling; Step S13: performing feature extraction on the remote sensing image to obtain feature maps at different levels.

3. The method for remote sensing image target detection based on manifold learning according to claim 1, characterized in that: In the step S2, it specifically includes: Step S21: construct a feature fusion network; Step S22: using the feature fusion network to receive the multi-level remote sensing image feature maps from the feature extraction backbone network, performing a fusion operation on the remote sensing image feature maps of different levels to obtain a fused feature map of the remote sensing image target; Among them, the feature maps of different levels include at least shallow feature maps, middle feature maps and deep feature maps, and the fusion method includes at least upsampling, horizontal connection and dimensional splicing.

4. The method for remote sensing image target detection based on manifold learning according to claim 3, characterized in that: In the step S22, it specifically includes: Step S221: traverse the shallow feature map, the middle feature map, and the deep feature map, and use 1×1 convolution to make the number of channels of the feature map consistent; Step S222: perform a size upsampling operation on the feature map with the same number of channels, adjust the size of the shallow feature map to the size of the middle feature map, and then perform dimension splicing or channel addition to obtain the original fused feature map.

5. The method for remote sensing image target detection based on manifold learning according to claim 1, characterized in that: In the step S3, it specifically includes: The manifold learning method is used to reduce the dimensionality of the feature maps of different levels extracted by the feature extraction backbone network, including shallow feature maps, middle feature maps and deep feature maps. The manifold learning method includes high-dimensional space feature representation, distance estimation, adjacency matrix construction and low-dimensional mapping.

6. The method for remote sensing image target detection based on manifold learning according to claim 5, characterized in that: In the step S31, it specifically includes: Step S311: First, vectorize the multi-level feature graph from the feature extraction backbone network and define F∈R H ×W×C To represent a certain level of feature map from the feature extraction backbone network, the size is H×W and the number of channels is C; F=[x1,x2,…,x i ,…,x N ] Among them, [x1,x2,…,x i ,…,x N ] represents the sequence data obtained after the feature map F is stretched, N = H × W, x i ∈R N*C Represents the data at a certain position of the feature map, i∈[1,2,…,N]; Step S312: Use distance measurement method to determine the sample x in the sequence i The k nearest neighbor nodes are constructed to construct the adjacency matrix: in is the weight information, Calculate x i Node and x ji The distance between nodes, and finally obtain the weight matrix W of high-dimensional features = {w1,w2,…,w N }∈R k*N , and get K distances x i The nearest neighboring nodes of a node, obtain the adjacency matrix; Step S313: Use the weight matrix W of the high-dimensional features in step S312 to transform the low-dimensional space target matrix to ensure that the local linear properties of the low-dimensional space are the same as those of the original input space. Define the low-dimensional space target matrix Y = {y1, y2, ..., y N }∈R d*N , Each sample of Y is represented as a mapping from high-dimensional space to low-dimensional space, the sample length is d, and the corresponding local linear property constraint of the low-dimensional target matrix Y is expressed as: in, Indicates that the sum of the values of all samples in the same dimension is 0. After the weight matrix transformation, the required P-dimensional data is obtained, and the matrix Y consisting of the P eigenvectors corresponding to the minimum matrix Y under the constraint conditions is obtained. P ={y1,y2,…,y P } T , get the feature map Y after dimensionality reduction P ∈R P*d ; Step S314: Use the shape reshaping operation to reduce the dimension of the feature map Y obtained in step S313. P ={y1,y2,…,y P } T Process it to normalize it to the original feature map F∈R H×W×C domain to obtain the feature map size that meets the subsequent fusion requirements.

7. The method for remote sensing image target detection based on manifold learning according to claim 1, characterized in that: In the step S4, it specifically includes: Step S41: performing feature normalization on the multiple feature maps obtained in step S3 after dimensionality reduction; Step S42: Fusing the normalized dimensionality reduction feature map with the fusion feature map of the remote sensing image target obtained in step S2 to obtain a fused feature map. Wherein, the fusion method includes an addition operation, a feature concatenation operation or a dual fusion operation; Step S43: Filter the fused feature map obtained in step S42 through the attention mechanism to obtain the optimal channel information representation.

8. The method for remote sensing image target detection based on manifold learning according to claim 7, characterized in that: In step S41, the dimension reduction vectors that do not meet the image pixel threshold range are normalized to the range of 0 to 255 to obtain data representation within the appropriate channel range.

9. An electronic device, characterized in that: include: One or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to enable the electronic device to perform the remote sensing image target detection method based on manifold learning as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, implement the remote sensing image target detection method based on manifold learning according to any one of claims 1 to 8.

Images