Deep Learning-Based Feature Extraction Method for Faint Celestial Targets

Abstract

This invention belongs to the technical field of image processing, specifically relating to a deep learning-based method for feature extraction of faint celestial targets. It aims to address the problem that existing technologies struggle to effectively detect faint celestial targets, thus affecting detection accuracy. The proposed method includes: acquiring a ground-based optical image to be detected; inputting the ground-based optical image into a pre-constructed target neural network to obtain a target feature map output by the target neural network, wherein the target feature map indicates the detailed and semantic information of faint celestial targets in the ground-based optical image; the target neural network includes a backbone network, a feature extraction module, and a feature fusion module. Based on the proposed method, the feature extraction module (Res-MBDA) and the feature fusion module (CAFFM) enhance the focus on information about faint celestial targets, effectively improving the detection rate of faint celestial targets.

Description

Technical Field

[0001] This invention belongs to the technical field of image processing, specifically relating to a method for extracting features of faint celestial targets based on deep learning. Background Technology

[0002] Data-driven deep learning methods can learn target features from data and are widely used in celestial target detection. Given the large scale differences among celestial targets, existing detection modules focus on extracting multi-scale characteristics of the target, effectively detecting celestial targets of different sizes through downsampling methods such as concatenated convolutional layers and feature pyramid structures.

[0003] However, existing modules ignore the large dynamic range of brightness distribution of celestial targets when extracting multi-scale features of targets. The downsampling method (a technique commonly used in signal processing and image processing to reduce the sampling rate or resolution of data) ignores the effective preservation of faint information of targets, resulting in the loss of information of faint celestial targets with low contrast with the background, which affects the detection accuracy. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, namely the difficulty in effectively detecting faint celestial targets and thus affecting detection accuracy, this invention proposes a deep learning-based feature extraction method for faint celestial targets. The method includes: Acquire optical images of the foundation to be inspected; The ground-based optical image is input into a pre-constructed target neural network to obtain a target feature map output by the target neural network. The target feature map is used to indicate the detailed and semantic information of faint celestial objects in the ground-based optical image. The target neural network includes: A backbone network is used to perform multi-level convolution on the ground-based optical image to obtain input feature maps at multiple scales. The feature extraction module is used to extract features from the input feature map to obtain an output feature map. The output feature map has the same resolution as the input feature map. The feature extraction module consists of multiple branches, and the dilation rate of the dilated convolution corresponding to each branch increases sequentially, so that each branch obtains the target information of the input feature map according to the sequentially increasing receptive field range. The target information includes the detailed information, context information and multi-scale information of the input feature map. The feature fusion module is used to extract the lowest-level feature map and the highest-level feature map from the output feature map and perform adaptive weighted fusion processing to obtain a fused feature map. The fused feature map is then stacked with the high-resolution feature map in the channel dimension and convolutional operation to generate the target feature map. The high-resolution feature map is the output feature map that has not been downsampled.

[0005] In some preferred embodiments, the input feature map F of the feature extraction module includes all shallow feature maps of the same layer. and feature maps of two adjacent layers and .

[0006] In some preferred embodiments, the input feature map F of the feature extraction module satisfies: ; In the formula, This indicates a feature stacking operation performed along the channel dimension. This represents a convolution operation with a stride of 2 and a kernel size of 4. This represents the deconvolution operation corresponding to the convolution operation.

[0007] In some preferred embodiments, the processing of the input feature map F by the feature extraction module includes: ; In the formula, This represents the output feature map of the feature extraction module. Indicates the use of calculation The intermediate feature map, This represents the standard convolution operation. This indicates the CBAM attention mechanism based on residual connections. Represents the ReLU activation function. This indicates an element-wise addition operation.

[0008] In some preferred embodiments, the satisfy: ; ; In the formula, This represents the output feature map of the corresponding branch. , indicating three different branches, This represents dilated convolution, with the subscript indicating the dilation rate. This represents the standard convolution operation. This refers to the scSE attention mechanism.

[0009] In some preferred embodiments, the processing procedure of the feature fusion module satisfies: ; ; ; ; ; In the formula, This is the lowest level feature map. This is the top-level feature map. This is the processing result based on the lowest and highest feature maps. and These represent the attention features in the channel dimension and the spatial dimension, respectively. The results are based on attention features derived from channel and spatial dimensions. and These represent global average pooling operations across channel dimensions and across spatial dimensions, respectively. This represents a global max-pooling operation across spatial dimensions. This indicates a channel mixing operation. This indicates a group convolution, with the subscript indicating the kernel size. This represents the sigmoid activation function. This indicates the fusion weight.

[0010] In some preferred embodiments, the weighted feature map of the top-level feature map and the bottom-level feature map satisfies: ; ; In the formula, The weighted feature map of the top-level feature map. The weighted feature map of the lowest-level feature map. This is the top-level feature map. This is the lowest level feature map.

[0011] In some preferred embodiments, the feature fusion module includes at least two branches to preserve detailed information of the input feature map, and the feature fusion module finally outputs a fused feature map. satisfy: ; In the formula, This represents a standard convolution operation with a kernel of 1. The weighted feature map of the top-level feature map. The weighted feature map of the lowest-level feature map. This is the top-level feature map. This is the lowest level feature map.

[0012] In some preferred embodiments, the process of stacking and convolving the channel dimensions of the fused feature map and the high-resolution feature map in the output feature map satisfies: ; In the formula, This represents the standard convolution operation. This represents the fused feature map that is finally output by the feature fusion module. This represents layer 0, which is the high-resolution feature layer without downsampling, from the first high-resolution feature map to the j-th high-resolution feature map, where j represents the feature map number of the second-to-last position.

[0013] In some preferred embodiments, the method further includes: The target feature map is input into a pre-constructed inference network to obtain the target segmentation result output by the inference network; Based on the target segmentation results, target location and prediction box information are obtained by performing connected component extraction and centroid localization operations. The target location and prediction box information are used to indicate faint celestial targets in the ground-based optical image.

[0014] The beneficial effects of this invention are: Considering the large differences in scale and limited information about faint celestial objects, the method proposed in this invention takes into account both multi-scale and faint object information when extracting celestial object features. This alleviates the problem of decreased target detection rate caused by the loss of faint object information due to downsampling during multi-scale feature extraction. The feature extraction module (Res-MBDA) effectively extracts detailed, contextual, and scale information of celestial objects, improving the utilization rate of faint object information. The feature fusion module (CAFFM) adaptively weights and fuses the output feature maps. The fused feature map fully retains the detailed and semantic information of faint celestial objects, improving the fusion quality. Both modules are plug-and-play on various networks and are highly efficient. This method enhances the network's focus on faint celestial objects and improves the network detection rate. Attached Figure Description

[0015] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating a method for extracting features of faint celestial targets based on deep learning, as proposed in an embodiment of the present invention. Figure 2 This is a schematic diagram of the target neural network processing flow proposed in an embodiment of the present invention; Figure 3This is a schematic diagram of the Res-MBDA feature extraction module proposed in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the CAFFM feature fusion module proposed in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a computer system proposed in an embodiment of the present invention. Detailed Implementation

[0016] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] Please refer to Figure 1 The first embodiment of this application provides a method for extracting features of faint celestial objects based on deep learning, the method comprising: Step S10: Obtain the optical image of the ground foundation to be detected; Step S20: Input the ground-based optical image into the pre-constructed target neural network to obtain the target feature map output by the target neural network. The target feature map is used to indicate the detailed information and semantic information of faint celestial objects in the ground-based optical image. The target neural network includes: A backbone network is used to perform multi-level convolution on the ground-based optical image to obtain input feature maps at multiple scales. The feature extraction module is used to extract features from the input feature map to obtain an output feature map. The output feature map has the same resolution as the input feature map. The feature extraction module consists of multiple branches, and the dilation rate of the dilated convolution corresponding to each branch increases sequentially, so that each branch obtains the target information of the input feature map according to the sequentially increasing receptive field range. The target information includes the detailed information, context information and multi-scale information of the input feature map. The feature fusion module is used to extract the lowest-level feature map and the highest-level feature map from the output feature map and perform adaptive weighted fusion processing to obtain a fused feature map. The fused feature map is then stacked with the high-resolution feature map in the channel dimension and convolutional operation to generate the target feature map. The high-resolution feature map is the output feature map that has not been downsampled.

[0019] It should be noted that, considering the large differences in the scale of celestial targets and the large dynamic range of brightness distribution, this embodiment uses a densely nested U-Net network as the backbone network mentioned above. Specifically, it can be DNA-Net (i.e., Dense nested attention network) based on an infrared small target detection network.

[0020] The aforementioned backbone network is also used to upsample / downsample the output feature map obtained by the feature extraction module. Specifically, sampling is achieved through convolution / deconvolution. The parameters of this sampling method are learnable, allowing for adaptive adjustment of sampling parameters based on image characteristics and task requirements. This provides greater flexibility and effectively enhances the network's ability to represent and reconstruct features of weak celestial objects, helping the backbone network to maintain and enhance information about weak targets. After upsampling / downsampling, to maximize the use of the limited information from faint celestial objects, all un-downsampled high-resolution feature maps and the fused feature map are stacked along the channel dimension, and the final target feature map is obtained through a simple convolutional layer.

[0021] Specifically, please refer to Figure 2 , Figure 2 This embodiment illustrates the process of processing ground-based optical images using a target neural network. Specifically, the feature extraction module in this embodiment is a multi-branch dilated attention block with residual connections (Res-MBDA), and the feature fusion module in this embodiment is a cross-attention feature fusion module (CAFFM).

[0022] In one feasible implementation, Res-MBDA receives feature map information from three directions as input, ensuring that the input features fully contain information about faint targets. In order to effectively utilize the limited information of faint targets during feature extraction, Res-MBDA abandons the traditional downsampling method and uses a multi-branch parallel structure with dilated convolutions of different dilation coefficients and scSE attention mechanism to extract multi-scale features of the target and enhance the ability to associate contextual information between features. In particular, a linear mapping branch can be designed to retain information about faint targets in the input feature map. Finally, the CBAM attention mechanism based on residual connections is used to further mine the correlation between features of different dimensions in the fused feature map and output the results.

[0023] In one feasible implementation, CAFFM achieves adaptive weighted fusion of the top-level and bottom-level feature maps: First, element-wise addition is performed on the two feature maps to achieve a coarse blending. Attention maps are then generated in both the spatial and channel dimensions of the blended feature map to highlight the importance of features at different spatial and channel locations. Element-wise addition of the attention maps in both dimensions effectively preserves important target information across different dimensions, preventing it from being overwhelmed by noise or background interference. Channel shuffling and group convolution are then performed to further ensure sufficient flow between features. A sigmoid activation function is used to generate the corresponding fusion weights for the two feature maps. Furthermore, to preserve the original detailed and semantic information in the two feature maps, two additional branches can be designed to transmit the original feature map information. Finally, element-wise addition and convolution operations are performed on the information from the four branches to achieve adaptive weighted fusion of the feature maps. Finally, to fully utilize the limited information of faint celestial targets, all high-resolution feature maps and the CAFFM fused feature map are stacked and convolved along the channel dimension to generate the aforementioned target feature map.

[0024] This embodiment primarily proposes a feature extraction module, Res-MBDA, and a feature fusion module, CAFFM. Res-MBDA is mainly used to mine and utilize limited information from faint celestial objects within a wide range of scales. It continuously enhances the feature representation of faint celestial object information through module stacking, generating high-quality feature maps (i.e., high-resolution feature maps). CAFFM is mainly used for adaptive weighted fusion of the lowest and highest-level feature maps, thereby highlighting detailed and semantic information of faint celestial objects in the fused feature map, improving the quality of feature map fusion. Furthermore, to fully utilize the limited information of faint celestial objects, all high-resolution feature maps and CAFFM fused feature maps are stacked and convolved along the channel dimension. The final generated feature map highlights key regions of faint celestial objects, improving the network's celestial object detection rate. Based on this method, faint celestial object information can be fully preserved and utilized throughout the entire processing flow, and the effective interaction between detailed and semantic information improves the celestial object detection rate of ground-based optical images.

[0025] Specifically, the input feature map F of the feature extraction module includes all shallow feature maps of the same layer. and feature maps of two adjacent layers and .

[0026] For example, applying this method to the infrared small target detection network DNA-Net (Dense nested attention network, an instance of a densely nested U-Net network) allows for feature extraction using Res-MBDA on the same-level nodes of the backbone network. Let the j-th feature map of the i-th layer be numbered as... The input information for Res-MBDA comes from three directions: all shallow feature maps in the same layer. and feature maps of two adjacent layers and .

[0027] More specifically, the input feature map F of the feature extraction module satisfies: ; In the formula, This indicates a feature stacking operation performed along the channel dimension. This represents a convolution operation with a stride of 2 and a kernel size of 4. This represents the deconvolution operation corresponding to the convolution operation.

[0028] More specifically, the processing of the input feature map F by the feature extraction module includes: ; In the formula, This represents the output feature map of the feature extraction module. Indicates the use of calculation The intermediate feature map, This represents the standard convolution operation. This indicates the CBAM attention mechanism based on residual connections. Represents the ReLU activation function. This indicates an element-wise addition operation.

[0029] More specifically, the satisfy: ; ; In the formula, This represents the output feature map of the corresponding branch. , indicating three different branches, This represents dilated convolution, with the subscript indicating the dilation rate. This represents the standard convolution operation. This refers to the scSE attention mechanism.

[0030] Based on this embodiment, Res-MBDA is used to fully mine target detail information, context information and scale information in the feature map, and the scSE attention mechanism is used to integrate the above information in the spatial dimension and channel dimension. By suppressing useless information, the beneficial information is enhanced and the output feature map has the same resolution as the input feature map. This effectively preserves the information of dark and weak targets and achieves the enhanced representation of dark and weak target information, thereby improving the network's attention to dark and weak targets.

[0031] Further, please refer to Figure 3 ,exist Figure 3 In the Res-MBDA feature extraction module shown, It is mainly used for changing the number of channels. The purpose of using 1×3 and 3×1 forms in the second and third branches is to reduce the amount of computation. In essence, it is equal to 3×3 convolution. This is used to provide different receptive field ranges to extract detailed, contextual, and scale information of celestial targets, achieved by taking different void ratio parameters (rate). By integrating feature information through the interaction of spatial and channel dimensions, and suppressing useless information to enhance the representation of useful information, the Res-CBAM attention mechanism is mainly used to generate attention maps of features, ensuring that important information of the target is preserved in the output feature map and is not overwhelmed by other information.

[0032] In this embodiment, after processing by the Res-MBDA feature extraction module, a series of output feature maps will be obtained. After being fused by the CAFFM feature fusion module, these output maps will finally obtain target feature maps that can effectively indicate information about faint celestial objects.

[0033] In this embodiment, the processing flow of the feature fusion module includes: ; ; ; ; ; In the formula, This is the lowest level feature map. This is the top-level feature map. This is the processing result based on the lowest and highest feature maps. and These represent the attention features in the channel dimension and the spatial dimension, respectively. The results are based on attention features derived from channel and spatial dimensions. and These represent global average pooling operations across channel dimensions and across spatial dimensions, respectively. This represents a global max-pooling operation across spatial dimensions. This indicates a channel mixing operation. This indicates a group convolution, with the subscript indicating the kernel size. This represents the sigmoid activation function. This indicates the fusion weight.

[0034] Specifically, the weighted feature map of the top-level feature map and the low-level feature map satisfies: ; ; In the formula, The weighted feature map of the top-level feature map. The weighted feature map of the lowest-level feature map. This is the top-level feature map. This is the lowest level feature map.

[0035] More specifically, to further preserve the original information in the input feature maps, especially the detailed information in the low-level feature maps which is easily buried in the background, two additional branches are constructed to preserve the information in the input feature maps. Therefore, the cross-attention fusion feature of the final output of CAFFM can be represented as: ; In the formula, This represents a standard convolution operation with a kernel of 1. The weighted feature map of the top-level feature map. The weighted feature map of the lowest-level feature map. This is the top-level feature map. This is the lowest level feature map.

[0036] Further, please refer to Figure 4 ,exist Figure 4 In the CAFFM feature fusion module shown, the lowest level feature map is first processed. and top-level features Coarse feature blending is achieved through element-wise addition. Spatial and channel attention maps are then used to obtain feature attention maps in the spatial and channel dimensions of the blended feature map, respectively. Element-wise addition of these two attention maps enables interaction between high-level and low-level feature maps in both spatial and channel dimensions. Channel shuffling (CS) and group convolution (GConv) further promote feature flow and fusion with relatively low computational cost, ensuring sufficient interaction between features at different spatial locations and in different channels. Finally, a sigmoid activation function is applied to generate feature map fusion weights. To maximize the retention of information in the input feature map, two additional branches are added to retain information from the input and channel maps. Finally, CAFFM performs element-wise addition of the feature maps from these four branches to complete the adaptive weighted fusion of the feature maps.

[0037] In this embodiment, the fused feature map obtained by CAFFM is stacked with all high-resolution feature maps in the backbone network in terms of channel dimension. This is because the high-resolution feature map contains the richest information about faint celestial objects. By stacking, the limited information of faint celestial objects can be fully utilized, thereby improving the network's detection rate for such targets.

[0038] Specifically, the process of stacking the fused feature map with the remaining output feature maps along the channel dimension satisfies: ; In the formula, This represents the standard convolution operation. This represents the fused feature map that is finally output by the feature fusion module. This represents the first to the j-th high-resolution feature maps in layer 0 (i.e., the high-resolution feature layer without downsampling), where j represents the feature map index of the second-to-last position.

[0039] In this embodiment, the above method further includes: inputting the target feature map into a pre-constructed inference network to obtain the target segmentation result output by the inference network; and obtaining the target location and prediction box information by performing connected component extraction and centroid localization operations based on the target segmentation result, wherein the target location and prediction box information are used to indicate faint celestial targets in the ground-based optical image.

[0040] Specifically, the pre-constructed inference network in this embodiment can be a semantic segmentation network based on a fully convolutional structure, which can generate target segmentation results through pixel-by-pixel classification. At the same time, based on the target segmentation results, combined with the "connected component extraction → centroid localization → prediction box generation" process known to those skilled in the art, the final faint celestial target can be obtained. The algorithm used in this process and the relevant parameters of the prediction box information can all be implemented using any feasible solution in the art, and this embodiment does not limit it.

[0041] The second embodiment of this application provides a feature extraction system for faint celestial targets based on deep learning, including: The data acquisition module is used to acquire optical images of the ground foundation to be inspected. The data output module is used to input the ground-based optical image into a pre-constructed target neural network to obtain a target feature map output by the target neural network. The target feature map is used to indicate the detailed information and semantic information of faint celestial objects in the ground-based optical image. The target neural network includes: A backbone network is used to perform multi-level convolution on the ground-based optical image to obtain input feature maps at multiple scales. The feature extraction module is used to extract features from the input feature map to obtain an output feature map. The output feature map has the same resolution as the input feature map. The feature extraction module consists of multiple branches, and the dilation rate of the dilated convolution corresponding to each branch increases sequentially, so that each branch obtains the target information of the input feature map according to the sequentially increasing receptive field range. The target information includes the detailed information, context information and multi-scale information of the input feature map. The feature fusion module is used to extract the lowest-level feature map and the highest-level feature map from the output feature map and perform adaptive weighted fusion processing to obtain a fused feature map. The fused feature map is then stacked with the high-resolution feature map in the channel dimension and convolutional operation to generate the target feature map. The high-resolution feature map is the output feature map that has not been downsampled.

[0042] A third embodiment of this application also proposes an electronic device, comprising: At least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor to implement the method as described in the first embodiment.

[0043] The fourth embodiment of this application also proposes a computer-readable storage medium storing computer instructions for execution by the computer to implement the method described in the first embodiment.

[0044] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system suitable for using a server to implement the methods, systems, and apparatus embodiments of this application. Figure 5 The server shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0045] like Figure 5As shown, the computer system includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in Read Only Memory (ROM) 302 or programs loaded from storage section 308 into Random Access Memory (RAM) 303. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An Input / Output (I / O) interface 305 is also connected to the bus 304.

[0046] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 303 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0047] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.

[0048] More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0049] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0050] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0051] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0052] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0053] The technical solution of the present invention has now been described in conjunction with the preferred embodiments shown in the accompanying drawings.

[0054] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for extracting features of faint celestial targets based on deep learning, characterized in that, The method includes: Acquire optical images of the foundation to be inspected; The ground-based optical image is input into a pre-constructed target neural network to obtain a target feature map output by the target neural network. The target feature map is used to indicate the detailed and semantic information of faint celestial objects in the ground-based optical image. The target neural network includes: A backbone network is used to perform multi-level convolution on the ground-based optical image to obtain input feature maps at multiple scales. The feature extraction module is used to extract features from the input feature map to obtain an output feature map. The output feature map has the same resolution as the input feature map. The feature extraction module consists of multiple branches, and the dilation rate of the dilated convolution corresponding to each branch increases sequentially, so that each branch obtains the target information of the input feature map according to the sequentially increasing receptive field range. The target information includes the detailed information, context information and multi-scale information of the input feature map. The feature fusion module is used to extract the bottom-level feature map and the top-level feature map from the output feature map and perform adaptive weighted fusion processing to obtain a fused feature map. The fused feature map is then stacked with the high-resolution feature map in the channel dimension and convolutional operation to generate the target feature map. The high-resolution feature map is the output feature map that has not been downsampled. The feature extraction module is a Res-MBDA module. The Res-MBDA module receives feature map information from three directions as input and extracts multi-scale features of the target through a multi-branch parallel structure of dilated convolution with different dilation coefficients and scSE attention mechanism. The feature fusion module is a CAFFM module. The CAFFM module has two additional branches, which are used to transmit the original information of the bottom-level feature map and the top-level feature map, respectively. The CAFFM module performs element-wise addition and convolution operations on the bottom-level feature map, the top-level feature map, and the original feature maps transmitted by the two additional branches to achieve the adaptive weighted fusion processing. The feature fusion module ultimately outputs a fused feature map. satisfy: ； In the formula, This represents a standard convolution operation with a kernel of 1. The weighted feature map of the top-level feature map. The weighted feature map of the lowest-level feature map. This is the top-level feature map. This is the lowest level feature map; The processing procedure of the feature fusion module satisfies: ； ； ； ； ； The weighted feature map of the top-level feature map and the bottom-level feature map satisfies: ； ； In the formula, This is the lowest level feature map. This is the top-level feature map. This is the processing result based on the lowest and highest feature maps. and These represent the attention features in the channel dimension and the spatial dimension, respectively. The results are based on attention features derived from channel and spatial dimensions. and These represent global average pooling operations across channel dimensions and across spatial dimensions, respectively. This represents a global max-pooling operation across spatial dimensions. This indicates a channel mixing operation. This indicates a group convolution, with the subscript indicating the kernel size. This represents the sigmoid activation function. Indicates the fusion weight. This represents the standard convolution operation. This indicates a feature stacking operation performed along the channel dimension. The weighted feature map of the top-level feature map. It is a weighted feature map of the lowest-level feature map.

2. The method for extracting features of faint celestial targets based on deep learning according to claim 1, characterized in that, The input feature map F of the feature extraction module includes all shallow feature maps of the same layer. and feature maps of two adjacent layers and ; , where is the layer number of the feature map. This refers to the sequence number of the feature maps within the same level.

3. The method for extracting features of faint celestial targets based on deep learning according to claim 2, characterized in that, The input feature map F of the feature extraction module satisfies: ； In the formula, This represents a convolution operation with a stride of 2 and a kernel size of 4. This represents the deconvolution operation corresponding to the convolution operation.

4. The method for extracting features of faint celestial targets based on deep learning according to claim 1, characterized in that, The processing of the input feature map F by the feature extraction module includes: ； In the formula, This represents the output feature map of the feature extraction module. Indicates the use of calculation The intermediate feature map, This represents the standard convolution operation. This indicates the CBAM attention mechanism based on residual connections. Represents the ReLU activation function. This indicates an element-wise addition operation.

5. The method for extracting features of faint celestial targets based on deep learning according to claim 4, characterized in that, The satisfy: ； ； In the formula, This represents the output feature map of the corresponding branch. , indicating three different branches, This represents dilated convolution, with the subscript indicating the dilation rate. This represents the standard convolution operation. This refers to the scSE attention mechanism.

6. The method for extracting features of faint celestial targets based on deep learning according to claim 1, characterized in that, The process of stacking and convolving the fused feature map with the high-resolution feature map in the output feature map according to the channel dimension satisfies: ； In the formula, This represents the fused feature map that is finally output by the feature fusion module. This represents layer 0, which is the high-resolution feature layer without downsampling, from the first high-resolution feature map to the j-th high-resolution feature map, where j represents the feature map number of the second-to-last position.

7. The method for extracting features of faint celestial targets based on deep learning according to claim 1, characterized in that, The method further includes: The target feature map is input into a pre-constructed inference network to obtain the target segmentation result output by the inference network; Based on the target segmentation results, target location and prediction box information are obtained by performing connected component extraction and centroid localization operations. The target location and prediction box information are used to indicate faint celestial targets in the ground-based optical image.

Images