Cross-scale space target detection method and device

Abstract

The invention discloses a cross-scale space target detection method and a cross-scale space target detection device. The method comprises the following steps: sequentially processing an original image through a plurality of convolutional layers of an enhanced kernel clustering initial network, extracting multi-scale features and context features, extracting a relationship among long-distance pixels, and outputting a multi-level feature map; inputting the input feature X and the input feature Y into an enhanced feature fusion module of a feature pyramid network through the feature pyramid network; extracting channel enhancement features and spatial enhancement features of the input feature X and the input feature Y, and performing weighted fusion to obtain a target fusion feature map; and sending the target fusion feature map into a detector, carrying out interaction on the object query and the target fusion feature map, and predicting to obtain an object category and bounding box coordinates. According to the scheme, the detection capability of a target with severe size change in a space environment is remarkably improved, and geometric deformation of satellite components caused by view angle change can be better processed. The detection process is simplified while the performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for cross-scale spatial target detection. Background Art

[0002] With the rapid development of space technology and the continuous growth of the number of spacecraft in orbit, on-orbit servicing and maintenance of failed satellites has become a critical technology area for ensuring the sustainable utilization of space resources and reducing the risk of space debris. The autonomous and intelligent nature of on-orbit capture, repair, and refueling tasks places extremely high demands on the servicing spacecraft's ability to accurately perceive and understand the status of target satellites. Accurate and robust detection of failed satellites and their critical components (such as solar panels, docking mechanisms, communication antennas, and engine nozzles) is a prerequisite and core component for completing complex on-orbit operations.

[0003] Currently, target detection in a space environment faces a series of unique and severe challenges. First, the scale of the target varies greatly: during the approach of the capture mission, the target satellite gradually transitions from a fuzzy point or surface target at a distance to a complex structure with many components that can be clearly distinguished at close range. This places extremely high demands on the cross-scale adaptability of the detection algorithm. Secondly, the imaging conditions are complex and changeable: the space background is usually a deep dark environment, and the surface of the target satellite may be affected by uneven lighting such as direct sunlight, earth reflection, or its own shadow, resulting in low image contrast and poor signal-to-noise ratio. In addition, the relative motion between the observation satellite and the target satellite, the attitude adjustment of the service spacecraft, and the operation of the robotic arm may all introduce image jitter and motion blur, further increasing the difficulty of target detection.

[0004] Existing object detection algorithms, whether region proposal-based two-stage detectors (such as Faster R-CNN) or efficient single-stage detectors (such as the YOLO series), have limitations when directly applied to such tasks. Two-stage detectors are slow and struggle to meet real-time requirements; while single-stage detectors are fast, they face bottlenecks in small object detection accuracy and their reliance on non-maximum suppression (NMS) post-processing. In recent years, Transformer-based end-to-end detectors (such as DETR and its variant RT-DETR) have simplified the process by eliminating NMS. However, standard models still need to improve their feature extraction and fusion capabilities when dealing with the extreme cross-scale characteristics of spatial objects and complex imaging conditions.

[0005] Existing target detection methods still have shortcomings in addressing the above challenges. Therefore, there is an urgent need for a new target detection method and system that can effectively cope with the challenges of the space environment and achieve high-precision and high-robustness detection of key satellite components across scales. Summary of the Invention

[0006] This solution aims to at least solve the technical problems existing in the prior art. To this end, the first aspect of the present invention proposes a cross-scale spatial target detection method, the method comprising: The original image is sequentially processed through multiple convolutional layers of the enhanced poly-kernel inception network, multiple convolutions are used to extract multi-scale features and contextual features of the original image, and global average pooling and strip convolution are used to extract the relationship between long-distance pixels in the original image, and a multi-level feature map is output from the last convolutional layer; Performing upsampling and channel dimension adjustment on the multi-level feature maps and the multiple feature maps output by the convolutional layer through a feature pyramid network, and inputting the obtained input features X and Y into an enhanced feature fusion module of the feature pyramid network; fusing the input feature X and the input feature Y through the enhanced feature fusion module, extracting the channel enhancement feature and the spatial enhancement feature of the input feature X and the input feature Y, and performing weighted fusion on the channel enhancement feature, the spatial enhancement feature, the input feature X, and the input feature Y to obtain a target fused feature map; The target fused feature map is fed into a Transformer-based detector, an object query is generated through an IoU-aware query selection mechanism, the object query and the target fused feature map are interacted, and the object category and bounding box coordinates of the target object in the original image are predicted.

[0007] Optionally, each of the multiple convolutional layers includes a first processing channel consisting of a feedforward network and a second processing channel consisting of a plurality of E_PKI blocks connected, and each of the convolutional layers is processed according to the following steps: Obtaining layer input information from the previous convolutional layer, and dividing the layer input information into two halves according to the channel size of the first processing channel and the second processing channel to obtain first input information and second input information respectively; Feeding the first input information into the feedforward network for feature extraction processing to obtain first output information; Feeding the second input information to the E_PKI block for feature extraction processing to obtain second output information; The first output information and the second output information are connected and then a 1×1 convolution operation is performed to obtain layer output information, and the layer output information is transmitted to the next convolution layer.

[0008] Optionally, the E_PKI block includes an E_PKI module and a CAA module, and feeding the second input information to the E_PKI block for feature extraction processing to obtain the second output information includes: The E_PKI module uses multiple convolutions in parallel to extract multi-scale features and contextual features of the second input information; the multiple convolutions include 1×1 convolution, 3×3 convolution, 3×3 dilated convolution, and 3×3 deformable convolution; the contextual features refer to other regional features in the image corresponding to the second input information that have semantic or spatial associations with the current region; The E_PKI module combines the multi-scale features and the context features by element-by-element addition to obtain a combined feature; performs a 1×1 convolution operation on the combined feature to obtain a first convolution feature; The CAA module compresses the second input information into a global feature vector through global average pooling, and maps the global feature vector into a convolution kernel size parameter using a two-layer perceptron; The CAA module performs one-dimensional convolution on the convolution kernel size parameter in the horizontal and vertical directions respectively to obtain the horizontal convolution kernel size and the vertical convolution kernel size; The CAA module generates an attention map by applying a Sigmoid function to the horizontal convolution kernel size and the vertical convolution kernel size; The E_PKI block multiplies the first convolution feature and the attention map to obtain a product; performs a 1×1 convolution operation on the product to obtain second output information.

[0009] Optionally, processing the multi-level feature maps and the multiple feature maps output by the convolutional layer through a feature pyramid network, and inputting the processed input features X and input features Y into an enhanced feature fusion module of the feature pyramid network, includes: The feature pyramid network performs upsampling processing on the multi-level feature map to obtain an upsampled image, and the upsampled image is input as the input feature X into the enhanced feature fusion module; The feature pyramid network receives the second feature map, the third feature map, and the fourth feature map output by the second convolutional layer, the third convolutional layer, and the fourth convolutional layer of the enhanced core aggregation initial network; The second feature map, the third feature map, and the fourth feature map are respectively subjected to a 1x1 convolution layer to adjust the channel dimension to obtain a second adjusted map, a third adjusted map, and a fourth adjusted map; The second adjustment map, the third adjustment map, and the fourth adjustment map are input as input features Y into the enhanced feature fusion module.

[0010] Optionally, the step of fusing the input feature X and the input feature Y through the enhanced feature fusion module, extracting channel enhancement features and spatial enhancement features of the input feature X and the input feature Y, and performing weighted fusion on the channel enhancement features, the spatial enhancement features, the input feature X, and the input feature Y to obtain a target fused feature map includes: The enhanced feature fusion module fuses the input feature X and the input feature Y to obtain an initial fused feature map; The enhanced feature fusion module performs channel feature extraction and spatial feature extraction on the initial fusion feature map to obtain channel enhancement features and spatial enhancement features; Using a learning-based attention weighting mechanism to fuse the channel enhancement feature and the spatial enhancement feature to obtain a fused enhancement feature; A weighted sum is performed on the fusion enhancement feature, the input feature X, and the input feature Y to obtain a target fusion feature map.

[0011] Optionally, the method for calculating the loss value during the detector training process includes the following steps: Obtain the predicted bounding box and predicted category of the sample object predicted by the initial detector to be trained in this training, and obtain the true bounding box and true category of the sample object from the sample data; Calculate the loss value between the predicted category and the true category to obtain the classification loss; Calculating a regression loss based on the position and shape of the predicted bounding box and the true bounding box; The weighted sum of the classification loss and the regression loss is taken to obtain the loss value of this training.

[0012] Optionally, calculating the regression loss according to the position and shape of the predicted bounding box and the true bounding box includes: Determining a bounding box intersection-to-union ratio based on the intersection and union of the predicted bounding box and the true bounding box; Determining a first weight coefficient in the horizontal direction and a second weight coefficient in the vertical direction according to the width and height of the true bounding box and a hyperparameter for adjusting the influence of the aspect ratio on the weight; Obtaining the center coordinates of the predicted bounding box and the true bounding box, and obtaining the diagonal length of the minimum enclosing box that can simultaneously contain the predicted bounding box and the true bounding box; Determining a distance shape item based on the first weight coefficient, the second weight coefficient, the center point coordinates of the predicted bounding box and the true bounding box, and the diagonal length; the distance shape item is used to measure the shape difference between the predicted bounding box and the true bounding box; Determining a shape penalty term according to the width and height of the predicted bounding box, the width and height of the true bounding box, and the first weight coefficient and the second weight coefficient; The regression loss is calculated based on the bounding box intersection-over-union ratio, the distance shape term, and the shape penalty term.

[0013] A second aspect of the present invention provides a cross-scale spatial object detection device, comprising: A feature extraction module is used to sequentially process the original image through multiple convolutional layers of the enhanced poly-kernel inception network, use multiple convolutions to extract multi-scale features and contextual features of the original image, and use global average pooling and strip convolution to extract the relationship between long-distance pixels in the original image, and output a multi-level feature map from the last convolutional layer; An input module is configured to perform upsampling processing and channel dimension adjustment processing on the multi-level feature maps and the multiple feature maps output by the convolutional layer through a feature pyramid network, and input the processed input features X and input features Y into the enhanced feature fusion module of the feature pyramid network; a fusion module, configured to fuse the input feature X and the input feature Y through the enhanced feature fusion module, extract the channel enhancement feature and the spatial enhancement feature of the input feature X and the input feature Y, and perform weighted fusion on the channel enhancement feature, the spatial enhancement feature, the input feature X, and the input feature Y to obtain a target fused feature map; A detection module is configured to feed the target fused feature map into a Transformer-based detector, generate an object query through an IoU-aware query selection mechanism, interact with the object query and the target fused feature map, and predict the object category and bounding box coordinates of the target object in the original image.

[0014] A third aspect of the present invention proposes an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the cross-scale spatial target detection method as described in the first aspect.

[0015] The fourth aspect of the present invention proposes a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the cross-scale spatial target detection method as described in the first aspect.

[0016] The embodiments of the present invention have the following beneficial effects: The cross-scale spatial target detection method provided by an embodiment of the present invention comprises: sequentially processing the original image through multiple convolution layers of an enhanced core-aggregation initial network, using multiple convolutions to extract the multi-scale features and context features of the original image, and using global average pooling and strip convolution to extract the relationship between long-distance pixels in the original image, outputting a multi-level feature map from the last convolution layer; performing upsampling and channel dimension adjustment on the multi-level feature map and the multiple feature maps output by the convolution layer through a feature pyramid network, and inputting the processed input features X and Y into the enhanced feature pyramid network of the feature pyramid network. In the fusion module, the input features X and Y are fused through the enhanced feature fusion module, and the channel enhancement features and spatial enhancement features of the input features X and Y are extracted. The channel enhancement features, the spatial enhancement features, and the input features X and Y are weightedly fused to obtain a target fusion feature map. The target fusion feature map is fed into a Transformer-based detector, and an object query is generated through an IoU-aware query selection mechanism. The object query and the target fusion feature map are interacted to predict the object category and bounding box coordinates of the target object in the original image. This solution significantly improves the detection capability of targets with drastic size changes in a spatial environment through the synergistic effect of the enhanced poly-kernel initiation network and the feature pyramid network. In addition, the application of deformable convolution in the enhanced poly-kernel initiation network enables the model to better handle the geometric deformation of satellite components caused by changes in viewpoint. While improving performance, the detection process is simplified without the need for non-maximum suppression post-processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A network structure diagram of a cross-scale spatial target detection method provided by an embodiment of the present invention; Figure 2 A flowchart of the steps of a cross-scale spatial target detection method provided by an embodiment of the present invention; Figure 3 A schematic diagram of a processing channel of a convolutional layer provided in an embodiment of the present invention; Figure 4 A schematic diagram of an E_PKI block provided in an embodiment of the present invention; Figure 5 This is a structural block diagram of a cross-scale spatial target detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this solution, not all of them. Based on the embodiments of this solution, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this solution.

[0019] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values ​​may be based on additional conditions or values ​​beyond the stated in practice.

[0020] Figure 1 A network structure diagram of a cross-scale spatial target detection method provided by an embodiment of the present invention.

[0021] like Figure 1 As shown in the figure, the network structure mainly consists of three modules. (1) On the left is the enhanced poly-kernel inception network (E_PKINet), which serves as the backbone network and is responsible for extracting features from the original image. It also adds an adjustment module for improvement. E_PKINet uses a non-expanded multi-scale convolution kernel mixed with deformable convolution and dilated convolution to extract target features of different scales and capture local context. In addition, an improved contextual anchor attention (CAA) module is introduced in parallel to capture long-range image context information. These two components work together to improve the performance of E_PKINet. (2) In the middle is the optimized multi-scale feature pyramid network (iFPN). Its core task is to efficiently fuse the multi-level feature maps output from different stages of the enhanced PKINet backbone network. It can effectively cope with the huge scale changes of key components in spatial target detection (such as solar panels, bodies, etc.) and enhance the detection ability of small-sized targets. (3) On the right is the detection head, which consists of two parallel branches, classification and regression, and is used to finally output the detection bounding box and classification results.

[0022] Figure 2 A flow chart of the steps of a cross-scale spatial target detection method provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps: Step 101: sequentially process the original image through multiple convolutional layers of the enhanced poly-kernel inception network, use multiple convolutions to extract multi-scale features and context features of the original image, and use global average pooling and strip convolution to extract the relationship between long-distance pixels in the original image, and output a multi-level feature map from the last convolution layer.

[0023] The original image is received and features are extracted using an Enhanced Kernel-Polymerized Inception Network (E_PKINet) as the backbone network. E_PKINet includes a combination of multi-scale convolutional kernels, a deformable convolution module, and a contextual anchor attention (CAA) module. This effectively extracts object features across different scales, captures rich local contextual information, and adapts to geometric deformations of spatial objects, ultimately outputting a multi-level feature map.

[0024] The Enhanced Kernel-Polymerized Inception Network (E_PKINet) arranges multiple non-expanded depthwise convolution kernels of varying sizes in parallel and introduces dilated convolutions to expand the receptive field, making it suitable for capturing features at different scales and contextual features. Deformable convolutions are added to adjust the sampling positions of the convolution kernels by learning offsets, better adapting to geometric deformations of the target. These enhancements extract dense texture features across different receptive fields. These texture features are adaptively fused along the channel dimension, enabling the collection of local contextual information.

[0025] To further incorporate distant image context, the improved Context Anchor Attention (CAA) mechanism utilizes global average pooling and strip convolution to capture relationships between distant pixels and enhance features within the central region. Adaptive kernel resizing is also incorporated into the CAA module. By introducing a lightweight kernel size prediction mechanism, the convolution kernel dynamically adjusts based on input features, enabling more flexible capture of contextual information at different scales.

[0026] By combining E_PKINet and improved CAA, the adaptive feature extraction of local and global contextual information is promoted, thereby improving the performance of spatial cross-scale object detection.

[0027] As an optional embodiment, each of the multiple convolutional layers includes a first processing channel consisting of a feedforward network and a second processing channel consisting of a plurality of E_PKI blocks connected, and each convolutional layer is processed according to the following steps: Step 1011: Obtain layer input information from the previous convolutional layer, and divide the layer input information into two halves according to the channel sizes of the first processing channel and the second processing channel to obtain first input information and second input information respectively; Step 1012: Feed the first input information into the feedforward network for feature extraction to obtain first output information; Step 1013: Feed the second input information to the E_PKI block for feature extraction to obtain second output information. Step 1014: Concatenate the first output information and the second output information and perform a 1×1 convolution operation to obtain layer output information, and transmit the layer output information to the next convolution layer.

[0028] In steps 1011 to 1014, as Figure 1 As shown, the multiple convolutional layers can be four convolutional layers, each convolutional layer corresponding to one stage.

[0029] Figure 3 A schematic diagram of a processing channel of a convolutional layer provided in an embodiment of the present invention.

[0030] like Figure 3 As shown, the convolutional layer Input After downsampling and convolution, it is split into the first input information and the second input information and fed into two channels: (1) (2) in, represents a 3×3 convolution operation, and downsample represents a downsampling operation.

[0031] Represents the convolutional layer The input, where This symbol represents the set of real numbers. In the context of deep learning, it means that the elements that make up the tensor (multidimensional array) are all real numbers; C represents the number of channels in the feature map. In each layer of the neural network, the channels capture different features of the input data. In the formula, Refers to the l The number of channels in the layer, and It means that the number of channels of this layer is halved, that is, the input information is divided into two halves according to the channel size and sent to two processing channels respectively; H represents the height of the feature map (Height), usually in pixels; W represents the width of the feature map (Width), usually in pixels.

[0032] The first processing channel is the Feed-Forward Network (FFN). It receives the first input information , processing to obtain the first output information .

[0033] The full meaning is: variable Is a three-dimensional tensor whose internal values ​​are all real numbers. The dimension of this tensor is half the number of channels ( ),high( ) and width ( ) are defined together, these dimension parameters are related to the first l Layer feature map related.

[0034] The second processing channel consists of a series of E_PKI blocks, receiving the second input information , processing to obtain the second output information Convolutional layer The final output is: (3) Among them, Concat represents a concatenation operation.

[0035] As an optional embodiment, the E_PKI block includes an E_PKI module and a CAA module, and step 1013 includes: Step 10131: The E_PKI module uses multiple convolutions in parallel to extract multi-scale features and contextual features of the second input information; the multiple convolutions include 1×1 convolution, 3×3 convolution, 3×3 dilated convolution, and 3×3 deformable convolution; the contextual features refer to other regional features in the image corresponding to the second input information that have semantic or spatial associations with the current region; Step 10132: The E_PKI module combines the multi-scale features and the context features by element-by-element addition to obtain a combined feature; and performs a 1×1 convolution operation on the combined feature to obtain a first convolution feature. Step 10133: The CAA module compresses the second input information into a global feature vector through global average pooling, and maps the global feature vector into a convolution kernel size parameter using a two-layer perceptron; Step 10134: The CAA module performs one-dimensional convolution on the convolution kernel size parameter in the horizontal and vertical directions respectively to obtain a horizontal convolution kernel size and a vertical convolution kernel size; Step 10135: The CAA module generates an attention map by applying a Sigmoid function to the horizontal convolution kernel size and the vertical convolution kernel size. Step 10136: The E_PKI block multiplies the first convolution feature and the attention map to obtain a product; performs a 1×1 convolution operation on the product to obtain second output information.

[0036] In step 10131-step 10132, an E_PKI block consists of an E_PKI module and an improved CAA module. Different from general object detection, spatial cross-scale object detection aims to locate and identify objects of different sizes in an image.

[0037] Figure 4 A schematic diagram of an E_PKI block provided in an embodiment of the present invention.

[0038] like Figure 4 As shown, each E_PKI block contains an E_PKI module and an improved CAA module.

[0039] To address the challenges associated with large object scale variations, we introduce the E_PKI module to capture multi-scale texture features. Figure 1 As shown in the figure, the E_PKI module includes extracting multi-scale features by using convolution kernels of different sizes in parallel, such as 1×1 convolution, 3×3 convolution, Dilated Convolution 3×3 (hole convolution), and Deformable Convolution 3×3 (deformable convolution), capturing contextual information across multiple scales.

[0040] Dilated convolution is central to capturing contextual information. The E_PKI module utilizes multiple convolutions in parallel, including a dilated 3x3 convolution. Compared to standard convolution, dilated convolution expands its receptive field by introducing gaps between kernel elements. This means that when calculating a point on the output feature map, it can "see" a wider area on the input feature map. This "wider area" represents contextual information.

[0041] The E_PKI module utilizes multiple convolutions in parallel, including 1x1, 3x3, dilated 3x3, and deformable 3x3 convolutions. This design allows the module to simultaneously extract information of varying scales and ranges from input features. Standard 3x3 convolutions prioritize compact local features, while dilated convolutions focus on broader contextual features.

[0042] The outputs of these parallel branches are combined by element-by-element addition to form a combined feature that includes local details and contextual information.

[0043] Then perform a 1×1 convolution operation on the combined feature to obtain the first convolution feature, which is expressed as follows: (4) in, Represents the first convolutional feature. Represents a combination of features.

[0044] 1×1 convolution is a channel fusion mechanism used to integrate features of different receptive field sizes.

[0045] In this way, the E_PKI module can capture extensive contextual information without compromising the integrity of local texture features.

[0046] In steps 10133-10136, the E_PKI module in the E_PKI block focuses on extracting multi-scale local context information. To capture long-range context information, we further integrate an improved CAA module into the E_PKI block. The improved CAA aims to capture the interdependencies between distant pixels while enhancing central features.

[0047] Traditional convolution operations have a limited receptive field, focusing primarily on the local neighborhood around a pixel. However, when understanding a large object or complex scene, pixels that are far apart in the image may also be related (for example, the two ends of a large satellite). "Grasping the interdependencies between distant pixels" means that this module is able to go beyond the local scope and capture and utilize this long-range global context.

[0048] In the specific implementation, the first step is to introduce a lightweight kernel size prediction mechanism in the CAA module to achieve adaptive kernel size adjustment, so that the convolution kernel can be dynamically adjusted according to the input features, thereby more flexibly capturing contextual information of different scales. A kernel size prediction subnetwork is added to the CAA module, with the input being the feature map of the current stage and the output being the dynamic kernel size. .

[0049] In the second step, after obtaining the dynamic kernel size, the CAA module compresses the second input information into a global feature vector through global average pooling and uses a two-layer perceptron to map the global feature vector to the convolution kernel size parameter. It is expressed as follows: (5) Among them, σ is the Sigmoid function, MLP represents the perceptron, GAP represents the global pooling operation, Represents n second input information.

[0050] The third step is to use the single kernel size parameter predicted in the first step One-dimensional convolution is performed in the horizontal and vertical directions respectively to obtain the horizontal convolution kernel size and the vertical convolution kernel size.

[0051] Specifically, the parameter It will be used to define the width of the horizontal one-dimensional convolution kernel and the height of the vertical one-dimensional convolution kernel, so as to flexibly capture contextual information of different scales. The horizontal convolution kernel size Defined as (1, ), vertical convolution kernel size Defined as ( , 1).

[0052] In the fourth step, the CAA module generates an attention map by applying the Sigmoid function to the horizontal and vertical convolution kernel sizes. This is represented as follows: (6) in, represents the horizontal convolution kernel size, Represents the vertical convolution kernel size.

[0053] The Sigmoid function ensures that the attention map In the range (0,1).

[0054] Finally, the E_PKI block multiplies the first convolution feature and the attention map to obtain the product; a 1×1 convolution operation is performed on the product to obtain the second output information.

[0055] In the above steps, the improved CAA module achieves "grasping the interdependence between distant pixels" through the following operations: Global Average Pooling (GAP): compresses the entire feature map into a global feature vector, which itself captures global contextual statistics.

[0056] Dynamic 1D Conv: Performs one-dimensional convolution in both the horizontal and vertical directions. This operation, often referred to as strip pooling or strip convolution, effectively aggregates features across rows and columns, thereby capturing long-range dependencies in both the horizontal and vertical directions.

[0057] After capturing long-range contextual information, the module doesn't enhance all features equally. Instead, it aims to leverage this contextual information to identify and enhance the core feature regions that are most important for object detection. "Central features" here can be understood as features that are crucial for object recognition, not necessarily the physical center of the image.

[0058] The improved CAA module implements “enhanced center features” through the following operations: The module uses global pooling and dynamic one-dimensional convolution to generate an attention map, which represents the importance of features at different locations. Multiplying this attention map with the output of the E_PKI module amplifies the feature response of important regions, known as "center features," while suppressing irrelevant background regions.

[0059] Step 102: Upsampling and channel dimension adjustment are performed on the multi-level feature maps and the multiple feature maps output by the convolutional layer through a feature pyramid network, and the input features X and Y obtained by the processing are input into the enhanced feature fusion module of the feature pyramid network.

[0060] In order to achieve more adaptive feature aggregation and replace the simple element-by-element addition mode in the traditional feature pyramid network (FPN), the present invention optimizes the traditional feature pyramid network to obtain an optimized feature pyramid network (iFPN).

[0061] This paper designs and introduces an enhanced feature fusion module (EFM) at each fusion node. The core innovation of EFM lies in its internal dual-path feature processing and adaptive weighting mechanism. EFM first processes features from the upper layer (input feature X) and the backbone network lateral connection (input feature Y) in parallel.

[0062] Among them, the input feature X and the input feature Y are obtained by upsampling and channel dimension adjustment of multiple feature maps output by multi-level feature maps and convolutional layers.

[0063] As an optional embodiment, step 102 includes: Step 1021: The feature pyramid network performs upsampling processing on the multi-level feature map to obtain an upsampled image, and inputs the upsampled image as an input feature X into an enhanced feature fusion module; Step 1022: The feature pyramid network receives the second feature map, the third feature map, and the fourth feature map output by the second convolutional layer, the third convolutional layer, and the fourth convolutional layer of the enhanced core aggregation initial network. Step 1023: Adjust the channel dimension of each of the second feature map, the third feature map, and the fourth feature map through a 1x1 convolution layer to obtain a second adjusted map, a third adjusted map, and a fourth adjusted map; Step 1024: Input the second adjustment map, the third adjustment map, and the fourth adjustment map as input features Y into an enhanced feature fusion module.

[0064] In steps 1021 to 1024 , the input feature X comes from the previous layer of the “top-down” path in the improved feature pyramid network (iFPN), that is, a deeper layer with lower resolution.

[0065] like Figure 1 As shown in the figure, in the feature pyramid network, the upper feature map will be upsampled and then sent to the EFM of the next layer as the input feature X, and fused with the lateral connection (input feature Y) of the layer.

[0066] The input feature Y comes from the lateral connection of the backbone network (E_PKI network). Specifically, it is the feature map output from the corresponding layer of the backbone network (such as C3, C4, C5), which is adjusted to the channel dimension through a 1x1 convolution layer before being input into the EFM.

[0067] Step 103: The input feature X and the input feature Y are fused through the enhanced feature fusion module, and the channel enhancement features and the spatial enhancement features of the input feature X and the input feature Y are extracted. The channel enhancement features, the spatial enhancement features, and the input feature X and the input feature Y are weightedly fused to obtain a target fused feature map.

[0068] The core innovation of EFM lies in its internal dual-path feature processing and adaptive weighting mechanism.

[0069] EFM uses two processing paths: one path uses parallel multi-scale one-dimensional convolution kernels to extract channel features to capture inter-channel dependencies of different granularities and generate channel-enhanced features. The other path is responsible for spatial feature extraction and generates spatially enhanced features.

[0070] Subsequently, the channel enhancement features, the spatial enhancement features and the original input features (input feature X, the input feature Y) are weightedly fused to obtain a target fused feature map.

[0071] This design enables iFPN to autonomously judge and enhance important features based on the context, significantly improving the detection accuracy and robustness of key satellite components of different sizes, distances, and postures in the space environment.

[0072] As an optional embodiment, step 103 includes: Step 1031: The enhanced feature fusion module fuses the input feature X and the input feature Y to obtain an initial fused feature map. Step 1032: The enhanced feature fusion module performs channel feature extraction and spatial feature extraction on the initial fusion feature map to obtain channel enhancement features and spatial enhancement features. Step 1033: Using a learning-based attention weighting mechanism to fuse the channel enhancement feature and the spatial enhancement feature to obtain a fused enhancement feature; Step 1034: Perform weighted summation on the fusion enhancement feature, the input feature X, and the input feature Y to obtain a target fusion feature map.

[0073] In steps 1031-1034, EFM first fuses the input features X and Y. One path uses parallel multi-scale one-dimensional convolution kernels (k=3, 5, 7) to extract channel features, capturing inter-channel dependencies at varying granularities and generating channel-enhanced features R(X⊕Y). The other path extracts spatial features, generating spatially enhanced features L(X⊕Y).

[0074] Subsequently, a learning-based attention weighting mechanism is used to dynamically fuse the two enhanced features, weight the original input features, and finally output the target fused feature map Z.

[0075] Step 104: Send the target fusion feature map to a Transformer-based detector, generate an object query through an IoU-aware query selection mechanism, interact with the object query and the target fusion feature map, and predict the object category and bounding box coordinates of the target object in the original image.

[0076] The target fused feature map is fed into a Transformer-based detection head, which first generates object queries through an IoU-aware query selection mechanism. The Transformer decoder then interacts with the object queries and the fused feature map to directly predict the object category and bounding box coordinates of the target object.

[0077] As an optional embodiment, the method for calculating the loss value during the detector training process includes the following steps: Step 201: Obtain the predicted bounding box and predicted category of the sample object predicted by the initial detector to be trained in this training, and obtain the true bounding box and true category of the sample object from the sample data; Step 202: Calculate the loss value between the predicted category and the true category to obtain the classification loss; Step 203: Calculate the regression loss based on the position and shape of the predicted bounding box and the true bounding box; Step 204: perform a weighted summation of the classification loss and the regression loss to obtain a loss value for this training.

[0078] In steps 201-204, the bounding box regression loss, as an important component of the detector's localization branch, plays an important role in the target detection task. Existing bounding box regression methods usually consider the geometric relationship between the GT box and the predicted box, and use the relative position and shape of the bounding box to calculate the loss, while ignoring the impact of inherent properties such as the shape and scale of the bounding box on the bounding box regression. In order to make up for the shortcomings of existing research, the present invention introduces a bounding box regression method that focuses on the shape and scale of the bounding box itself: the Shape IoU method. This method can calculate the loss by focusing on the shape and scale of the bounding box itself, thereby making the bounding box regression more accurate. At the same time, the classification loss Focal Loss is added, and the two are balanced by dynamic weights. The results show that our method can effectively improve the detection performance and outperform existing methods, achieving state-of-the-art performance in different detection tasks.

[0079] Specifically, the loss value of this training is calculated through the following joint loss function: (7) in, represents the total loss value, represents the classification loss, represents the regression loss. 、 Represent the weights of classification loss and regression loss respectively.

[0080] The initial weight is set to .

[0081] The introduction of the joint loss function in this scheme, especially Shape-IoU, makes the bounding box regression more accurate and effectively reduces the positioning error.

[0082] As an optional embodiment, step 203 includes: Step 2031: Determine a bounding box intersection-to-union ratio based on the intersection and union of the predicted bounding box and the true bounding box.

[0083] The calculation formula for the bounding box intersection-union ratio is as follows: (8) Where IoU represents the intersection over union of bounding boxes, and B represents the area of ​​the predicted bounding box. gt Represents the area of ​​the Ground Truth Bounding Box.

[0084] Step 2032: Determine a first weight coefficient in the horizontal direction and a second weight coefficient in the vertical direction according to the width and height of the true bounding box and a hyperparameter for adjusting the influence of the aspect ratio on the weight.

[0085] The first weight coefficient ww and the second weight coefficient hh The calculation formula is as follows: (9) (10) in, , Represents the width and height of the ground truth bounding box. scale is a hyperparameter used to adjust the influence of the aspect ratio on the weight.

[0086] Step 2033: Obtain the center coordinates of the predicted bounding box and the true bounding box, and obtain the diagonal length of the minimum enclosing box that can simultaneously contain the predicted bounding box and the true bounding box.

[0087] Step 2034: Determine a distance shape term based on the first weight coefficient, the second weight coefficient, the center point coordinates of the predicted bounding box and the true bounding box, and the diagonal length; the distance shape term is used to measure the shape difference between the predicted bounding box and the true bounding box.

[0088] The distance shape term is calculated as follows: (11) in,( xc,yc ) represents the center point coordinates of the predicted bounding box. Represents the coordinates of the center point of the true bounding box. c represents the diagonal length of the minimum closure box that can contain both the predicted box and the true box.

[0089] Step 2035: Determine a shape penalty term based on the width and height of the predicted bounding box, the width and height of the true bounding box, the first weight coefficient, and the second weight coefficient.

[0090] The calculation formula of the shape penalty term is as follows: (12) (13) in, It is a loss term that measures the difference in shape between the predicted box and the true box, that is, the shape penalty term.

[0091] ω t (i.e. ω w and ωh ) represents the difference between the predicted bounding box and the true bounding box in width or height. The larger the value of ωt, the greater the difference between the predicted shape and the true shape; when ωt is 0, it means that the size matches perfectly. When t=w, ω t It refers to ω w ; When t=h, ω t It refers to ω h .

[0092] The role of is to convert an unbounded size difference value ω t , smoothly and nonlinearly converted into a bounded score that can be understood as "shape similarity". This design makes the penalty for shape differences smoother and more stable during model optimization.

[0093] also, w,h denote the width and height of the predicted bounding box respectively. θ is a fixed hyperparameter used to control the curvature of the loss, which is set to 4 here.

[0094] Step 2036: Calculate the regression loss based on the bounding box intersection-over-union ratio, the distance shape term, and the shape penalty term.

[0095] The bounding box regression loss is calculated as: (14) Among them, IoU represents the intersection over union of bounding boxes, represents the shape distance term, represents the shape penalty term.

[0096] The present invention has the following beneficial effects compared with the prior art through the above technical solution: Improved cross-scale detection performance: Through the synergy of E_PKINet and iFPN, especially the enhanced utilization of shallow features and the design of bidirectional information flow, the detection ability of targets with drastic size changes in spatial environments is significantly improved.

[0097] Enhanced adaptability to deformable targets: The application of deformable convolution in E_PKINet enables the model to better handle the geometric deformation of satellite components caused by changes in perspective.

[0098] Improved positioning accuracy: The introduction of the joint loss function, especially Shape-IoU, makes bounding box regression more accurate and effectively reduces positioning error.

[0099] It maintains the advantages of end-to-end detection: while improving performance, it does not require NMS (non-maximum suppression) post-processing, simplifying the detection process.

[0100] In response to the problems faced by on-orbit detection of key components of failed space satellites, such as drastic changes in target scale, complex imaging conditions, and limited performance of existing detection algorithms in these scenarios, the present invention proposes and implements a cross-scale space target detection model. The core goal of the present invention is to improve the detection accuracy and robustness of cross-scale satellite key components in complex space environments. The model introduces the optimized poly-kernel inception network PKINet as the backbone, and utilizes its multi-scale convolution, deformable convolution and context-anchored attention mechanism to improve cross-scale feature extraction and deformation adaptation capabilities. At the same time, the improved feature pyramid network iFPN is adopted in the neck network, and the accuracy of bounding box regression is optimized in combination with the joint loss function. Experimental results on the constructed dataset show that the method of the present invention achieved 77.49% in the mAP indicator, which is 1.06% higher than the baseline RT-DETR, and shows superiority in both recall rate and specific component AP, verifying the effectiveness of the proposed improvement strategy in improving the performance of space target detection.

[0101] In summary, the cross-scale spatial object detection method provided by the embodiment of the present invention sequentially processes the original image through multiple convolutional layers of the enhanced poly-kernel inception network, uses multiple convolutions to extract multi-scale features and contextual features of the original image, and adopts global average pooling and strip convolution to extract the relationship between long-distance pixels in the original image, and outputs a multi-level feature map from the last convolutional layer; Performing upsampling processing and channel dimension adjustment processing on the multi-level feature maps and the multiple feature maps output by the convolution layer through a feature pyramid network, and inputting the processed input features X and input features Y into an enhanced feature fusion module of the feature pyramid network; fusing the input features X and the input features Y through the enhanced feature fusion module, extracting channel enhancement features and spatial enhancement features of the input features X and the input features Y, and performing weighted fusion on the channel enhancement features, the spatial enhancement features, the input features X and the input features Y to obtain a target fused feature map; The target fusion feature map is fed into a Transformer-based detector, where an object query is generated through an IoU-aware query selection mechanism. The object query and the target fusion feature map are interacted to predict the object category and bounding box coordinates of the target object in the original image. This solution significantly improves the detection capability of targets with drastic size changes in space environments through the synergistic effect of the enhanced poly-kernel inception network and the feature pyramid network. In addition, the application of deformable convolution in the enhanced poly-kernel inception network enables the model to better handle the geometric deformation of satellite components caused by changes in viewpoint. While improving performance, it eliminates the need for non-maximum suppression post-processing, simplifying the detection process.

[0102] Figure 5 This is a structural block diagram of a cross-scale spatial target detection device provided by an embodiment of the present invention. Figure 5 As shown, the device 300 includes: A feature extraction module 301 is configured to sequentially process an original image through multiple convolutional layers of an enhanced poly-kernel inception network, extract multi-scale features and contextual features of the original image using multiple convolutions, and employ global average pooling and strip convolution to extract relationships between distant pixels in the original image, outputting a multi-level feature map from the last convolutional layer. An input module 302 is configured to perform upsampling and channel dimension adjustment on the multi-level feature maps and the multiple feature maps output by the convolutional layer through a feature pyramid network, and input the processed input features X and Y into an enhanced feature fusion module of the feature pyramid network; A fusion module 303 is configured to fuse the input feature X and the input feature Y through the enhanced feature fusion module, extract channel enhancement features and spatial enhancement features of the input feature X and the input feature Y, and perform weighted fusion on the channel enhancement features, the spatial enhancement features, and the input feature X and the input feature Y to obtain a target fused feature map. The detection module 304 is used to feed the target fusion feature map into a Transformer-based detector, generate an object query through an IoU-aware query selection mechanism, interact with the object query and the target fusion feature map, and predict the object category and bounding box coordinates of the target object in the original image.

[0103] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0104] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, which can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0105] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0106] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A cross-scale spatial target detection method, characterized in that: The method comprises: The original image is sequentially processed through multiple convolutional layers of the enhanced poly-kernel inception network, multiple convolutions are used to extract multi-scale features and contextual features of the original image, and global average pooling and strip convolution are used to extract the relationship between long-distance pixels in the original image, and a multi-level feature map is output from the last convolutional layer; Performing upsampling and channel dimension adjustment on the multi-level feature maps and the multiple feature maps output by the convolutional layer through a feature pyramid network, and inputting the obtained input features X and Y into an enhanced feature fusion module of the feature pyramid network; fusing the input feature X and the input feature Y through the enhanced feature fusion module, extracting the channel enhancement feature and the spatial enhancement feature of the input feature X and the input feature Y, and performing weighted fusion on the channel enhancement feature, the spatial enhancement feature, the input feature X, and the input feature Y to obtain a target fused feature map; The target fused feature map is fed into a Transformer-based detector, an object query is generated through an IoU-aware query selection mechanism, the object query and the target fused feature map are interacted, and the object category and bounding box coordinates of the target object in the original image are predicted.

2. The method according to claim 1, characterized in that Each of the multiple convolutional layers includes a first processing channel consisting of a feedforward network and a second processing channel consisting of a plurality of E_PKI blocks connected together, and each of the convolutional layers is processed according to the following steps: Obtaining layer input information from the previous convolutional layer, and dividing the layer input information into two halves according to the channel size of the first processing channel and the second processing channel to obtain first input information and second input information respectively; Feeding the first input information into the feedforward network for feature extraction processing to obtain first output information; Feeding the second input information to the E_PKI block for feature extraction processing to obtain second output information; The first output information and the second output information are connected and then a 1×1 convolution operation is performed to obtain layer output information, and the layer output information is transmitted to the next convolution layer.

3. The method according to claim 2, characterized in that The E_PKI block includes an E_PKI module and a CAA module, and the second input information is fed to the E_PKI block for feature extraction processing to obtain second output information, including: The E_PKI module uses multiple convolutions in parallel to extract multi-scale features and contextual features of the second input information; the multiple convolutions include 1×1 convolution, 3×3 convolution, 3×3 dilated convolution, and 3×3 deformable convolution; the contextual features refer to other regional features in the image corresponding to the second input information that have semantic or spatial associations with the current region; The E_PKI module combines the multi-scale features and the context features by element-by-element addition to obtain a combined feature; performs a 1×1 convolution operation on the combined feature to obtain a first convolution feature; The CAA module compresses the second input information into a global feature vector through global average pooling, and maps the global feature vector into a convolution kernel size parameter using a two-layer perceptron; The CAA module performs one-dimensional convolution on the convolution kernel size parameter in the horizontal and vertical directions respectively to obtain the horizontal convolution kernel size and the vertical convolution kernel size; The CAA module generates an attention map by applying a Sigmoid function to the horizontal convolution kernel size and the vertical convolution kernel size; The E_PKI block multiplies the first convolution feature and the attention map to obtain a product; performs a 1×1 convolution operation on the product to obtain second output information.

4. The method according to claim 1, wherein The step of processing the multi-level feature maps and the multiple feature maps output by the convolutional layer through a feature pyramid network and inputting the processed input features X and Y into an enhanced feature fusion module of the feature pyramid network includes: The feature pyramid network performs upsampling processing on the multi-level feature map to obtain an upsampled image, and the upsampled image is input as the input feature X into the enhanced feature fusion module; The feature pyramid network receives the second feature map, the third feature map, and the fourth feature map output by the second convolutional layer, the third convolutional layer, and the fourth convolutional layer of the enhanced core aggregation initial network; The second feature map, the third feature map, and the fourth feature map are respectively subjected to a 1x1 convolution layer to adjust the channel dimension to obtain a second adjusted map, a third adjusted map, and a fourth adjusted map; The second adjustment map, the third adjustment map, and the fourth adjustment map are input as input features Y into the enhanced feature fusion module.

5. The method according to claim 1, wherein The enhanced feature fusion module is used to fuse the input feature X and the input feature Y, and the channel enhancement feature and the spatial enhancement feature of the input feature X and the input feature Y are extracted, and the channel enhancement feature, the spatial enhancement feature and the input feature X and the input feature Y are weightedly fused to obtain a target fused feature map, including: The enhanced feature fusion module fuses the input feature X and the input feature Y to obtain an initial fused feature map; The enhanced feature fusion module performs channel feature extraction and spatial feature extraction on the initial fusion feature map to obtain channel enhancement features and spatial enhancement features; Using a learning-based attention weighting mechanism to fuse the channel enhancement feature and the spatial enhancement feature to obtain a fused enhancement feature; A weighted sum is performed on the fusion enhancement feature, the input feature X, and the input feature Y to obtain a target fusion feature map.

6. The method according to claim 1, characterized in that The method for calculating the loss value during the detector training process includes the following steps: Obtain the predicted bounding box and predicted category of the sample object predicted by the initial detector to be trained in this training, and obtain the true bounding box and true category of the sample object from the sample data; Calculate the loss value between the predicted category and the true category to obtain the classification loss; Calculating a regression loss based on the position and shape of the predicted bounding box and the true bounding box; The weighted sum of the classification loss and the regression loss is taken to obtain the loss value of this training.

7. The method according to claim 6, characterized in that The calculating the regression loss according to the position and shape of the predicted bounding box and the true bounding box includes: Determining a bounding box intersection-to-union ratio based on the intersection and union of the predicted bounding box and the true bounding box; Determining a first weight coefficient in the horizontal direction and a second weight coefficient in the vertical direction according to the width and height of the true bounding box and a hyperparameter for adjusting the influence of the aspect ratio on the weight; Obtaining the center coordinates of the predicted bounding box and the true bounding box, and obtaining the diagonal length of the minimum enclosing box that can simultaneously contain the predicted bounding box and the true bounding box; Determining a distance shape item based on the first weight coefficient, the second weight coefficient, the center point coordinates of the predicted bounding box and the true bounding box, and the diagonal length; the distance shape item is used to measure the shape difference between the predicted bounding box and the true bounding box; Determining a shape penalty term according to the width and height of the predicted bounding box, the width and height of the true bounding box, and the first weight coefficient and the second weight coefficient; The regression loss is calculated based on the bounding box intersection-over-union ratio, the distance shape term, and the shape penalty term.

8. A cross-scale spatial target detection device, characterized in that: The device comprises: A feature extraction module is used to sequentially process the original image through multiple convolutional layers of the enhanced poly-kernel inception network, use multiple convolutions to extract multi-scale features and contextual features of the original image, and use global average pooling and strip convolution to extract the relationship between long-distance pixels in the original image, and output a multi-level feature map from the last convolutional layer; An input module is configured to perform upsampling processing and channel dimension adjustment processing on the multi-level feature maps and the multiple feature maps output by the convolutional layer through a feature pyramid network, and input the processed input features X and input features Y into the enhanced feature fusion module of the feature pyramid network; a fusion module, configured to fuse the input feature X and the input feature Y through the enhanced feature fusion module, extract the channel enhancement feature and the spatial enhancement feature of the input feature X and the input feature Y, and perform weighted fusion on the channel enhancement feature, the spatial enhancement feature, the input feature X, and the input feature Y to obtain a target fused feature map; A detection module is configured to feed the target fused feature map into a Transformer-based detector, generate an object query through an IoU-aware query selection mechanism, interact with the object query and the target fused feature map, and predict the object category and bounding box coordinates of the target object in the original image.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the cross-scale spatial target detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the cross-scale spatial target detection method according to any one of claims 1 to 7.

Images