A lightweight star catalog target detection method and system

Abstract

This invention discloses a lightweight star catalog target detection method and system. The method includes: constructing a student model, which employs a lightweight feature extractor and a Transformer decoder with width and depth pruning; constructing a teacher model, which employs an un-lightweight feature extractor and a Transformer decoder without width and depth pruning; adjusting the parameters of the teacher and student models, calculating the value of the loss function, stopping the iteration when the value of the loss function is minimized, and obtaining trained teacher and student models; and using the trained student model to perform lightweight star catalog target detection. The advantages of this invention are: lightweight, low computational complexity, and the ability to achieve 3D position detection of star catalog targets.

Description

Technical Field

[0001] This invention relates to the field of 3D target detection technology, specifically to a lightweight star catalog target detection method and system. Background Technology

[0002] Using probes to carry out deep space exploration missions is an important way for humanity to explore the universe. As an important component of probes, rovers can move, explore, and sample on the surface of celestial bodies. Their flexibility greatly expands the probe's detection range and functions. However, when carrying out missions, rovers inevitably encounter various obstacles such as rocks. Accurately identifying and locating these obstacles is crucial to ensuring the safety of the rover and advancing the exploration mission.

[0003] Currently, deep learning algorithms for target detection on celestial surfaces mainly fall into two categories: active scanning based on lidar and passive perception based on optical cameras. Lidar, due to its high cost and energy consumption, is primarily used for landing area perception and obstacle avoidance during probe landing. On rovers, optical cameras, with their large receptive field, low deployment cost, and versatility for various missions, are widely used in major deep space exploration projects worldwide. Some methods utilize convolutional UNet networks to detect obstacles in input images. However, in current deep space exploration missions, image-based obstacle detection can mostly only locate the two-dimensional position of obstacles on the image plane. It requires combining other measurement methods, such as stereo matching between binocular cameras, to obtain depth information in order to determine the actual spatial location of the obstacles. For example, the celestial surface obstacle recognition method and device disclosed in Chinese Patent Publication No. CN 112528808A can only identify the obstacle category, but cannot obtain the obstacle's spatial location. The image-based obstacle detection methods described above have significant drawbacks: they have high computational overhead, which severely affects the real-time performance of the detection algorithm; and when dealing with detection environments with simple textures or large differences in lighting, the matching accuracy drops significantly, which seriously affects the robustness of the obstacle avoidance algorithm.

[0004] In recent years, some deep learning methods in the fields of intelligent mobile robots and autonomous driving have attempted to directly predict the 3D position information of obstacles from explicit 2D image input. Compared with traditional methods for predicting 2D bounding boxes, methods that estimate 3D bounding boxes solely from images face greater challenges because the lack of depth information makes explicitly recovering 3D information from 2D input data an ill-posed problem. Furthermore, traditional image detection models are computationally complex and resource-intensive, making them difficult to adapt to the limited computing resources and storage capabilities of space observation equipment. Meanwhile, targets on celestial surfaces are characterized by sparse distribution and blurred boundaries, requiring image detection algorithms to maintain high accuracy while achieving lightweight design to meet the demands of real-time processing and energy efficiency optimization. Summary of the Invention

[0005] The technical problem to be solved by this invention is how to provide a lightweight, low-computational-complexity 3D position detection method for star catalog targets.

[0006] This invention solves the above-mentioned technical problems through the following technical means: a lightweight star catalog target detection method, comprising:

[0007] S1. Construct a student model. The student model uses a lightweight feature extractor connected to a Transformer decoder that has undergone width and depth pruning. S2. Construct a teacher model, which uses a feature extractor without lightweight processing and connects it to a Transformer decoder without width and depth pruning. S3. Adjust the parameters of the teacher model and student model, calculate the value of the loss function, and stop iterating when the value of the loss function is minimized. This will give you the trained teacher model and student model. Use the trained student model to output the object category and 3D bounding box through the detection head.

[0008] Furthermore, the execution process of the teacher model and the student model is as follows: The feature extractor extracts 2D features from the stereo image pairs to obtain 2D multi-view features. In the 3D coordinate generator, the coordinates of the stereo image pairs are transformed using camera parameters to generate 3D coordinates. The generated 3D coordinates, together with the 2D multi-view features, are input into the implicit 3D feature encoder to obtain an implicit 3D representation. The implicit 3D representation is flattened and input into the Transformer decoder, where it interacts with the object query generated by the query generator to obtain an updated object query. The updated object query of the last layer of the Transformer decoder is input into the detection head to predict the object category and 3D bounding box.

[0009] Furthermore, the lightweight feature extractor includes: Provide stereo images of the Martian surface. , Indicates the left eye image. Representing the right eye image, ResNet-34 is used as a lightweight feature extractor to extract 2D features from the binocular image pair, resulting in 2D multi-view features.

[0010] Furthermore, the unweighted feature extractor is ResNet-50.

[0011] Furthermore, the width pruning includes: The Transformer decoder consists of L sequentially connected decoder layers. Each layer includes a multi-head attention module and a feedforward network module, with the multi-head attention module connected to the feedforward network module. Each multi-head attention module includes... The first attention point, for the first The importance score of the h-th attention head in layer h. The estimate using a first-order Taylor expansion is:

[0012] in, Indicates the first Add attention head to the layer The loss value of the student model afterwards, This indicates that no attention head has been added. The loss value of the student model at that time. Indicates the first Add attention head to the layer Then, the partial derivative of the student model loss with respect to the attention head is calculated. The remaining part after expanding the first-order function of each attention head; after obtaining the importance score of each attention head, sort them in descending order of importance score to form an attention head sorting sequence; and then prune them according to a preset ratio. Once the number of attention heads retained in the attention head sorting sequence is determined, the retained set is:

[0013] The remaining set is the set of attention points to be pruned:

[0014] in, The first The first layer of attention head sorting sequence One's attention, , Indicates the floor function; Focus on pruning The linear mapping parameters of the attention heads are removed from the multi-head attention module. For the output projection matrix of the multi-head attention module, the row blocks corresponding to the pruned attention heads are deleted.

[0015] Furthermore, the width pruning also includes: For the feedforward network module, it includes two layers of linear transformation:

[0016] Wherein, FFN represents the feedforward network module. This represents the input data for FFN. This represents the activation function. and They are two linear mappings. and There are two biases; the intermediate layer neurons of the FFN are divided into There are groups, each containing s neurons, satisfying... , Representing the intermediate layer dimension, the importance score of each group of neurons is calculated as follows:

[0017] in, Indicates the first A group of neurons were added. The loss value of the student model afterwards, This indicates that no neurons were added. The loss value of the student model at that time. Indicates the first A group of neurons were added. Then, the partial derivative of the student model loss with respect to the neuron is taken. The remaining part after expanding the first-order function of the importance of each group of neurons; after obtaining the importance score of each group of neurons, sort them in descending order of importance score to form a neuron ranking sequence; determine the set of retained groups in the neuron ranking sequence according to a predefined pruning ratio. and the group to be pruned , for those belonging The neuron group, deleted from its The corresponding column, and The corresponding bias items are deleted synchronously. The corresponding row in the middle.

[0018] Furthermore, the depth pruning includes: Suppose the Transformer decoder contains sequentially connected... There are 1 decoder layer, and each decoder layer is numbered sequentially according to the forward propagation order of the network. For indexes The decoder layer determines whether to perform pruning based on the following congruence judgment condition:

[0019] in, This indicates a congruence relation, and mod represents the modulo operation. This represents rounding down. Preset pruning probability; When the congruence condition is true, for index 1 The decoder layer is removed from the Transformer decoder structure.

[0020] Furthermore, the loss function is:

[0021] in, and For predefined hyperparameters, For the loss of 3D object detection, For characteristic consistency distillation loss, This represents the loss during distillation in a hidden state.

[0022] Furthermore, the characteristic consistency distillation loss The calculation method is as follows:

[0023] in, and These represent the features output by the feature extractors for the student model and the teacher model, respectively. Indicates MSE loss; The hidden state distillation loss The calculation method is as follows

[0024] in, and The Transformer decoders representing the student model and teacher model respectively are the first two. Layer decoder layer output features.

[0025] The present invention also provides a system for performing the above-described lightweight star catalog target detection method, the system comprising: The student model building module is used to build student models, which use a lightweight feature extractor connected to a Transformer decoder that has undergone width and depth pruning. The teacher model building module is used to build a teacher model, which uses a feature extractor without lightweight processing and connects it to a Transformer decoder without width pruning and depth pruning. The model training module is used to adjust the parameters of the teacher model and the student model, calculate the value of the loss function, and stop iterating when the value of the loss function is minimized, thus obtaining the trained teacher model and student model. The trained student model is then used to output the object category and 3D bounding box through the detection head.

[0026] The advantages of this invention are: (1) The teacher model of the present invention is used to provide high-quality feature representation and intermediate layer information. The student model learns the knowledge in the teacher model through training. The two work together to improve the target detection effect. In addition, the student model uses a lightweight feature extractor and a Transformer decoder that has been processed by width pruning and depth pruning, thereby reducing the computational overhead and storage requirements of the model, thus realizing lightweight and low computational complexity 3D position detection of star table targets.

[0027] (2) This invention enables the direct prediction of the 3D position of a target on a star surface from binocular images without relying on lidar or explicit 3D reconstruction. Specifically, by constructing a 3D coordinate generator, the 2D pixels in the binocular images are mapped to 3D space, and the generated 3D coordinates are input together with 2D multi-view features into an implicit 3D feature encoder for fusion modeling, enabling the network to simultaneously perceive the spatial position and semantic information of the target at the feature level. This method avoids the high dependence on texture and lighting conditions in traditional stereo matching or depth estimation processes, improves the robustness of 3D localization under conditions of sparse target distribution and blurred boundaries in star surface scenes, and is suitable for complex environments such as deep space exploration.

[0028] (3) This invention employs a lightweight strategy combining structural pruning and distillation training, which significantly reduces model complexity and parameter size while maintaining high detection accuracy. On one hand, by lightweighting and replacing the backbone network (i.e., the feature extractor) and jointly pruning the Transformer decoder in both width and depth directions, the number of neurons, multi-head attention scale, and decoder layers are effectively reduced, thus lowering the computational overhead and storage requirements of the model from a structural perspective. On the other hand, the introduction of a teacher model for supervision during the training phase, through feature consistency distillation loss and hidden state distillation loss, alleviates the performance degradation problem caused by model compression. Experimental results show that even with a significant reduction in the number of parameters, the lightweight model can still maintain detection accuracy close to that of the teacher model, thus achieving a good balance between model performance and resource consumption, making it suitable for deployment in star surface exploration equipment with limited computational resources. Attached Figure Description

[0029] Figure 1 This is a general framework diagram of a lightweight star catalog target detection method disclosed in an embodiment of the present invention; Figure 2 This is a schematic diagram of an implicit 3D feature encoder in a lightweight star catalog target detection method disclosed in an embodiment of the present invention. Figure 3 This is a flowchart of lightweight network training in a lightweight star catalog target detection method disclosed in an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example 1 like Figure 1 As shown, Embodiment 1 of the present invention provides a lightweight method for detecting targets on a Martian surface. The main technical approach of this method is to provide a pair of binocular images of the Martian surface. , Indicates the left eye image. Representing the right-eye image, ResNet50 is used to extract 2D features from the binocular image pair, resulting in 2D multi-view features. In the 3D coordinate generator, the camera frustum space is first discretized into a 3D mesh. Then, the coordinates of the mesh are transformed using camera parameters to generate coordinates in 3D world space. The generated 3D coordinates, along with the 2D multi-view features, are input into an implicit 3D feature encoder to obtain an implicit 3D representation. This implicit 3D representation is further input into a Transformer decoder and interacts with an object query generated by a query generator to obtain an updated object query. The updated object query is then input into the detection head, i.e. Figure 1 The detection decoder in the code is used to predict object categories and 3D bounding boxes.

[0032] Continue reading Figure 1 The technical principle of the 3D coordinate generator is as follows: In the 3D coordinate generator, the camera frustum space corresponding to each image in the binocular image pair is first discretized into a three-dimensional mesh. Each point in the three-dimensional mesh is represented as: .in This represents the coordinates of the i-th pixel in the image plane. It is the first A depth value, For the first An index of depth values, with a predefined depth range. ], The minimum depth value, The highest depth value, the predefined first Depth value Then the 3D coordinates of each point in the 3D mesh. It can be calculated based on the projection matrix of each camera:

[0033] (1) In the formula: , These are the camera's internal and external parameters, respectively. For camera collection, 1 represents the left eye camera and the right eye camera, respectively. Representing the i-th pixel in the image plane of the 3D mesh, respectively. The x-coordinate, y-coordinate, and vertices of the point corresponding to the depth. This represents the i-th pixel in the image plane of the 3D mesh calculated using the intrinsic and extrinsic parameters of camera n. The x-coordinate, y-coordinate, and vertices of the point corresponding to the depth. H and W represent the height and width of the 2D feature map, respectively. Furthermore, the generated 3D coordinates are regularized as follows: (2) In the formula: This represents the area in 3D space that needs to be detected. These are the upper and lower limits of the horizontal axis. These are the upper and lower limits of the ordinate. Here, represents the upper and lower bounds of the vertical coordinate. The regularized set of coordinates can be written as: (3) In the formula: It is A dimensional vector.

[0034] The structure of the implicit 3D feature encoder is as follows: Figure 2 As shown, the aim is to model the input 2D features and associate them with 3D location information to obtain an implicit 3D spatial representation. Specifically, given 2D features... and 3D coordinates First of all Input to 3D position encoder In, and convert it into 3D position embedding Then, through a 1×1 convolutional layer The 2D features are transformed and added to the 3D position embedding to form a position-aware implicit 3D representation. Finally, the position-aware implicit 3D representation is flattened and used as input to the Transformer decoder. Figure 2 In this context, C represents the number of channels. The final position-aware implicit 3D features can be represented by the following formula: (4) (5) The technical principle of the query generator and decoder is as follows: the original object detection framework DETR directly uses a set of learnable parameters as the initial object query. To alleviate the convergence difficulty in 3D scenes, this invention first initializes a set of learnable anchor points in the 3D world space using a uniform distribution from 0 to 1. Then, the coordinates of the 3D anchor points are input into a small multilayer perceptron with two linear layers to generate the initial object query. For the decoder, it follows the standard Transformer decoder in DETR, which includes L decoder layers. The interaction process in the decoder layers can be described as follows: (6) In the formula: This represents the l-th layer of the decoder. This represents the updated object query for layer l. Here, M and C represent the number of queries and channels, respectively. In each decoder layer, the object query interacts with the location-aware 3D implicit representation through multi-head attention and a feedforward network. After iterative interactions, the updated object query can be used to predict the 3D location of obstacles on the ground.

[0035] The detection head mainly consists of two branches: classification and regression. The updated object query from the Transformer decoder is fed into the detection head, which predicts the probability of the object class and the 3D bounding box. Let... and Let them represent the set of true values ​​and the set of predicted values, respectively. The actual coordinates of the anchor point. Let the predicted coordinates of the anchor point be represented. Then, the loss of 3D object detection can be summarized as: (7) (8) (9) In the formula: This represents the overall target detection loss. Represents classification loss. Indicates regression loss, For the predicted classification probability, , , For predefined hyperparameters, This represents the predicted category label under the optimal match. This represents the predicted bounding box under the optimal matching. Indicates if, Indicates other.

[0036] The above content is from the paper "Yang Wenfei, Jiang Han, Pan Xiaoyang, Li Maodeng, Zhou Ye, Zhang Tianzhu. Implicit 3D Representation Learning for Obstacle Detection on Star Surfaces [J]. Journal of Deep Space Exploration (Chinese and English), 2025, 12(2): 172-178.DOI:10.15982 / j.issn.2096-9287.2025.20240044". It mainly introduces the specific application scenarios of the lightweight method of this invention. The focus of this invention is to provide a lightweight design for the target detection method mentioned in the paper. The following details the lightweight method mentioned in this invention, which includes two parts: a pruning strategy and a training method, as follows: (1) Pruning strategy.

[0037] In the star catalog target detection method designed in this invention, the overall network structure mainly consists of two parts: a CNN backbone network (i.e., a feature extractor) and a Transformer decoder. The CNN backbone network is used to extract multi-scale two-dimensional features from the star catalog image, while the Transformer decoder is used to model the implicit three-dimensional features and output the target detection result. This invention prunes both the backbone network and the Transformer decoder. The overall technical approach is as follows: Figure 3 As shown.

[0038] Lightweighting of the backbone network. For the CNN backbone network, to reduce model complexity, this invention uses the simpler ResNet-34 instead of the original ResNet-50 as the feature extractor. By reducing the number of network layers and parameter size, while maintaining the basic feature representation ability, the computational overhead and storage requirements of the model are effectively reduced, thereby improving the model's applicability in resource-constrained scenarios.

[0039] Width pruning strategy for the Transformer decoder. The Transformer decoder consists of L decoder layers, each including a multi-head attention (MHA) module and a feed-forward network (FFN) module, with the MHA and FFN modules connected. To reduce the number of parameters and computational cost in the width direction of the decoder, this invention performs width pruning on the attention heads and intermediate neurons in the feed-forward network.

[0040] In the multi-head attention module, let the first... Layer contains There are 1 attention head, and the output of each attention head can be represented as: (10) in, , , , The linear mapping parameters corresponding to this attention head. , , These are the query vector, key vector, and value vector. They are sorted according to the importance of the attention head in the fine-tuning model, and then arranged in descending order of importance along the width. Specifically, for the... The importance score of the h-th attention head in layer h. The estimate using a first-order Taylor expansion is: (11) in, Indicates the first Add attention head to the layer The loss value of the student model afterwards, This indicates that no attention head has been added. The loss value of the student model at that time. Indicates the first Add attention head to the layer Then, the partial derivative of the student model loss with respect to the attention head is calculated. The remaining part after expanding the first-order function for each attention head; after obtaining the importance score for each attention head, sort them in descending order of importance score to form the attention head sorting sequence: (12) in, Indicates the first Layer attention head importance sorting index. Width pruning uses a preset pruning ratio. Once the number of attention heads retained in the attention head sorting sequence is determined, the retained set is: (13) The remaining set is the set of attention points to be pruned: (14) in, The first The first layer of attention head sorting sequence One's attention, , This indicates the floor function.

[0041] Treating the pruning collection In this invention, the attention heads are implemented using a structured deletion approach. This involves removing the linear mapping parameters corresponding to the attention head from the multi-head attention module and simultaneously adjusting the output projection matrix to ensure consistent dimensionality across the pruned attention modules. Specifically, multi-head attention typically includes three sets of linear mappings. , , It can be viewed as being obtained by concatenating the submatrices of each head: (15) in, .from , , Delete by index The corresponding column blocks yield the pruned matrix. , , Furthermore, for the output projection matrix... Delete the row block corresponding to the pruned attention head to obtain This ensures that the output dimension of the pruned multi-head attention module remains unchanged, allowing for normal integration with subsequent residual connections.

[0042] For feedforward network modules, two linear transformations are typically included: (16) Wherein, FFN represents the feedforward network module. This represents the input data for FFN. This represents the activation function. and They are two linear mappings. and There are two biases. Represents the intermediate layer dimension. This provides the output dimension for the multi-head attention module. Since evaluating all neurons individually is time-consuming, this invention employs a neuron grouping evaluation method: dividing the intermediate layer neurons into... There are groups, each containing s neurons, satisfying... The importance of each group of neurons is calculated similarly to formula (11), and the specific calculation is as follows: (17) in, Indicates the first A group of neurons were added. The loss value of the student model afterwards, This indicates that no neurons were added. The loss value of the student model at that time. Indicates the first A group of neurons were added. Then, the partial derivative of the student model loss with respect to the neuron is taken. The remaining part after expanding the first-order function of the importance of each group of neurons; after obtaining the importance score of each group of neurons, sort them in descending order of importance score to form a neuron ranking sequence; determine the set of retained groups in the neuron ranking sequence according to a predefined pruning ratio. and the group to be pruned Then, structured deletion is performed, targeting items belonging to... The neuron group, deleted from its The corresponding column (and) (corresponding bias term in the middle), to obtain Delete simultaneously The corresponding row in the middle, get ,in , The predefined pruning ratio.

[0043] The Transformer decoder employs a depth pruning strategy. After width pruning, to further reduce the model's computational complexity and parameter size, this invention also prunes the Transformer decoder in the depth direction, i.e., by reducing the effective number of layers in the Transformer decoder to achieve model lightweighting. In this embodiment, depth pruning uses a method based on a preset pruning probability. The rule-based layer pruning strategy. Specifically, suppose the Transformer decoder originally contains There are 1 decoder layer, and each decoder layer is numbered sequentially according to the forward propagation order of the network as follows: (18) To determine which decoder layers participate in pruning, a layer index is introduced for each decoder layer. The decision on whether to prune this layer is based on the following congruence criteria: (19) in This indicates a congruence relation, and mod represents the modulo operation. This represents rounding down. When the above congruence condition is true, for index ... The decoder layer is pruned, meaning it is removed from the Transformer decoder structure. This method allows for approximately uniform pruning of decoder layers along the depth direction, avoiding information transmission interruptions caused by concentrated deletion of adjacent layers. Simultaneously, during depth pruning, the last layer of the Transformer decoder is excluded from pruning to ensure the stability of the object detection output and provide effective alignment for subsequent teacher model supervision during training.

[0044] (2) Training methods Construction of teacher and student models. During the training phase, teacher and student models are constructed, including: The teacher model uses ResNet-50 as the feature extractor and a Transformer decoder without width and depth pruning. The student model uses ResNet-34 as the feature extractor and a Transformer decoder that has been pruned with both width and depth.

[0045] The teacher model is used to provide high-quality feature representations and intermediate layer information, while the student model learns the knowledge from the teacher model through distillation training.

[0046] Feature Consistency Distillation Loss. To constrain the student model to approximate the teacher model in the feature space, this invention uses mean squared error (MSE) loss to constrain the consistency of the backbone network output features of the teacher and student models. The calculation method is as follows: (20) in, For characteristic consistency distillation loss, and The features extracted from the student model and teacher model are respectively used to extract their output features. Furthermore, intermediate layer distillation is performed to pass hidden state knowledge, i.e.: (twenty one) in, To conceal the distillation loss, and The Transformer decoders representing the student model and teacher model respectively are the first two. The decoder layer outputs features. The final loss function can be expressed as: (twenty two) in, and For predefined hyperparameters, Loss for 3D object detection.

[0047] Adjust the parameters of the teacher model and the student model, calculate the value of the loss function, and stop iterating when the value of the loss function is minimized. This will give you a trained teacher model and a trained student model. Use the trained student model to perform lightweight star catalog target detection.

[0048] As shown in Table 1 below, compared with existing methods, the lightweight star catalog target detection method proposed in this invention can achieve high accuracy while ensuring high efficiency.

[0049] (1) This invention systematically and lightweightly designs the star catalog target detection network, and jointly optimizes the backbone network structure, neuron width, multi-head attention width, and Transformer decoder depth while maintaining the consistency of the overall detection framework. Specifically, by replacing the backbone network with ResNet50 and ResNet34, and combining width pruning and depth pruning strategies, the model parameter size is reduced from 141M to 97M, which significantly reduces the model's storage overhead and computational complexity, making it easier to deploy and run the model in application environments with limited computing resources.

[0050] (2) While achieving structural lightweighting, this invention introduces a training supervision mechanism for the student model using a teacher model. Through feature consistency distillation and hidden state distillation, it effectively alleviates the performance degradation problem caused by model compression. Experimental results show that even with a significant reduction in network size, the student model can still maintain an object detection accuracy of 81.2%, while the teacher model only shows a limited decrease in accuracy. This achieves a good balance between model lightweighting and detection performance, enhancing the engineering practical value of the lightweight star catalog object detection method.

[0051] Table 1 Model Performance

[0052] The lightweight celestial surface target detection method of this invention has wide applications in fields such as satellite remote sensing and deep space exploration. Through an efficient target detection model, it enables real-time identification and localization of celestial or ground targets in resource-constrained environments, effectively applicable to missions such as space exploration, improving system processing speed and accuracy while reducing computational overhead, and supporting efficient edge computing and intelligent decision-making.

[0053] Example 2 Embodiment 2 of the present invention also provides a system for performing the lightweight star catalog target detection method described in Embodiment 1, the system comprising: The student model building module is used to build student models, which use a lightweight feature extractor connected to a Transformer decoder that has undergone width and depth pruning. The teacher model building module is used to build a teacher model, which uses a feature extractor without lightweight processing and connects it to a Transformer decoder without width pruning and depth pruning. The model training module is used to adjust the parameters of the teacher model and the student model, calculate the value of the loss function, and stop iterating when the value of the loss function is minimized, thus obtaining the trained teacher model and student model. The trained student model is then used to output the object category and 3D bounding box through the detection head.

[0054] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight method for detecting targets in a star catalog, characterized in that, include: S1. Construct a student model. The student model uses a lightweight feature extractor connected to a Transformer decoder that has undergone width and depth pruning. S2. Construct a teacher model, which uses a feature extractor without lightweight processing and connects it to a Transformer decoder without width and depth pruning. S3. Adjust the parameters of the teacher model and student model, calculate the value of the loss function, and stop iterating when the value of the loss function is minimized. This will give you the trained teacher model and student model. Use the trained student model to output the object category and 3D bounding box through the detection head.

2. The lightweight star catalog target detection method according to claim 1, characterized in that, The execution process of the teacher model and student model is as follows: The feature extractor extracts 2D features from the stereo image pairs to obtain 2D multi-view features. In the 3D coordinate generator, the coordinates of the stereo image pairs are transformed using camera parameters to generate 3D coordinates. The generated 3D coordinates, together with the 2D multi-view features, are input into the implicit 3D feature encoder to obtain an implicit 3D representation. The implicit 3D representation is flattened and input into the Transformer decoder, where it interacts with the object query generated by the query generator to obtain an updated object query. The updated object query of the last layer of the Transformer decoder is input into the detection head to predict the object category and 3D bounding box.

3. The lightweight star catalog target detection method according to claim 2, characterized in that, The lightweight feature extractor includes: Provide stereo images of the Martian surface. , Indicates the left eye image. Representing the right eye image, ResNet-34 is used as a lightweight feature extractor to extract 2D features from the binocular image pair, resulting in 2D multi-view features.

4. The lightweight star catalog target detection method according to claim 1, characterized in that, The un-lightweighted feature extractor is ResNet-50.

5. The lightweight star catalog target detection method according to claim 1, characterized in that, The width pruning includes: The Transformer decoder consists of L sequentially connected decoder layers. Each layer includes a multi-head attention module and a feedforward network module, with the multi-head attention module connected to the feedforward network module. Each multi-head attention module includes... The first attention point, for the first The importance score of the h-th attention head in layer h. The estimate using a first-order Taylor expansion is: in, Indicates the first Add attention head to the layer The loss value of the student model afterwards, This indicates that no attention head has been added. The loss value of the student model at that time. Indicates the first Add attention head to the layer Then, the partial derivative of the student model loss with respect to the attention head is calculated. The remaining part after expanding the first-order function of each attention head; after obtaining the importance score of each attention head, sort them in descending order of importance score to form an attention head sorting sequence; and then prune them according to a preset ratio. Once the number of attention heads retained in the attention head sorting sequence is determined, the retained set is: The remaining set is the set of attention points to be pruned: in, The first The first layer of attention head sorting sequence One point of attention, , Indicates the floor function; Focus on pruning The linear mapping parameters of the attention heads are removed from the multi-head attention module. For the output projection matrix of the multi-head attention module, the row blocks corresponding to the pruned attention heads are deleted.

6. The lightweight star catalog target detection method according to claim 5, characterized in that, The width pruning also includes: For the feedforward network module, it includes two layers of linear transformation: Wherein, FFN represents the feedforward network module. This represents the input data for FFN. This represents the activation function. and They are two linear mappings. and There are two biases; the intermediate layer neurons of the FFN are divided into There are groups, each containing s neurons, satisfying... , Representing the intermediate layer dimension, the importance score of each group of neurons is calculated as follows: in, Indicates the first A group of neurons were added. The loss value of the student model afterwards, This indicates that no neurons were added. The loss value of the student model at that time. Indicates the first A group of neurons were added. Then, the partial derivative of the student model loss with respect to the neuron is taken. The remaining part after expanding the first-order function of the importance of each group of neurons; after obtaining the importance score of each group of neurons, sort them in descending order of importance score to form a neuron ranking sequence; determine the set of retained groups in the neuron ranking sequence according to a predefined pruning ratio. and the group to be pruned , for those belonging The neuron group, deleted from its The corresponding column, and The corresponding bias items are deleted synchronously. The corresponding row in the middle.

7. The lightweight star catalog target detection method according to claim 1, characterized in that, The deep pruning includes: Suppose the Transformer decoder contains sequentially connected... There are 1 decoder layer, and each decoder layer is numbered sequentially according to the network forward propagation order. For indexes The decoder layer determines whether to perform pruning based on the following congruence judgment condition: in, This indicates a congruence relation, and mod represents the modulo operation. This represents rounding down. Preset pruning probability; When the congruence condition is true, for index 1 The decoder layer is removed from the Transformer decoder structure.

8. The lightweight star catalog target detection method according to claim 1, characterized in that, The loss function is: in, and For predefined hyperparameters, For the loss of 3D object detection, For characteristic consistency distillation loss, This represents the loss during distillation in a hidden state.

9. A lightweight star catalog target detection method according to claim 8, characterized in that, The characteristic consistency distillation loss The calculation method is as follows: in, and These represent the features output by the feature extractors for the student model and the teacher model, respectively. Indicates MSE loss; The hidden state distillation loss The calculation method is as follows in, and The Transformer decoders representing the student model and teacher model respectively are the first two. Layer decoder layer output features.

10. A system for implementing a lightweight star catalog target detection method according to any one of claims 1-9, characterized in that, The system includes: The student model building module is used to build student models, which use a lightweight feature extractor connected to a Transformer decoder that has undergone width and depth pruning. The teacher model building module is used to build a teacher model, which uses a feature extractor without lightweight processing and connects it to a Transformer decoder without width pruning and depth pruning. The model training module is used to adjust the parameters of the teacher model and the student model, calculate the value of the loss function, and stop iterating when the value of the loss function is minimized, thus obtaining the trained teacher model and student model. The trained student model is then used to output the object category and 3D bounding box through the detection head.

Images