Dense target detection optimization method based on mask generation type distillation and cross-task consistency

Abstract

The invention discloses a dense target detection optimization method based on mask generation type distillation and cross-task consistency, and the method comprises the steps: obtaining a student model mask graph based on a task perception mask generation algorithm, training a student model through the student model mask graph and a teacher model feature graph, and obtaining a student model with the enhanced feature representation capability. Training student models based on the regression task loss, the classification task loss and the feature map alignment loss to obtain a calibrated student model; the calibrated student model is used to detect the dense target to obtain a target detection result. The problem that the student model is difficult to fully learn task-related knowledge of the teacher model due to the fact that a traditional knowledge distillation method cannot effectively process feature demand differences of classification and regression tasks in dense target detection is solved. And an existing method is easily interfered by background noise and target overlapping in a dense scene, so that the effect of characteristic distillation is influenced.

Description

Technical Field

[0001] The present invention relates to the field of knowledge distillation technology, and more specifically to a dense object detection optimization method based on mask generative distillation and cross-task consistency. Background Art Knowledge distillation (KD) is a widely used model compression and performance improvement technique in deep learning. Its core concept is to transfer knowledge from a large, high-performance teacher model to a smaller, more computationally efficient student model, thereby preserving or even improving the student model's performance while maintaining inference speed. This technique, first proposed by Hinton et al. in 2015, has since been widely applied in computer vision tasks such as image classification, object detection, and semantic segmentation. With the increasing demand for deploying deep learning models in real-world scenarios, such as in resource-constrained environments like mobile devices and embedded systems, knowledge distillation has attracted considerable attention due to its ability to significantly improve model inference efficiency. However, when applying knowledge distillation to the complex task of dense object detection, traditional methods face numerous challenges, which has driven continued research in this area.

[0002] Dense object detection is a key branch of object detection. Unlike traditional sparse object detection, its main characteristic is that it must process a large number of densely packed objects. These objects are often accompanied by scale variations, mutual occlusion, and background noise, forcing the model to strike a balance between global semantic information and local geometric details. Dense object detection is crucial in many practical applications, such as pedestrian and vehicle detection in autonomous driving, crowd analysis in video surveillance, and object tracking in drone vision. However, traditional knowledge distillation methods often perform poorly in dense object detection. This is because dense object detection places higher demands on model robustness and feature representation capabilities. Traditional methods often ignore the differences in feature requirements between classification and regression tasks during the knowledge transfer process, resulting in the student model struggling to fully learn the task-related knowledge of the teacher model, thus limiting detection performance improvements.

[0003] Specifically, the object detection task can usually be decomposed into two subtasks: classification (determining the object category) and regression (predicting the object's bounding box coordinates). In dense object detection, due to the high-density distribution of objects and complex background interference, the classification task relies more on the semantic information of the object's center, while the regression task is more sensitive to the geometric details of the boundary area. Traditional knowledge distillation methods often achieve knowledge transfer by directly aligning the feature maps of the teacher model and the student model or outputting logits (category probabilities and bounding box coordinates). However, this approach does not fully consider the differences in feature requirements between the aforementioned tasks, making it difficult for the student model to balance the optimization requirements of both during the learning process. In addition, background noise and target overlap in dense scenes further exacerbate the difficulty of knowledge transfer, causing the student model to learn redundant or even erroneous features, affecting the final detection effect. Summary of the Invention The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the first aspect of the present invention proposes a dense target detection optimization method based on mask generative distillation and cross-task consistency, comprising: based on a task-aware mask generation algorithm, assigning a first ratio of mask occlusion to the classification task sensitive area of the student model feature map, assigning a second ratio of mask occlusion to the regression task sensitive area of the student model feature map, obtaining a student model mask map, using the student model mask map and the teacher model feature map to train the student model to obtain a student model with enhanced feature representation ability, wherein the first ratio is lower than the second ratio; using a bounding box regression algorithm based on probability modeling to calculate the regression task loss of the student model with enhanced feature representation ability; using a task-aware adaptive temperature scaling algorithm to calculate the classification task loss of the student model with enhanced feature representation ability; based on a spatial adaptive feature alignment algorithm to calculate the feature map alignment loss of the student model with enhanced feature representation ability; training the student model based on the regression task loss, the classification task loss and the feature map alignment loss to obtain a calibrated student model; using the calibrated student model to detect dense targets to obtain target detection results.

[0004] Optionally, before assigning a mask of a first proportion to the sensitive area of the classification task, the method further includes: setting a true input bounding box annotation for the student model feature map, and determining the central target of the target to be detected based on the true input bounding box annotation; determining a heat map generation function based on the central target of the target to be detected and a Gaussian kernel decay rate that adaptively changes with the size of the target to be detected; and using the heat map generation function to process each pixel of the target to be detected to determine a classification heat map.

[0005] Optionally, before assigning a second-scale mask to the regression task sensitive area, the method further includes: expanding the true input bounding box annotated area of the target to be detected outward by a preset pixel value to obtain an extended area, and obtaining a dilated area based on an area formed by the extended area and the true input bounding box annotated area; constructing a binary mask, setting pixel values in the dilated area to 1 and pixel values in the non-dilated area to 0, to obtain a regression sensitive mask.

[0006] Optionally, the method assigns a mask occlusion of a first proportion to the sensitive area of the classification task and assigns a mask occlusion of a second proportion to the sensitive area of the regression task to obtain a student model mask map, including: assigning a mask occlusion of a first proportion to the classification heat map to obtain a mask map with global semantic features; assigning a mask occlusion of a second proportion to the mask map with global semantic features to obtain a student model mask map with edge detail information.

[0007] Optionally, the bounding box regression algorithm using probability modeling calculates the regression task loss of the student model with enhanced feature representation ability, including: obtaining the four-dimensional Gaussian distribution function obeyed by the prediction boxes output by the teacher model and the student model; determining the expected coordinates and the expected coordinate covariance matrix of the prediction box output by the teacher model based on the four-dimensional Gaussian distribution function; and the expected coordinates and the expected coordinate covariance matrix of the prediction box output by the student model; calculating the overall 2-Wasserstein distance of the prediction box based on the expected coordinates of the prediction box output by the student model and the expected coordinates of the prediction box output by the teacher model, and based on the expected coordinate covariance matrix of the prediction box diagram output by the student model and the expected coordinate covariance matrix of the prediction box output by the teacher model; calculating the averaged overall 2-Wasserstein distance to obtain the regression task loss of the student model with enhanced feature representation ability.

[0008] Optionally, before obtaining the regression task loss function of the student model based on the averaged overall 2-Wasserstein distance, the method further includes: performing Cholesky decomposition on both the expected coordinate covariance matrix of the prediction box output by the teacher model and the expected coordinate covariance matrix of the prediction box output by the student model.

[0009] Optionally, the task-aware adaptive temperature scaling algorithm calculates the classification task loss of the student model with enhanced feature representation capability, including: obtaining the classification outputs of the teacher model and the student model respectively; scaling the classification outputs of the teacher model and the student model respectively based on the temperature parameter to obtain the scaled teacher model classification output and the scaled student model classification output; processing the teacher model classification output using the softmax function to obtain the output probability of the teacher model; processing the student model classification output using the softmax function to obtain the output probability of the student model; calculating the KL divergence between the output probability of the teacher model and the output probability of the student model to obtain the classification task loss of the student model with enhanced feature representation capability.

[0010] Optionally, the feature map alignment loss of the student model with enhanced feature representation capability is calculated based on the spatial adaptive feature alignment algorithm, including: mapping the teacher model map feature map and the student model feature map to the same number of common channels; splicing the teacher model map feature map and the student model feature map with the same number of common channels to obtain a joint feature map; processing the joint feature map based on an offset prediction network to obtain a predicted offset of the student model feature map relative to the teacher model feature map; using deformable convolution to adjust the sampling position of the student feature map based on the predicted offset to obtain the student model feature map after feature alignment; processing the alignment loss of the teacher model feature map and the student model feature map after feature alignment based on a preset alignment loss function to obtain the feature map alignment loss of the student model with enhanced feature capability.

[0011] Optionally, the structure of the offset prediction network includes: a first convolutional layer, a second convolutional layer, and a third convolutional layer connected in sequence; wherein the input channel of the first convolutional layer is 2, the output channel is 128, the stride is 1, the padding=1, and the convolution kernel is , the activation function is ReLU; the input channel of the second convolutional layer is 128, the output channel is 64, the stride is 1, padding=1, and the convolution kernel is The activation function is ReLU; the third convolution layer has 64 input channels, 2 output channels, and a convolution kernel of .

[0012] Optionally, before assigning a first-ratio mask occlusion to the classification task sensitive area of the student model feature map based on the task-aware mask generation algorithm, the method also includes: using the teacher model and the student model to extract features of the input image to be detected, respectively, to obtain the corresponding teacher model feature map and the student model feature map, and identifying the classification task sensitive area and the regression task sensitive area in the student model feature map.

[0013] The embodiment of the present invention provides a dense target detection optimization method based on mask generative distillation and cross-task consistency. Compared with the existing technology, its beneficial effects are as follows: based on the task-aware mask generation algorithm, the classification task sensitive area of the student model feature map is given a mask occlusion of a first ratio, and the regression task sensitive area of the student model feature map is given a mask occlusion of a second ratio to obtain a student model mask map, and the student model mask map and the teacher model feature map are used to train the student model to obtain a student model with enhanced feature representation ability, wherein the first ratio is lower than the second ratio, and the mask-based feature distillation task is realized. The dynamic mask mechanism is used to suppress background noise interference in the task sensitive area, thereby improving the robustness of feature expression and solving the noise problem in dense scenes; the bounding box regression algorithm of probability modeling is used to calculate the regression task loss of the student model with enhanced feature representation ability; based on task perception The adaptive temperature scaling algorithm is used to calculate the classification task loss of the student model with enhanced feature representation ability; the feature map alignment loss of the student model with enhanced feature representation ability is calculated based on the spatial adaptive feature alignment algorithm; the student model is trained separately based on the regression task loss, classification task loss and feature map alignment loss to obtain a calibrated student model, which realizes the feature requirements of cross-task distillation for the student model, performs differentiated optimization, and improves the student model's ability to learn task-related features; the calibrated student model is used to detect dense targets to obtain target detection results, which solves the problem that traditional knowledge distillation methods cannot effectively handle the feature requirement differences of classification and regression tasks in dense target detection, which makes it difficult for the student model to fully learn the task-related knowledge of the teacher model, and the existing methods are easily affected by background noise and target overlap in dense scenes, which affects the effect of feature distillation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] To more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0015] Figure 1 A flowchart of a dense object detection optimization method based on mask-generating distillation and cross-task consistency provided by an embodiment of the present invention; Figure 2 The overall flow chart of knowledge distillation for the dense object detection optimization method based on mask-generating distillation and cross-task consistency provided by the embodiment of the present invention; Figure 3 Structural diagram of the spatially adaptive feature alignment network for the dense object detection optimization method based on mask-generative distillation and cross-task consistency provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0017] This specification provides method operation steps as described in the embodiments or flowcharts, but may include more or fewer operation steps based on routine or non-inventive work. When implemented in a real system or server product, the methods shown in the embodiments or figures may be executed sequentially or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0018] Currently, knowledge distillation methods for object detection tasks can be categorized into the following three main groups, which, to a certain extent, represent the technical solutions most similar to the present invention. The following details the implementation principles of each method and its limitations in dense object detection.

[0019] 1. Feature-based Distillation Method Feature-based distillation methods achieve knowledge transfer by aligning the intermediate layer feature maps of the teacher model and the student model. This method assumes that the feature maps of the teacher model contain rich task-related information, and the student model can learn similar expressive capabilities by imitating these feature maps. For example, in some existing technologies, the student model is directly optimized to approximate the feature representation of the teacher model by calculating the L2 loss between the feature maps of the teacher model and the student model. However, in dense target detection scenarios, this method is easily interfered by background noise and target overlap. Since the background area in dense scenes often occupies a large proportion of the feature map, directly aligning the feature maps may cause the student model to learn a large number of invalid or even erroneous background features, thereby reducing detection performance.

[0020] To improve this problem, some studies have further proposed hierarchical alignment strategies. For example, the Hierarchical Feature Matching (HFM) method improves the accuracy of knowledge transfer by hierarchically aligning multi-scale feature maps. This method can adapt to changes in object scale to a certain extent, but its performance in dense object detection is still limited. This is because the background area in multi-scale feature maps still accounts for a large proportion. Direct alignment may introduce more noise, limiting further improvement of the student model's performance.

[0021] 2. Distillation method based on logits Logits-based distillation methods optimize the student model by imposing constraints on the output layers of the teacher and student models (typically classification probabilities and bounding box coordinates). For example, some existing techniques apply a KL divergence loss to the category probabilities of the classification task and an L2 loss to the bounding box coordinates of the regression task, respectively, to achieve direct knowledge transfer. This method can transfer the category and location information of the teacher model to a certain extent, but it has significant drawbacks in dense object detection. Because classification and regression tasks have different feature requirements (classification relies on semantic information, while regression focuses on geometric details), directly aligning logits does not account for this heterogeneity, resulting in inefficient knowledge transfer. Furthermore, the unbounded output of the regression task often dominates training, limiting improvements in classification performance and making it difficult to achieve coordinated optimization of the two.

[0022] 3. Task-aware methods In order to solve the problem of differences between classification and regression tasks in target detection, some studies have begun to explore task-specific knowledge distillation methods. For example, Zhang et al. proposed a task-aware feature distillation method in 2020, which achieves task decoupling by assigning different feature channels to classification and regression tasks, thereby improving the effectiveness of knowledge transfer. However, in dense target detection, due to the complexity of target overlap and background noise, the actual effect of this method is still limited. For example, in some existing technologies, a masking mechanism is introduced to enhance the student model's ability to generate teacher features. This method uses a uniform random masking strategy to block some feature areas, forcing the student model to learn more meaningful features. However, this random masking strategy ignores the different dependencies of classification and regression tasks on feature areas, which may lead to the loss of information in key areas, thereby reducing the distillation effect.

[0023] In summary, the shortcomings of existing technologies can be summarized as follows: 1. They ignore the differences in feature requirements between classification and regression tasks (e.g., classification focuses on the semantics of object center, while regression focuses on boundary details). This makes it difficult for the student model to fully learn task-related knowledge, limiting performance improvement. 2. In dense scenes, background noise and object overlap interfere with the feature distillation process, reducing the efficiency of knowledge transfer and insufficient robustness. 3. During cross-task knowledge transfer, the optimization objectives of classification and regression tasks conflict (e.g., distribution scale mismatch and gradient imbalance), affecting the overall performance of the student model and making collaborative optimization difficult.

[0024] To address the aforementioned technical deficiencies, this application provides a dense object detection optimization method based on masked generative distillation and cross-task consistency. This method improves student model performance through task-aware masked feature distillation (TAMD) and cross-task distillation (BCD). This method also uses task-specific mask matrices to perform differentiated optimization based on the feature requirements of classification and regression tasks, enhancing feature learning capabilities and suppressing background noise. This method further balances the training contributions of classification and regression tasks through probabilistic regression distillation, classification distillation and response calibration, and spatially adaptive feature alignment, alleviating performance bottlenecks caused by task conflicts.

[0025] The technical solution of this application is described in detail below with reference to the accompanying drawings.

[0026] Figure 1 This is an embodiment of the present application that provides a dense target detection optimization method based on mask generation distillation and cross-task consistency. It should be noted that this method can be executed by any terminal or by a server, and is not limited here. Figure 1 As shown, the dense object detection optimization method based on mask generative distillation and cross-task consistency can include: S10. Based on the task-aware mask generation algorithm, a first ratio of mask occlusion is assigned to the classification task sensitive area of the student model feature map, and a second ratio of mask occlusion is assigned to the regression task sensitive area of the student model feature map to obtain a student model mask map. The student model is trained using the student model mask map and the teacher model feature map to obtain a student model with enhanced feature representation capability, wherein the first ratio is lower than the second ratio.

[0027] It should be noted that before executing step S10, the dense object detection optimization method based on mask generative distillation and cross-task consistency may also include the following execution process: The teacher model and student model are used to extract features of the input image to be detected, respectively, to obtain the corresponding teacher model feature map and student model feature map, and to identify the classification task sensitive areas and regression task sensitive areas in the student model feature map.

[0028] The feature extraction process involves inputting the same image to be detected into both a teacher model (TeacherModel) and a student model (StudentModel). Feature maps are extracted from the intermediate layers of the teacher model (such as the output layer of the backbone network or the first layer of a specific task head). The teacher model typically has stronger feature representation capabilities, and its feature maps contain rich semantic and detailed information. Feature maps are extracted from the corresponding layers of the student model. Due to the student model's lighter structure, its feature maps are typically coarse and require alignment with the teacher's features through distillation learning. After feature extraction from the input image to be detected, it is necessary to identify sensitive regions in the student model's feature maps for classification and regression. Since classification focuses on global semantic information (such as object category), sensitive regions are typically located at the center of the detected object, while regression focuses on local geometric details and are located near the object's boundaries. When reconstructing the teacher's features, the student model must not only strengthen learning of the object's center but also guide the student to focus on edge details.

[0029] In particular, in dense object detection, traditional knowledge distillation methods often assume that the student model can directly replicate the complete features of the teacher model. However, in complex scenarios (such as occlusion, small objects, or dense arrays), this assumption often fails to hold. This is because the detection model's overreliance on local features can lead to neglecting global information. Especially in the case of occlusion, the student model struggles to learn robust feature representations through imitation.

[0030] Existing generative distillation methods generally use uniform random masks, ignoring the differences in task characteristics, resulting in a mismatch between the masked area and the task target. In addition, there is a large amount of background noise in the feature map of the dense detection head, and directly aligning all areas may introduce invalid supervision signals. To this end, this application proposes a task-aware mask generative distillation method. Its core idea is to force the student model to learn the generation ability of teacher features in key areas, such as classification and regression sensitive areas, through a differentiated mask strategy, while suppressing background noise interference. The first goal of this application is to enable the generative distillation process to be optimized for the different characteristics of classification and regression tasks through differentiated mask design. Therefore, this application is optimized for classification tasks and regression tasks, that is, a low-proportion mask is applied to the center area of the object to retain semantic information, and a high-proportion mask is applied to the background area to suppress noise.

[0031] Before executing step S10, the dense object detection optimization method based on mask generative distillation and cross-task consistency may include the following execution process: S01. Set a true input bounding box annotation for the student model feature map, and determine the center target of the target to be detected based on the true input bounding box annotation; S02. Determine a heat map generation function based on the central target of the target to be detected and a Gaussian kernel decay rate that changes adaptively with the size of the target to be detected; S03. Use the heat map generation function to process each pixel of the target to be detected and determine the classification heat map By executing steps S01 to S03, the processor can selectively transfer the global semantic features of the teacher model to the student model, thereby avoiding interference from low-quality background features.

[0032] In the actual execution process, the processor responds to the preset instructions first and sets the real input bounding box to be marked as ,in, is the index of the target to be detected. Then, the processor calculates the target based on the center target of the target to be detected. To generate a Gaussian heat map, that is, for each pixel A response value is assigned that reflects its "importance" to the classification task. To accommodate different object sizes and multi-scale features, the heatmap is normalized to ensure that the responses of each region have a uniform interpretability in the overall feature map. This process is equivalent to introducing a soft attention mechanism, which enables the global semantic features of the teacher model to be selectively transferred to the student model, thereby avoiding interference from low-quality background features. The above process can be expressed as the following specific formula:

[0033]

[0034]

[0035] Where, Represents a heat map, The value represents the importance of each position to the classification task, and σ is the decay rate of the Gaussian kernel, which is adaptively determined based on the size of the object to be detected (default σ = 5). During the distillation process, the student model reconstructs the teacher features by focusing on high-response regions (i.e., the center of the object), thereby better capturing the discriminative semantic information required for classification. This is similar to the guiding principle in "hint-based learning," which prioritizes the most discriminative information when faced with an information bottleneck, improving the student model's feature representation capabilities.

[0036] Before executing step S10, the dense object detection optimization method based on mask generative distillation and cross-task consistency may further include the following execution process: S04, expanding the true input bounding box annotated area of the target to be detected outward by a preset pixel value to obtain an extended area, and obtaining a dilated area based on an area formed by the extended area and the true input bounding box annotated area; S05. Construct a binary mask, set the pixel values in the expanded area to 1, and the pixel values in the non-expanded area to 0, to obtain a regression-sensitive mask.

[0037] The processor can achieve the goal of selectively transferring the global semantic features of the teacher model to the student model by executing steps S04-S05, thereby avoiding interference from low-quality background features.

[0038] In the actual implementation process, this application proposes a sensitive area modeling method based on edge information to force the student model to focus on the target edge when reconstructing information, thereby improving the regression accuracy. Specifically, the processor uses the real bounding box information to expand the target edge outward by a fixed pixel value (for example, =3), construct an expanded area, and obtain the expanded area based on the expanded area and the true input bounding box annotation area. This process is similar to the expansion operation commonly used in target detection, which aims to highlight the edge details. The processor constructs a binary mask based on the expanded area and the non-expanded area. , where "1" represents the expansion area and "0" represents the non-expansion area. This binary processing can explicitly encode the edge information. During the knowledge distillation process, the teacher model guides the student model to finely restore the geometric structure of the target by applying stronger supervision signals to these areas. During the feature reconstruction process, the student model focuses on training high-weight edge areas, which not only improves the accuracy of boundary regression, but also effectively models the local structure of the target under low-information conditions. The generated expansion area can be expressed as ,in, Bk represents the ground truth input bounding box annotation area, and the dilation area is used to ensure that the mask covers key geometric features. The above process can be expressed as follows:

[0039] in, Regression Sensitive Mask In an embodiment of the present application, step S10 may include the following execution process: S101, assigning a mask of a first ratio to the classification heat map to obtain a mask map with global semantic features; S102: assigning a mask occlusion of a second scale to the mask map with global semantic features to obtain a student model mask map with edge detail information.

[0040] Specifically, based on the classification heat map and regression sensitive mask obtained in steps S01-S03 and S04-S05, this application further designs a dynamic mask generation algorithm, which aims to combine task-related prior encoding and prior mask with random sampling to achieve "selective perturbation" in the teacher-student information transmission process. In other words, the processor can use the classification heat map or regression sensitive mask , calculate the corresponding mask generation probability for each pixel position For classification tasks, this application assigns a lower mask ratio (default 20%) to the center of the target (high response area) and a higher ratio to the edge area to avoid the destruction of global semantic information. For regression tasks, the opposite is true, and the edge area obtains a higher mask ratio (default 40%) to force the student model to focus on edge details during reconstruction. Finally, the processor generates the final binary mask matrix by performing Bernoulli sampling on each pixel position. This randomized mask generation strategy can be viewed as a "perturbation injection" mechanism that simulates information loss or noise conditions, prompting the student model to learn more robust feature representations. This is similar to the strategy of approximating the teacher distribution reconstruction in generative distillation. The above process can be expressed in the following specific formula:

[0041]

[0042] in, represents the mask generation probability, Stask ( i , j ) represents a regression task or a classification task, M ( i , j ) represents a binary mask matrix. Indicates the first ratio and second ratio corresponding to the classification task or regression task.

[0043] It is worth noting that dense object detection models (such as YOLO and RetinaNet) usually adopt a multi-task learning framework, which generally contains the following two output branches: classification branch and regression branch. The output category probability distribution of the classification branch is ,in, is the number of prediction boxes, is the number of categories. This branch generally uses cross entropy loss, which aims to minimize the difference between the predicted distribution and the true label, thereby achieving semantic discrimination of the target. The output boundary coordinates of the regression branch are , this branch minimizes the gap between the predicted coordinates and the true coordinates through L1 or L2 loss, ensuring accurate positioning of the target position and size.

[0044] However, in the process of knowledge distillation, if distillation loss is directly applied to these two types of tasks (generally, KL divergence is used for classification and L2 loss is used for regression), the following problems will occur: (1) Distribution scale mismatch: The classification output belongs to the probability distribution and its value range is [0,1], while the coordinate numerical range of the regression output is usually unbounded. This huge difference in numerical scale will lead to serious imbalance in the gradient amplitude during joint training, which will make one task dominate the optimization.

[0045] (2) Task conflict: Classification tasks focus on the semantic information of the target (e.g., the center of the object), while regression tasks focus on geometric details (e.g., the edges of the bounding box). Their optimization objectives are fundamentally different. Directly combining the two losses may introduce noise signals, causing the optimization process to favor local optimal or suboptimal solutions.

[0046] The logit-based cross-task distillation algorithm proposed in this application and the solutions to the above problems can be broken down into four steps: Probabilistic regression distillation, classification distillation and response calibration, spatial adaptive alignment, and alignment loss function. The above four steps can be implemented as follows: S20. Calculate the regression task loss of the student model with enhanced feature representation capability using a bounding box regression algorithm based on probabilistic modeling. Calculate the classification task loss of the student model with enhanced feature representation capability using a task-aware adaptive temperature scaling algorithm. Calculate the feature map alignment loss of the student model with enhanced feature representation capability using a spatially adaptive feature alignment algorithm. Train the student model based on the regression task loss, classification task loss, and feature map alignment loss, respectively, to obtain a calibrated student model.

[0047] refer to Figure 2 In one embodiment of the present application, step S20 may include the following execution process: S201, obtaining the four-dimensional Gaussian distribution function obeyed by the prediction boxes output by the teacher model and the student model; S202, determining the expected coordinates and the expected coordinate covariance matrix of the prediction box output by the teacher model based on the four-dimensional Gaussian distribution function; As well as the expected coordinates and expected coordinate covariance matrix of the predicted box output by the student model; S203, based on the expected coordinates of the prediction box output by the student model and the expected coordinates of the prediction box output by the teacher model, and based on the expected coordinate covariance matrix of the prediction box output by the student model and the expected coordinate covariance matrix of the prediction box output by the teacher model, calculate the overall 2-Wasserstein distance of the prediction box; Specifically, step S203 may include the following execution process: The expected coordinate covariance matrix of the prediction box output by the teacher model and the expected coordinate covariance matrix of the prediction box output by the student model are both subjected to Cholesky decomposition.

[0048] S204. Calculate the averaged overall 2-Wasserstein distance to obtain the regression task loss of the student model with enhanced learning ability.

[0049] In the actual implementation process, due to the occlusion, blur and scale change of the target object, the precise positioning of the bounding box is often accompanied by a certain degree of uncertainty. To solve this problem, this application proposes a bounding box regression method based on probability modeling, which aims to enable the model to more comprehensively describe the distribution characteristics of the target position by simultaneously predicting the mean and uncertainty information of the bounding box coordinates. Specifically, this application assumes that the bounding box coordinates of each predicted box obey a four-dimensional Gaussian distribution , where the mean vector represents the expected coordinates of the bounding box, which corresponds to the results predicted by the traditional regression method, and the covariance matrix is a 4×4 matrix that captures the uncertainty and correlation between coordinates. Unlike the traditional independence assumption (i.e., each coordinate is considered to be independently distributed), this application allows the covariance matrix is a non-diagonal matrix, which means it can reflect potential dependencies between coordinates, such as the geometric relationship between width w and height h. This modeling approach significantly enhances the model's expressiveness in complex scenes. For example, when the target object is partially occluded, the uncertainty of width and height may show some correlation, which the traditional independence assumption cannot capture. By introducing this richer probability distribution, the method of this application can better cope with the challenges brought by occlusion, blur or noise in target detection tasks.

[0050] In the specific execution process, the teacher model and the student model output the corresponding distribution parameters respectively, that is, the teacher model outputs the mean and the covariance matrix , the student model output mean and the covariance matrix . The goal of the student model is not only to learn the coordinate prediction values of the teacher model, but also to master the teacher model's ability to express the confidence of predictions for different samples. In order to measure the difference between the two Gaussian distributions, this application chooses Wasserstein distance as the loss function. Wasserstein distance is a measure of probability distribution matching. Compared with the traditional KL divergence, it has better robustness in low-dimensional space and exhibits higher smoothness in gradient propagation, which helps to improve the stability of training. For multi-dimensional Gaussian distributions, the square of the 2-Wasserstein distance can be calculated by a closed-form solution:

[0051] For a four-dimensional independent Gaussian distribution, the overall 2-Wasserstein distance can be expanded to:

[0052] Assume a total prediction boxes, then the regression distillation loss can be defined as the average of the Wasserstein distances of all prediction boxes:

[0053] During the optimization process, this loss function forces the student model to align the mean and uncertainty parameters of the teacher model on each prediction box, achieving knowledge transfer at the probability distribution level.

[0054] In the specific implementation of the algorithm, the teacher model needs to first generate the mean of each prediction box and the covariance matrix A feasible approach is to directly predict the elements of the covariance matrix through additional network branches, but this may lead to an increase in the number of parameters and unstable training. To solve this problem, this application uses Cholesky decomposition to represent the covariance matrix as a lower triangular matrix The product of ,in The diagonal elements of are ensured to be positive by the exponential activation function. This representation not only ensures the positive definiteness of the covariance matrix, but also significantly reduces the number of parameters that need to be predicted and improves training efficiency. For the teacher model, the covariance matrix It can also be estimated through post-processing, such as using Monte Carlo sampling or multiple prediction results of the teacher model to approximate the uncertainty. The student model predicts and , and through the loss function during training The key advantage of this approach is that it effectively improves the student model's ability to handle uncertain samples during training. For example, in occluded or blurred scenes, the student model can provide higher uncertainty estimates based on the covariance matrix, thereby avoiding overfitting to noisy samples.

[0055] Continue to refer Figure 2 In one embodiment of the present application, step S20 may include the following execution process: S203, obtaining the classification outputs of the teacher model and the student model; S204, scaling the classification outputs of the teacher model and the student model based on the temperature parameter to obtain the scaled classification output of the teacher model and the scaled classification output of the student model; S205, using the softmax function to process the classification output of the teacher model to obtain the output probability of the teacher model; S206, using the softmax function to process the classification output of the student model to obtain the output probability of the student model; S207. Calculate the KL divergence between the output probability of the teacher model and the output probability of the student model to obtain the classification task loss of the student model with enhanced feature representation ability.

[0056] Specifically, in dense object detection, classification tasks are crucial for semantic understanding of the target. Knowledge distillation, as an effective method for model compression and performance improvement, typically guides the learning of the student model through the output of the teacher model. However, directly distilling the classification outputs (logits) of the teacher and student models is constrained by two factors: first, the impact of the temperature parameter on the distribution of logits, and second, the alignment problem between the teacher and student in the feature map space. To this end, this application proposes a task-aware adaptive temperature scaling algorithm that aims to more accurately transfer the semantic knowledge of the teacher model and improve the classification performance of the student model in dense scenarios.

[0057] In the classification task, the probability distribution output by the teacher model needs to be smooth enough to better reflect the similarity relationship between categories. To this end, this application introduces the temperature parameter Scale the teacher and student logits. Let the classification logits of the teacher model be , the classification logits of the student model are , after temperature scaling, the probability distributions calculated by the softmax function are:

[0058] Experiments show that the temperature parameter The choice of has a significant impact on the distillation effect. (like =4) will make the probability distribution smoother, enhance the transmission of subtle differences between categories, and help the student model learn the semantic knowledge of the teacher model. From the perspective of information theory, higher temperature reduces the peak difference of logits, making the distribution more uniform, thereby amplifying the contribution of secondary categories and reducing the risk of overfitting. In general, the gradient amplitude of temperature scaling is similar to Inversely proportional, so this application adopts =4 strikes a balance between smooth distribution and gradient stability.

[0059] During the distillation process, this application uses KL divergence as the classification loss function to measure the difference in probability distribution between the student and teacher models:

[0060] This loss makes the student model as close as possible to the teacher model in terms of the softened probability distribution, thereby more fully inheriting the teacher's semantic information.

[0061] In target detection, the teacher and student models often have differences in structure or feature extraction modules, which may cause a certain offset in the key areas captured by the two in space. This spatial misalignment will introduce noise in the position information when directly aligning the classification logits, affecting the distillation effect. To solve this problem, this application introduces a spatial response calibration algorithm so that the classification responses of the teacher and the student can be more matched in the spatial dimension, thereby improving the accuracy of the transmission of key area information. If the classification logits are aligned directly, spatial misalignment will introduce position noise and reduce the distillation effect. To this end, this application introduces a spatial response calibration algorithm to enhance the consistency of the responses of the teacher and the student in the spatial dimension.

[0062] This application first extracts spatial attention weights from the feature map. Assume that the feature maps extracted by the teacher model and the student model at a certain layer are , , where H, W, and C represent the height, width, and number of channels, respectively. The spatial attention weight is calculated as follows:

[0063]

[0064] in, Represents the normalized weight distribution over the entire feature map.

[0065] After obtaining the spatial attention weights, the processor applies them to the calibration of the classification logits. Assume that the feature map corresponding to the classification logits in the spatial dimension is , , it can be calibrated by weighted fusion:

[0066] This weighted operation enables the student model to rely more on its own predictions in key areas (high attention weights) and autonomously capture the details of the area; while in non-critical areas (low attention weights), it tends to inherit the information of the teacher model, thereby reducing the prediction error caused by spatial offset.

[0067] After completing the spatial response calibration, the calibration logits distributions of the student model and the teacher model are and , the processor uses KL divergence for matching:

[0068] Should The loss forces the student to learn from the teacher model in the calibrated spatial dimensions, ensuring response consistency in key informative regions. However, calibration may cause the distribution to deviate from the original semantic information, causing the student model to learn suboptimal solutions.

[0069] In an embodiment of the present application, step S20 may further include the following execution process: S207, mapping the teacher model feature map and the student model feature map to the same number of common channels; S208, concatenating the teacher model feature map and the student model feature map with the same number of common channels to obtain a joint feature map; S209, processing the joint feature map based on the offset prediction network to obtain a predicted offset of the student model feature map relative to the teacher model feature map; In one embodiment of the present application, the structure of the offset prediction network includes: The first convolutional layer, the second convolutional layer, and the third convolutional layer are connected in sequence; Among them, the input channel of the first convolutional layer is 2, the output channel is 128, the stride is 1, padding=1, and the convolution kernel is , the activation function is ReLU; The second convolutional layer has 128 input channels, 64 output channels, a stride of 1, padding = 1, and a convolution kernel of Volume, activation function is ReLU; The third convolution layer has 64 input channels, 2 output channels, and a convolution kernel of .

[0070] S210, using deformable convolution, adjusting the sampling position of the student feature map based on the predicted offset to obtain a student model feature map after feature alignment; S211. Processing the alignment loss of the teacher model feature map and the feature map of the student model after feature alignment based on a preset alignment loss function to obtain the feature map alignment loss of the student model with enhanced feature capability.

[0071] During the knowledge distillation process, the feature maps extracted by the teacher model and the student model often have alignment deviations in the spatial dimension. Although the task outputs of both are optimized through probabilistic regression distillation and classification distillation methods, the misalignment of spatial features will still cause the distillation signal to be distorted at the spatial level. This distortion directly affects the effective inheritance of the teacher model's knowledge by the student model, especially in dense object detection tasks, where spatial accuracy is crucial to detection performance. For example, in the evaluation of the COCO dataset, a spatial offset of more than 2 pixels may cause the IoU of the target box to drop by about 10%, significantly reducing mAP. To address this problem, this application proposes a Spatial Adaptive Distillation (SAD) module that dynamically aligns the feature maps of the teacher and student models to enhance their consistency in the spatial dimension, thereby improving the knowledge transfer effect.

[0072] The core concept of the SAD module is to use deformable convolution to dynamically adjust the spatial position of the student model's feature map to align it with the teacher model's feature map. Traditional convolution operations use fixed-shape convolution kernels that cannot adapt to the spatial variations of the input feature map. Deformable convolution, on the other hand, learns a per-pixel offset, allowing the kernel to adaptively adjust its position based on the input content. This flexibility makes it particularly suitable for addressing spatial misalignment in dense object detection. The following is a detailed design and implementation of the SAD module.

[0073] The SAD module consists of three main components: feature concatenation layer, offset prediction network and feature deformation layer, such as Figure 3 shown.

[0074] Assume that the feature map extracted by the teacher model is , the feature map extracted by the student model is ,in and is the spatial resolution of the student model feature map, and are the number of channels of the teacher model and the student model respectively. Since the network structures of the teacher model and the student model may be different, the number of channels of the two models is usually inconsistent. In order to unify the channel dimensions, the processor first converts and Mapped to a common channel number ,get and The processor then concatenates the two in the channel dimension. The concatenation process can be performed by adding the feature maps to generate a joint feature map:

[0075] The concatenated feature map contains the spatial and semantic information of the teacher and student, providing the offset prediction network.

[0076] Offset prediction network output offset Represents the spatial displacement of each pixel of the student feature map relative to the teacher feature map. Among them, the total number of parameters of the prediction network is approximately (by For example), it accounts for a small proportion of the overall model parameters, ensuring computational efficiency.

[0077] According to the predicted offset , the processor can use deformable convolution to transform the student feature map Perform spatial adjustment. For each pixel position in the feature map , the new sampling position is:

[0078] Where, It is the student feature map after feature alignment, which can reduce the information loss caused by spatial offset.

[0079] This application dynamically adjusts the spatial position of the student feature map to align it with the teacher feature map as much as possible, thereby improving the transmission quality of the distilled signal. In order to quantify the alignment effect, this application introduces the alignment loss function , this loss measures the difference between the student features and the teacher features after deformation.

[0080]

[0081] This alignment loss function forces the deformed student feature map to be as close as possible to the teacher feature map at each spatial location, thereby ensuring the alignment of key information.

[0082] In order to avoid excessive deformation during the alignment process, a regularization term is usually required to constrain the magnitude of the offset. Therefore, this application uses the following regularized feature alignment loss to avoid excessive deformation during the alignment process. The expression of the regularized feature alignment loss function is:

[0083] in, is a hyperparameter that adjusts the effect of the regularization term ( =0.5 will give the best results).

[0084] S30. Detect dense targets using the calibrated student model to obtain target detection results.

[0085] This application can be verified through experiments, and the results of the experimental verification are as follows: Experimental verification shows that the student model performance is significantly improved on the COCO 2017 and FLIR datasets, especially in small and large target detection, which has practical application value.

[0086] Table 1 Comparison of experimental results of the method in this application with other methods

[0087] Table 2 Impact of masking strategies on COCO 2017

[0088] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0089] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are described briefly because they are generally similar to the method embodiments. For related portions, reference can be made to the description of the method embodiments.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the scope of protection of the present invention.

Claims

1. A dense object detection optimization method based on mask-generating distillation and cross-task consistency, characterized by: include: Based on a task-aware mask generation algorithm, a first-ratio mask is assigned to a classification task-sensitive region of a student model feature map, and a second-ratio mask is assigned to a regression task-sensitive region of the student model feature map to obtain a student model mask map. The student model mask map and the teacher model feature map are used to train the student model to obtain a student model with enhanced feature representation capability, wherein the first ratio is lower than the second ratio. The regression task loss of the student model with enhanced feature representation capability is calculated using a bounding box regression algorithm with probabilistic modeling. The classification task loss of the student model with enhanced feature representation capability is calculated using a task-aware adaptive temperature scaling algorithm. The feature map alignment loss of the student model with enhanced feature representation capability is calculated using a spatially adaptive feature alignment algorithm. The student model is trained based on the regression task loss, classification task loss, and feature map alignment loss to obtain a calibrated student model. The calibrated student model is used to detect dense targets and obtain target detection results.

2. The dense object detection optimization method based on mask-generating distillation and cross-task consistency according to claim 1, characterized in that: Before assigning a mask of a first scale to the classification task sensitive area, the method further includes: Set the true input bounding box annotation for the student model feature map, and determine the center target of the target to be detected based on the true input bounding box annotation; Determine a heat map generation function based on the central target of the target to be detected and a Gaussian kernel decay rate that changes adaptively with the size of the target to be detected; The heat map generation function is used to process each pixel of the target to be detected to determine the classification heat map.

3. The dense object detection optimization method based on mask-generating distillation and cross-task consistency according to claim 2 is characterized in that Before assigning a mask of a second scale to the regression task sensitive area, the method further includes: Expand the true input bounding box annotation area of the target to be detected outward by a preset pixel value to obtain an extended area, and obtain a dilated area based on the area formed by the extended area and the true input bounding box annotation area; A binary mask is constructed, and the pixel values belonging to the dilated area are set to 1, and the pixel values of the non-dilated area are set to 0 to obtain a regression-sensitive mask.

4. The dense object detection optimization method based on mask generative distillation and cross-task consistency according to claim 3 is characterized in that The classification task sensitive area is assigned a mask of a first ratio, and the regression task sensitive area is assigned a mask of a second ratio, to obtain a student model mask map, including: Assigning a first-scale mask to the classification heat map to obtain a mask map with global semantic features; A mask of a second scale is assigned to the mask map with global semantic features to obtain a student model mask map with edge detail information.

5. The dense object detection optimization method based on mask generative distillation and cross-task consistency according to claim 1, characterized in that The bounding box regression algorithm using probabilistic modeling calculates the regression task loss of the student model with enhanced feature representation capability, including: Get the four-dimensional Gaussian distribution function obeyed by the prediction boxes output by the teacher model and the student model; Determine the expected coordinates and expected coordinate covariance matrix of the prediction box output by the teacher model based on the four-dimensional Gaussian distribution function; As well as the expected coordinates and expected coordinate covariance matrix of the predicted box output by the student model; Based on the expected coordinates of the prediction box output by the student model and the expected coordinates of the prediction box output by the teacher model, as well as the expected coordinate covariance matrix of the prediction box map output by the student model and the expected coordinate covariance matrix of the prediction box output by the teacher model, the overall 2-Wasserstein distance of the prediction box is calculated; The averaged overall 2-Wasserstein distance is calculated to obtain the regression task loss of the student model with enhanced learning ability.

6. The dense object detection optimization method based on mask-generating distillation and cross-task consistency according to claim 1, characterized in that Before obtaining the regression task loss function of the student model based on the averaged overall 2-Wasserstein distance, the method further includes: The expected coordinate covariance matrix of the prediction box output by the teacher model and the expected coordinate covariance matrix of the prediction box output by the student model are both subjected to Cholesky decomposition.

7. The dense object detection optimization method based on mask-generating distillation and cross-task consistency according to claim 1, characterized in that The task-aware adaptive temperature scaling algorithm calculates the classification task loss of the student model with enhanced feature representation capability, including: Get the classification output of the teacher model and the student model respectively; Scaling the classification outputs of the teacher model and the student model based on the temperature parameter to obtain the scaled classification output of the teacher model and the scaled classification output of the student model; Use the softmax function to process the classification output of the teacher model to obtain the output probability of the teacher model; Use the softmax function to process the classification output of the student model to obtain the output probability of the student model; The KL divergence between the output probability of the teacher model and the output probability of the student model is calculated to obtain the classification task loss of the student model with enhanced feature representation ability.

8. The dense object detection optimization method based on mask generative distillation and cross-task consistency according to claim 1, characterized in that The feature map alignment loss of the student model with enhanced feature representation capability is calculated based on the spatial adaptive feature alignment algorithm, including: Map the teacher model feature map and the student model feature map to the same number of common channels; The teacher model feature map and the student model feature map with the same number of common channels are concatenated to obtain a joint feature map; The offset prediction network processes the joint feature map to obtain the predicted offset of the student model feature map relative to the teacher model feature map; Using deformable convolution, the sampling position of the student feature map is adjusted based on the predicted offset to obtain the student model feature map after feature alignment; The alignment loss of the teacher model feature map and the feature map of the student model after feature alignment is processed based on the preset alignment loss function to obtain the feature map alignment loss of the student model with enhanced feature capability.

9. The dense object detection optimization method based on mask-generating distillation and cross-task consistency according to claim 8, characterized in that The structure of the offset prediction network includes: The first convolutional layer, the second convolutional layer, and the third convolutional layer are connected in sequence; Among them, the input channel of the first convolutional layer is 2, the output channel is 128, the stride is 1, padding=1, and the convolution kernel is , the activation function is ReLU; The second convolutional layer has 128 input channels, 64 output channels, a stride of 1, padding = 1, and a convolution kernel of Volume, activation function is ReLU; The third convolution layer has 64 input channels, 2 output channels, and a convolution kernel of .

10. The dense object detection optimization method based on mask generative distillation and cross-task consistency according to claim 1, characterized in that Before assigning a first-scale mask to the classification task-sensitive area of the student model feature map based on the task-aware mask generation algorithm, the method further includes: The teacher model and student model are used to extract features of the input image to be detected, respectively, to obtain the corresponding teacher model feature map and student model feature map, and to identify the classification task sensitive areas and regression task sensitive areas in the student model feature map.

Images