Few-shot based target detection model training method and device

Abstract

The application relates to a few-sample-based target detection model training method and device, which comprises the following steps: in each round of training of the target detection model using a small sample set, the outer loop processing using a query sample graph is improved, typical correlation analysis and projection are performed on the features of the query sample graph and the support sample graph, the query features and the support features with the maximum typical correlation are obtained, then the projected query features are effectively enhanced based on the projected support features, the target query features containing more rich and detailed features are obtained, and target prediction and loss calculation are performed based on the target query features, so that the parameters of the current target detection model to be trained in the current round can be more accurately adjusted. Therefore, a target detection model with higher small target detection precision can be iteratively trained.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and apparatus for training a target detection model based on few samples. Background Technology

[0002] Few-sample small target detection has demonstrated significant advantages in practical applications such as autonomous driving, robotics, and intelligent surveillance, bringing great convenience to daily life. Especially in autonomous driving scenarios, the identification of small targets such as vehicles and pedestrians is crucial for system robustness; the ability to detect small targets under few-sample conditions directly determines the safety and reliability of the system. However, due to the limited number of samples, the model struggles to fully learn the detailed features of small targets, resulting in poor detection performance.

[0003] Therefore, how to improve the small target detection performance of the model by utilizing limited samples is an urgent problem to be solved. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for training a target detection model based on a small number of samples to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for training a target detection model based on a small number of samples, the method comprising: In each round of training of the object detection model using a small sample set, the first query feature is obtained by extracting features from the query sample image using the current object detection model to be trained in this round. Determine the query projection matrix and the support projection matrix; among them, the typical correlation is greatest after projecting the first query feature and the first support feature of the support sample map according to the corresponding query projection matrix and support projection matrix; The first query feature is projected into the second query feature based on the query projection matrix; the first support feature is projected into the second support feature based on the support projection matrix. The second query feature is enhanced based on the second supporting feature to obtain the target query feature; Target prediction is performed based on target query features to obtain target prediction results; The loss value is determined based on the target prediction results, and the parameters of the current target detection model are updated and adjusted based on the loss value.

[0006] Secondly, this application provides a training device for a target detection model based on a small number of samples, the device comprising: The feature extraction module is used to extract the first query features from the query sample image using the current target detection model to be trained in each round of training with a small sample set. The canonical relevance projection module is used to determine the query projection matrix and the support projection matrix; wherein, the canonical relevance of the first query feature and the first support feature of the support sample map after projection optimization according to the corresponding query projection matrix and support projection matrix is ​​maximized; the first query feature is projected into the second query feature based on the query projection matrix; and the first support feature is projected into the second support feature based on the support projection matrix. The feature enhancement module is used to enhance the second query feature based on the second supporting feature to obtain the target query feature; The prediction module is used to predict the target based on the target query features and obtain the target prediction result. The parameter update module is used to determine the loss value based on the target prediction result, and to update and adjust the parameters of the current target detection model based on the loss value.

[0007] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in the first aspect.

[0008] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0009] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0010] The aforementioned method, apparatus, computer device, computer-readable storage medium, and computer program product for training object detection models based on a small sample set perform canonical correlation analysis and projection on the features of both the query sample image and the support sample image in each round of training using a small sample set. This yields the query features and support features with the highest canonical correlation. Furthermore, based on the projected support features, the projected query features are effectively enhanced to obtain target query features containing richer details. Target prediction and loss calculation based on these target query features allow for more accurate adjustment of the parameters of the current object detection model to be trained in the current round. This enables iterative training of an object detection model with higher accuracy for small target detection. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a method for training a target detection model based on a small number of samples in one embodiment; Figure 2This is a schematic diagram of the system architecture of a target detection model training method based on few samples in one embodiment; Figure 3 This is a structural block diagram of a target detection model training device based on a small number of samples in one embodiment; Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. Unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by those skilled in the art. The technical terms used herein are for the purpose of describing specific examples and are not intended to limit the scope of protection of this application.

[0013] This application describes a method for training a target detection model based on a few samples using a computer device. The computer device is equipped with the target detection model to be trained. The target detection model trained using the method described in this application has higher detection accuracy for small targets compared to target detection models trained using traditional methods. In other words, this application can train a target detection model with higher detection accuracy for small targets under few-sample conditions (i.e., meta-learning training conditions based on a small sample set).

[0014] The main scenario in this case is as follows: Based on a meta-learning framework, a small sample set (i.e., a limited number of sample images, including a set of support sample images and a set of query sample images) is used to train an object detection model through multiple rounds of iterative training. Each round of meta-learning training includes two phases: an inner loop and an outer loop. The inner loop refers to quickly and temporarily updating the parameters of the object detection model to be trained using the set of support sample images (i.e., the support set). The model parameters updated in the inner loop are temporary and are not saved. The outer loop refers to testing the object detection model using the set of query sample images (i.e., the query set) after the inner loop, based on the object detection model with the temporary model parameters, calculating the loss, and backpropagating and iteratively updating the initial model parameters of the object detection model before this round of training (i.e., the initial model parameters before this round of training that were not temporarily updated).

[0015] One improvement of this application lies in supporting feature interaction enhancement between the support sample map and the query sample map. Specifically, during the outer loop phase, feature information relevant to the current query task—i.e., relevant support features—is extracted from the support sample map. The features extracted from the query sample map are called query features. Based on these relevant support features, the query features extracted from the query sample map are enhanced. Thus, even with only a small number of query sample maps, the query features are effectively supplemented based on the typical relevant support features of the support sample maps (it should be understood that there can be one or more support sample maps for supplementing query features), which to some extent achieves the supplementation of detailed features for small targets, resulting in target query features with richer details. Furthermore, target prediction can be performed more accurately based on the target query features, especially improving the detection accuracy for small targets, thereby enabling the training of a target detection model with higher accuracy for small targets. The following sections will describe in more detail the target detection model training method based on a small number of samples in the embodiments of this application.

[0016] like Figure 1 As shown, a method for training a target detection model based on a small number of samples is provided, and this method is applied to computer devices. The method specifically includes the following steps: S11, in each round of training of the object detection model using a small sample set, the first query feature is obtained by extracting features from the query sample image using the current object detection model to be trained in this round.

[0017] It should be understood that during the training phase, in each round of training, query sample maps and support sample maps are sampled from the training dataset in a contextual task sampling manner to execute steps S11 to S16, calculate the total loss value, update and adjust the parameters of the current object detection model to be trained in this round, and carry out the next round of iterative training until the preset iteration stopping condition is met (such as the model convergence condition or the preset number of iterations is met), and the trained object detection model is obtained.

[0018] The object detection model to be trained in each round is referred to as the current object detection model. For example, the current object detection model includes modules such as a feature extraction module, a canonical correlation projection module, a feature enhancement module, and a prediction head. In each round of training, training-related steps are performed through the modules in the current object detection model. For instance, in each round of training, the feature extraction module extracts features from the query sample image to obtain the first query features.

[0019] In some embodiments, the query sample map is convolved to extract deep semantic features, resulting in an initial query feature map. The initial query feature map is either the directly extracted deep semantic features or the query features after enhancing the deep semantic features (e.g., fusing keypoint features).

[0020] For example, the feature extraction module may include a meta-learning backbone network (CNN)21, a meta-learning backbone network (CNN)22, and a region proposal network (RPN). Figure 2 (RPN is shown in the image). Figure 2 As shown, the 21 pairs of query sample graphs are obtained through the meta-learning backbone network. D q Perform multi-scale convolution to obtain the initial query feature map. , Where C refers to the number of channels (i.e., depth) of the initial query feature map. and These are the height and width of the initial query feature map, respectively. The height / width of the initial query feature map is determined based on the downsampling factor (i.e., the downsampling step size) during feature extraction. For example, if the downsampling factor is 16, then... , Where H and W are the query sample graphs, respectively. D q Height and width.

[0021] Furthermore, based on RPN ( Figure 2 (Not shown in the image) Initial query feature map v poi Extract key point features and fuse them into the initial query feature map. v poi The first query feature is obtained from the data. v Specifically, based on RPN, the initial query feature map... v poi Multiple corresponding anchor boxes are generated, which can be denoted as A anchor boxes. For example, with 3 scales × 3 aspect ratios for each location, 9 anchor boxes can be generated for one location, and one initial query feature map is generated. v poi Generate A anchor boxes to cover different target sizes. The feature representation of each anchor box is as follows:

[0022] in, Let A represent the characteristics of the i-th anchor frame, and A be the total number of anchor frames. This represents the x-coordinate of the center point of the i-th anchor frame. This represents the ordinate of the center point of the i-th anchor frame. Let represent the width and height of the i-th anchor frame, respectively.

[0023] It should be understood that each anchor frame has a corresponding confidence score (i.e., a confidence score representing the foreground). Therefore, a first number of anchor frames can be selected as candidate regions based on the confidence scores of each anchor frame. For example, to select Top- The anchor boxes are used as candidate regions, where K < A, and K is the preset first number. Then, the key point features of each candidate region are extracted. For example, the features of any selected candidate region i... (i.e., feature map) Deformable convolution is applied to dynamically adjust the sampling position of the convolution kernel in order to extract key point features. The specific formula is as follows:

[0024] in, These are the key point features of the i-th candidate region. Here, represents the weight of the nth sampling point of the deformable convolution kernel, and N is the total number of sampling points of the deformable convolution kernel. This represents the x-coordinate offset of the nth sampling point. This represents the ordinate offset of the nth sampling point. Indicates the first Feature maps of candidate regions Above, obtain the coordinates. The eigenvalue at that location.

[0025] Furthermore, the key point features corresponding to each of the TopK candidate regions are compared with the initial query feature map. By merging, the first query feature is obtained. The specific formula is as follows:

[0026] in, This represents the key point features of the first candidate region. This represents the key point features of the Kth candidate region. For the connection operator.

[0027] In this embodiment, after extracting deep semantic features from the query sample image to generate a query feature map, the PRN is used to determine candidate regions from the feature map and focus on key points. The key point features are then fused into the original query feature map, which can reduce background interference and provide subsequent target localization capabilities.

[0028] S12, determine the query projection matrix and the support projection matrix; wherein, the typical correlation is maximized after projecting the first query feature and the first support feature of the support sample map according to the corresponding query projection matrix and support projection matrix.

[0029] Since utilizing relevant features from supporting sample graphs to enhance query features is one of the core inventive points of this case, therefore, as... Figure 2 As shown, in the outer loop processing, the meta-learning backbone network 22 can be used to support sample graphs. D r The first support feature is obtained by performing multi-scale convolution. , It should be understood that the meta-learning backbone network 22 shares parameters with the meta-learning backbone network 21.

[0030] Furthermore, the first query feature can be... v and the first supporting features of the supporting sample map Canonical correlation analysis was performed to determine the query projection matrix and support projection matrix that maximize the canonical correlation between the two projections.

[0031] In some embodiments, such as Figure 2 As shown, the current target detection model includes a canonical correlation projection module, and the query projection matrix is ​​determined based on the canonical correlation projection module. and support projection matrix Specifically, calculate the first query feature. v and first supporting features Cross covariance matrix ; Calculate the first autocovariance matrix of the first query feature. and the second autocovariance matrix of the first supporting feature Canonical correlation analysis was performed based on the cross covariance matrix, the first autocovariance matrix, and the second autocovariance matrix to solve for the query projection matrix and the support projection matrix.

[0032] In some examples, the first query feature v and first supporting features Cross covariance matrix The calculation formula is as follows:

[0033] in, It is the total number of observation pairs. It is a sub-support feature in the i-th observation pair. It is a subquery feature in the i-th observation pair. Indicate subquery characteristics transpose, Characterization and The correlation, It represents the total number of observation pairs.

[0034] For example, the cross covariance matrix can also be approximated by a moving average. :

[0035] in, ; To support the mean vector of the features, This is the mean vector of the query feature.

[0036] In some examples, maximizing the linear correlation between the two sets of features and normalizing the projected features yields an optimized function that supports canonical correlation analysis of the features and query features:

[0037] in, and These are the support projection matrix (used to project support features) and the query projection matrix (used to project query features). The trace of the matrix, To share subspace dimensions.

[0038] The cross covariance matrix First autocovariance matrix and the second autocovariance matrix Substituting the above optimization function, we can find the optimal support projection matrix that maximizes the above optimization function. and query projection matrix .

[0039] In some embodiments, the process of finding the maximum value of the optimization function in the canonical correlation analysis described above can be converted into solving the matrix. Perform singular value decomposition:

[0040] in, and These are the left and right singular vector matrices, respectively. It is a diagonal matrix, and the diagonal elements are canonical correlation coefficients.

[0041] The optimal support projection matrix can be solved more conveniently through singular value decomposition. and query projection matrix : .

[0042] S13, Project the first query feature into the second query feature based on the query projection matrix; project the first support feature into the second support feature based on the support projection matrix.

[0043] like Figure 2 As shown, using a support projection matrix For the first supporting feature Projection is performed to obtain the second support feature after projection. Using the query projection matrix For the first query feature v Projection is performed to obtain the second query feature. . It supports projection matrix transpose, Query projection matrix The transpose of .

[0044] This is equivalent to extracting supporting features relevant to the current query task within a shared subspace.

[0045] S14, enhance the second query feature based on the second supporting feature to obtain the target query feature.

[0046] Second support feature after projection With the second query feature These are optimized features, and the two typically have the highest correlation. They can suppress redundant components in the support features that are irrelevant to the current query task, while extracting more useful and discriminative features. Therefore, based on the projected second support feature... Able to perform second query features To enhance the target query features more effectively, we can obtain more useful and detailed features.

[0047] It should be understood that the projected second supporting features can be directly fused with the second query features to obtain the target query features. Alternatively, the second supporting features can be enhanced, and the enhanced third supporting features can be fused with the second query features to obtain the target query features.

[0048] In some embodiments, the second support feature is enhanced to obtain an enhanced third support feature (which can be denoted as...). ).For example, Figure 2 As shown, the current object detection model also includes a fractional augmentation module to be trained, which can be used to enhance the projected second support features. Perform feature enhancement and output third supporting features. The specific processing for feature enhancement of the second supporting feature will be described in detail below, and will not be discussed here.

[0049] like Figure 2 As shown, the current object detection model also includes a cross-sample attention module, which is based on the third support feature. With the second query feature By merging, the target query features are obtained. .

[0050] Specifically, the cross-sample attention module can calculate the similarity between the sub-query features of each position (i.e., query position) in the second query feature and the sub-support features of each position (i.e., support position) in the third support feature, and obtain the cross-sample attention weights corresponding to each sub-support feature.

[0051] For example, for the third supporting feature With projection query features Spatial expansion, respectively represented as and ,in For any query position Subquery features Calculate any support position Sub-support features With the characteristics of this subquery The similarity between them is calculated and normalized to obtain sub-support features. Corresponding cross-sample attention weights :

[0052] in, Characterization of support location Sub-support features Features of subqueries The similarity between them; yes The dimensions. All supported locations. The sub-support features correspond to the same subquery features Cross-sample attention weights The sum is 1.

[0053] Furthermore, based on cross-sample attention weights, the sub-support features are weighted and fused into the sub-query features to obtain the sub-query enhancement features for each position (i.e., the query position). The sub-query enhancement features for each position (i.e., the query position) are then combined to obtain the target query features. In this way, cross-sample attention alignment between the enhanced third support feature and the second query feature can be effectively achieved, improving the cross-sample feature alignment capability.

[0054] For example, any query position can be calculated using the following formula. Subquery enhancement features

[0055] Furthermore, the subqueries at all query locations are enhanced and reorganized: ;in, It is a query feature resulting from the combination of enhanced features of subqueries across all query positions.

[0056] Combined query features As a target query feature It can also be based on third support features. Generate dynamic convolution kernels to process query features. (Also known as the enhanced third query feature) Fine-tuning, using the fine-tuned query feature as the target query feature. It should be understood that using the fine-tuned query features as target query features can improve the localization and discrimination capabilities of small target detection when participating in subsequent target prediction.

[0057] In some embodiments, for the third supporting feature Perform global average pooling to obtain support description vectors. Where GAP() represents global average pooling. Based on support description vectors Generate channel-wise dynamic convolutional kernels using a multilayer perceptron (MLP): Apply dynamic convolution kernels to the aligned query features. (i.e., the enhanced third query feature), for The features of each channel c are dynamically convolved and fine-tuned to obtain the fourth query feature. Then, the third query feature and the fourth query feature are fused to obtain the target query feature. .

[0058] For example, the formula for dynamic convolution fine-tuning for each channel is as follows: ; in, It is the total number of channels. It is the third query feature The subquery feature of the c-th channel; It is the entire dynamic convolution kernel The kernel in the middle corresponds to the convolution kernel of the c-th channel; Pair query features The fine-tuned subquery feature is equivalent to the subquery feature of the c-th channel in the fourth query feature.

[0059] It should be understood that the fusion of the third and fourth query features essentially involves fusing the sub-query features corresponding to the same channel within the third and fourth query features. For example, in, This refers to the fused subquery features corresponding to channel c. Thus, the fused subquery features corresponding to all channels form the target query features. .

[0060] S15, target prediction is performed based on the target query features to obtain the target prediction result.

[0061] like Figure 2 As shown, the current object detection model includes a prediction head. This prediction head is a multi-task prediction head and may include a center heatmap head, a local offset head, a confidence head, and a direction head. The target query feature will be... For example, let's take the target query features as an example. Target prediction is performed after inputting the multi-task prediction head.

[0062] Specifically, after inputting the target query features into the multi-task prediction head, the center heatmap head predicts the center heatmap (denoted as the predicted center heatmap), the local offset head predicts the offset of the target center point (denoted as the predicted center point offset), the confidence head predicts the confidence of the candidate target (denoted as the predicted confidence), and the direction head predicts the direction angle of the candidate target (denoted as the predicted direction angle). These constitute the target prediction result. In addition to the above prediction information, the target prediction result may also include other information, which is not limited thereto.

[0063] In some embodiments, the center heatmap header is based on the center heatmap header weight. For target query features After applying a convolutional mapping, the target center is activated by a sigmoid function to predict the probability distribution of the target center point (i.e., the predicted probability that each position is the target center point), thus obtaining a predicted center heatmap. The specific formula is: ; in, For the Sigmoid function, Represents the heatmap of the prediction center any position in The predicted probability of the target center point.

[0064] In some embodiments, the local offset head is used to compensate for quantization errors introduced during feature map downsampling. The offset head is a single-layer convolutional network that desamples the target query features. Input to the offset head, output a set of predicted offsets of the center points of each positive sample point. The specific formula is: ; in, For offset header weights, This is the bias parameter. (Set) The Middle Location of positive sample points The center point prediction offset can be denoted as , It is the predicted number Location of positive sample points The x-coordinate offset of the center point, It is the predicted number Location of positive sample points The offset of the center point's ordinate.

[0065] In some embodiments, the confidence header is used to predict the confidence of candidate targets. This incorporates target query features. Input into the confidence header, output a set of predicted confidence scores for each candidate target. The specific formula is as follows:

[0066] in, For confidence head weights, Indicates position The prediction confidence level of the corresponding candidate target.

[0067] In some embodiments, the orientation head is used to predict the target's rotation angle, improving the model's ability to detect targets with arbitrary orientations. This incorporates target query features. Input the orientation head, and output the set of predicted orientation angles of the candidate targets corresponding to each positive sample point position. The specific formula is as follows:

[0068] in, For the direction head weight, Indicates the first Location of positive sample points The predicted orientation angle of the corresponding candidate target.

[0069] S16, determine the loss value based on the target prediction result, and update and adjust the parameters of the current target detection model based on the loss value.

[0070] Specifically, the center heatmap loss value is determined based on the difference between the predicted center heatmap and the corresponding true center heatmap. The local offset loss value is determined based on the difference between the predicted offset of the center point and the corresponding true offset of the center point; the confidence loss value is determined based on the difference between the predicted confidence of the candidate target and the corresponding true label; and the orientation angle loss value is determined based on the difference between the predicted orientation angle of the candidate target and the corresponding true orientation angle. Finally, the final loss value is determined based on the center heatmap loss value, the local offset loss value, the confidence loss value, and the orientation angle loss value.

[0071] The parameters of each training module of the current object detection model (e.g., feature extraction module, canonical correlation projection module, fractional augmentation module, prediction head, etc.) are updated and adjusted based on the loss value. It should be understood that this is not limited to the modules listed; for example, the parameters of the cross-sample attention module and the dynamic convolution module can also be adjusted.

[0072] In some embodiments, the central heat loss value The calculation formula is as follows:

[0073] in, For the true center heat map Middle position The probability of being the center point of the target. , and For the hyperparameters of the regulation factor, This represents the number of positive sample center points.

[0074] In some embodiments, the local offset loss value The calculation formula is as follows: ; in, It is the first The true x-coordinate offset of each positive sample point. It is the first The true ordinate offset of each positive sample point.

[0075] For example, let the sampling step size be... , No. The true center coordinates of the locations of the positive sample points are: The corresponding actual offset is: ; in, This indicates the floor function, which rounds down to the nearest integer, i.e., takes the largest integer not greater than the given number.

[0076] In some embodiments, confidence loss value The calculation formula is as follows:

[0077] in, It is a set of difficult sample points. It includes all positive sample points in the feature map, as well as the Top-M hard negative sample points selected according to the prediction confidence, where M is the preset number; yes The number of sample points in the sample. It is a set Middle position This corresponds to the true label of the candidate target. Calculating the confidence loss value using this formula can enhance the learning of hard-negative samples, thereby improving detection accuracy.

[0078] In some embodiments, the orientation angle loss value The calculation formula is as follows:

[0079] in, Indicates the first Location of positive sample points The true orientation angle of the corresponding candidate target, This represents the number of positive sample points. The loss function utilizes trigonometric functions to mitigate the periodicity of the angle. and The problem of discontinuity at nearby boundaries.

[0080] In some embodiments, the total loss value is obtained by weighted fusion of the center heatmap loss value, local offset loss value, confidence loss value, and orientation angle loss value, as shown in the following formula: ; in, These are the weighting coefficients for the loss of each task. During training, the model simultaneously learns target center localization, sub-pixel offset correction, candidate confidence evaluation, and orientation angle estimation by jointly optimizing the overall loss value.

[0081] In some embodiments, the target rotation bounding box parameters can be further decoded so that the positioning box used to define the target is displayed in the query sample image.

[0082] Specifically, the selected Top-K candidate regions are then fine-tuned based on the query features. The coordinates of the corresponding center point in the query feature map are used for nearest neighbor matching with the local maxima center response point (i.e., positive sample point) in the prediction center heatmap to obtain candidate targets that successfully match the local maxima center response point; for the th For each successfully matched candidate target, let the prior width and height of its corresponding candidate region in the original image (i.e., the original query sample image) be denoted as . ,Right now, Indicates the relationship with the first The width and height priors of candidate regions matched by each candidate target are used for decoding the rotated bounding box.

[0083] Furthermore, based on the target prediction results and the comparison with the first... The prior width and height of the candidate region matching the nth candidate target, for the nth candidate target Decoding the rotated bounding boxes of the candidate targets yields:

[0084]

[0085] in, Query features for the target The downsampling step size of the represented query feature map relative to the original query sample map. For the predicted first The coordinates of the center point of the rotated bounding box of each candidate target in the query sample image. For the predicted first The width and height of the rotated bounding boxes of each candidate target in the query sample image. For the first The predicted orientation angle of the rotated bounding box of each candidate target. No. The prediction confidence of each candidate target.

[0086] It should be understood that after updating and adjusting the parameters of the current object detection model based on the total loss value calculated in this round, the updated object detection model can be used as the current object detection model to be trained in the next round. This iterative training continues until the model training termination condition is met, resulting in the final object detection model. The object detection model trained by the method in this application is not limited to small object detection; it can detect any object. Compared with object detection models trained by traditional methods, the performance in small object detection is particularly outstanding.

[0087] In the usage phase of the trained object detection model, the current image to be detected is input into the model. Deep semantic features are extracted from this image through the meta-learning backbone network in the trained feature extraction module. Keypoint features are then extracted further based on the RPN network. These keypoint features are fused with the deep semantic features to obtain the final target image features. These target image features are then input into the prediction head to obtain the target detection result for the current image, thus marking the target's detection box (also known as the target rotated bounding box) in the current image. It should be understood that during model training, relevant feature information from the support sample image and the query sample image, which is beneficial for target detection in the query sample image, is fused into the query features of the query sample image for model parameter update training. Therefore, key modules in the model, such as the feature extraction module, keypoint extraction module, and prediction head, are trained more accurately, improving their processing accuracy and thus enhancing the overall detection accuracy of the object detection model. In this way, even without the assistance of information from the support sample image, the model can detect targets more accurately, especially small targets.

[0088] As can be seen from the above, the second supporting feature can be enhanced through the fractional-order enhancement module. Feature enhancement is performed to obtain the third supporting feature. The third supporting feature Fusing with the second support feature yields richer and more detailed target query features. The following section will describe in more detail how to integrate the second support feature. Feature enhancement is performed to obtain the third supporting feature. .

[0089] In some embodiments, a fractional Fourier transform model is constructed for the second supporting feature. For example... Figure 2 As shown, the fractional-order enhancement module can enhance the second supporting feature. Perform a fractional Fourier transform to separate the second support features. The phase and amplitude are determined to construct the corresponding fractional Fourier transform model.

[0090] For example, the formula for the fractional Fourier transform is as follows:

[0091] in, For time-domain variables, For fractional domain variables, The fractional rotation angle (i.e., the order of the fractional Fourier transform) is... α ), kernel function Specifically:

[0092] in, The imaginary unit, This is the Dirac function.

[0093] Furthermore, the second supporting feature Substituting the above fractional Fourier transform formula, we apply a fractional Fourier transform along a preset spatial dimension, i.e.:

[0094] Thus, the second support feature can be separated. Phase and amplitude: Phase and amplitude information are used to explain how structural information and energy distribution are represented in the fractional Fourier transform domain.

[0095] Please continue reading. Figure 2 In the second support feature After performing a fractional Fourier transform, differential optimization reconstruction can be performed to obtain the enhanced third support feature. That is, feature enhancement is performed during the differential optimization reconstruction process (for example, using a fractional-order total variational optimization model to implement support feature optimization for feature enhancement), thus enabling the third supporting feature. In addition to semantic features in the image, it also includes statistical features. While preserving the semantic features of the image, it further enhances high-frequency detail features such as edges and textures, and suppresses background noise, effectively achieving the enhancement of support features.

[0096] In some examples, the steps of differential optimization reconstruction include: (1) Establish the multiplier representation relation of the fractional derivative in the fractional Fourier transform domain: (Equation 1) in, Indicates support for feature function ofβ First derivative (i.e.) β It is the order of the fractional differential. Indicates support features The order of the process is α Fractional Fourier transform, For fractional domain variables, It is a multiplier in the FrFT domain (i.e., the domain of fractional Fourier transform). This is the inverse fractional Fourier transform. It should be understood that in the formula... Both refer to supporting feature variables in general, rather than specific supporting feature values.

[0097] In this way, the calculation of long-range convolutional fractional derivatives in the spatial domain can be transformed into multiplication operations in the transform domain, which can reduce the complexity of solving numerical problems after constructing a fractional total variation optimization model, thereby enhancing high-frequency details.

[0098] For ease of understanding, the derivation process of the above (Equation 1) is described below: In supporting features In this process, fractional derivatives enhance the high-frequency components of features while nonlinearly preserving the low-frequency components. Therefore, constructing support features... Fractional differential model: (Equation 2) in, Let be the gamma function, and s be the integration variable. The lower limit of integration, for Fractional integral operators of order. It should be understood that, since the derivative is the inverse operation of integration, in integer integrals... The second derivative is The inverse operation of the second integral is: So, fractional order The derivative of order, first do Order integral, then do it again. The ordinary integral of the second order is then used to construct the fractional derivative as follows: (Equation 3) make Targeting features The process of calculating the fractional derivative FrFT:

[0099] = (Next step will) (Replace with Equations 2 and 3) =

[0100]

[0101] =

[0102] =

[0103] = ; Furthermore, taking the inverse FrFT of both sides of the above equation and eliminating the phase compensation term:

[0104] because kernel function contains This cancels out the exponential term mentioned above, and the fractional derivative and FrFT satisfy:

[0105] in, , This is the inverse fractional Fourier transform. It is achieved by applying multipliers to the FrFT domain. Selective frequency bands and phase compensation enable higher frequency and more directional enhancement. Fractional Fourier transform provides the ability to find favorable projection angles, while fractional derivatives provide controllable power-law enhancement. The combination of the two is theoretically complementary, making it easier to separate target signals from background noise and enhance their discriminative power in scenarios with few samples and weak target energy.

[0106] (2) Construct a fractional total variation optimization model for the second support feature to enhance the second support feature and obtain the enhanced third support feature; wherein, the fractional gradient magnitude in the regularization term of the fractional total variation optimization model is determined according to the established multiplier representation relationship of the fractional derivative in the fractional Fourier transform domain.

[0107] It should be understood that constructing a fractional-order total variation optimization model can suppress noise and enhance target features, particularly high-frequency features. Specifically, the formula for the fractional-order total variation optimization model is expressed as follows: (Equation 4) in, Fractional gradient; Fractional gradient regularization function , These are parameters used to prevent numerical instability caused by the gradient magnitude approaching zero. , Let be the magnitude of the fractional gradient; This represents the smooth degradation operator acting on supporting features. This represents the second support feature after projection (i.e., the original support feature before total variation optimization, used for fidelity constraints). For balancing parameters.

[0108] The first term in the above fractional total variation optimization model It is a regular term, the second term. It's a fidelity-enhancing item; the key feature enhancement comes from the first item. The two-dimensional fractional gradient, when written out, is a vector composed of fractional derivatives along each direction: ; Therefore, based on the relationship established in (Equation 1) Each directional derivative can be written in multiplier form in the FrFT field, as follows: , ; Therefore, the gradient magnitude in the regularization term It is no longer an abstract symbol, but has become: ; That is, based on the multiplier representation relation of the fractional derivative in the fractional Fourier transform domain established in Equation 1, the fractional gradient magnitude in the regularization term of the fractional total variation optimization model can be realized by the FrFT domain multiplier.

[0109] It should be understood that the projected second support feature can be... As initial supporting features, these are substituted into (Equation 4) as initial values ​​for iteration, and the fractional-order total variation optimization model is solved iteratively. The support feature with the smallest value is equivalent to the second support feature. By performing feature enhancement, an enhanced third support feature can be obtained.

[0110] In some embodiments, the step of finding the supporting features that minimize the value of the fractional total variation optimization model includes: (2-1) Based on the perturbation direction test function, perturbation amplitude and support feature function, construct the perturbation function; substitute the perturbation function into the fractional total variation optimization model to obtain the perturbation energy function with respect to the perturbation amplitude.

[0111] Specifically, the test function is performed for any perturbation direction. Constructing the perturbation function : ;in, For the disturbance amplitude, To support the characteristic function.

[0112] The perturbation function Substitute into the fractional total variation optimization model In (Equation 4), we obtain the disturbance energy function with respect to the disturbance amplitude. , ,Right now:

[0113] (2-2) Take the derivative of the perturbation energy function with respect to the perturbation amplitude and take the value at the perturbation amplitude of 0 to obtain the first variation; set the first variation to zero to construct the Euler-Lagrange equation.

[0114] Specifically, for the perturbation energy function Regarding the amplitude of the disturbance Differentiate, and in Taking the value at point, we obtain the first variation. : ; Wherein, the two-dimensional fractional gradient satisfies:

[0115] make , The above formula can then be written as:

[0116] Furthermore, the adjoint operator of the fractional directional derivative is utilized. and convolution operators The adjoint operator ,have: ; Among them, if If the kernel is a symmetric Gaussian blur, then .

[0117] (2-3) Construct gradient flow dynamic equations based on Euler-Lagrange equations.

[0118] It should be understood that, if Fractional Total Variational Optimization Model The minimum point is then tested for any perturbation direction. All .therefore, Since it is always zero, we obtain the corresponding Euler-Lagrange equation:

[0119] To numerically solve the above Euler-Lagrange equations, an artificial time variable is introduced. Construct the gradient flow dynamic equation:

[0120] The gradient flow dynamic equation is updated along the energy decrease direction. This allows for a gradual approximation of the fractional-order total variation optimization model. The minimum solution.

[0121] It should be understood that the fractional directional derivative can be discretized using a Grünwald–Letnikov type difference approximation: for direction: ; for direction: ; in, For discrete grid step size, This represents the number of truncated terms.

[0122] It should be understood that discretization makes it easier to solve the dynamic equations of gradient flow numerically.

[0123] (2-4) The gradient flow dynamic equation is discretized in time using the explicit gradient descent method to iteratively solve for the support features until the support features that minimize the value of the fractional total variation optimization model are found.

[0124] Specifically, the iterative update formula is as follows:

[0125] in, Indicates the iteration step size. Indicates the first Support features are solved in the next iteration. Indicates the first Support features are solved in the next iteration. , .

[0126] The initial support features before iteration are the second support features. That is, the second supporting feature This is the first supporting feature to be substituted into the above iterative update formula. Based on the multiplier representation relation of the fractional derivative in the fractional Fourier transform domain established by (Equation 1), the iterative update formula... , , , , All of these can be equivalently rewritten as multiplier operations in the FrFT domain, thereby transforming the convolutional or differential calculations in the original iterative process into multiplier calculations in the transform domain (see above for details, which will not be repeated here) to simplify iterative calculations.

[0127] Through repeated iterations, after the iterations converge, the optimal feature enhancement result of the total variation optimization model is obtained, that is, the support feature that minimizes the value of the fractional total variation optimization model.

[0128] It should be understood that the support feature that minimizes the value of the fractional total variation optimization model can be directly used as the final enhanced third support feature. It can also serve as a preliminary enhancement of the fourth supporting feature. The fourth support feature is initially enhanced. Further enhancement yields a third supporting feature. .

[0129] In some embodiments, a fourth support feature is initially enhanced. Next, lightweight convolutional enhancement processing is performed on the fourth support feature, including: enhancing the fourth support feature... Lightweight convolutions are used to adjust the distribution of high-frequency enhancement features, resulting in optimized support features; these optimized support features are then... With the first supporting feature Fusion yields a third supporting feature. .

[0130] The specific formula is as follows: ;in, Indicates support for the fourth feature Optimized support features obtained by performing lightweight convolution.

[0131] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. At least some of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, and the execution order of these steps or stages is not necessarily sequential. Instead, they may be executed alternately or in turn with other steps or at least a portion of the steps or stages in other steps.

[0132] Based on the same inventive concept, this application also provides an apparatus for implementing the aforementioned training method for a target detection model based on few samples. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more embodiments of the target detection model training apparatus based on few samples provided below can be found in the limitations of the target detection model training method based on few samples described above.

[0133] In one embodiment, such as Figure 3 As shown, a training device for a target detection model based on few samples is provided. The device includes: The feature extraction module 301 is used to extract the first query features from the query sample image using the current target detection model to be trained in each round of training with a small sample set. The canonical relevance projection module 302 is used to determine the query projection matrix and the support projection matrix; wherein, the canonical relevance of the first query feature and the first support feature of the support sample map after projection optimization according to the corresponding query projection matrix and support projection matrix is ​​maximized; the first query feature is projected into a second query feature based on the query projection matrix; and the first support feature is projected into a second support feature based on the support projection matrix. Feature enhancement module 303 is used to enhance the second query feature based on the second supporting feature to obtain the target query feature; Prediction module 304 is used to predict the target based on the target query features and obtain the target prediction result; The parameter update module 305 is used to determine the loss value based on the target prediction result and update and adjust the parameters of the current target detection model based on the loss value.

[0134] The aforementioned training device for a target detection model based on a few samples can also implement the steps in the embodiments of this application. Each module in this device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0135] In one embodiment, a computer device is provided, which is a terminal or a server. The internal structure diagram of the computer device can be as follows: Figure 4 As shown, the computer device includes a processor, memory, input / output interfaces, and a communication interface. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external devices via a network connection. When the computer program is executed by the processor, it implements a method for training a target detection model based on few samples.

[0136] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0137] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the embodiments of this application.

[0138] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the embodiments of this application.

[0139] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the embodiments of this application.

[0140] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited thereto.

[0141] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above embodiments are merely illustrative of several implementation methods of this application and should not be construed as limiting the scope of this patent application. Those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for training a target detection model based on a small number of samples, characterized in that, The method includes: In each round of training of the object detection model using a small sample set, the first query feature is obtained by extracting features from the query sample image using the current object detection model to be trained in this round. Determine the query projection matrix and the support projection matrix; wherein, the typical correlation is maximized after projecting the first query feature and the first support feature of the support sample map according to the corresponding query projection matrix and the support projection matrix. The first query feature is projected into the second query feature based on the query projection matrix; the first support feature is projected into the second support feature based on the support projection matrix. The second query feature is enhanced based on the second supporting feature to obtain the target query feature; Target prediction is performed based on target query features to obtain target prediction results; The loss value is determined based on the target prediction result, and the parameters of the current target detection model are updated and adjusted based on the loss value.

2. The method according to claim 1, characterized in that, The step of extracting features from the query sample image to obtain the first query feature includes: Convolution is performed on the query sample map to extract deep semantic features, resulting in an initial query feature map; The region proposal network generates multiple anchor boxes corresponding to the initial query feature map, and selects the first number of anchor boxes as candidate regions based on the confidence score of each anchor box. Key point features are extracted from each candidate region, and the extracted key point features are fused with the features of the corresponding candidate regions in the initial query feature map to obtain the first query feature.

3. The method according to claim 1, characterized in that, The determination of the query projection matrix and the supporting projection matrix includes: Calculate the cross-covariance matrix of the first query feature and the first support feature; Calculate the first autocovariance matrix of the first query feature and the second autocovariance matrix of the first support feature; Canonical correlation analysis is performed based on the cross covariance matrix, the first autocovariance matrix, and the second autocovariance matrix to solve for the query projection matrix and the support projection matrix.

4. The method according to claim 1, characterized in that, The step of enhancing the second query feature based on the second supporting feature to obtain the target query feature includes: The second supporting feature is augmented to obtain the augmented third supporting feature; For each sub-query feature in the second query feature, the similarity between the sub-query feature and the sub-support features in the third support feature is calculated to obtain the cross-sample attention weights corresponding to each sub-support feature; based on the cross-sample attention weights, each sub-support feature is weighted and fused into the sub-query feature to obtain the sub-query enhancement feature at each position. The target query features are obtained by combining the enhanced features of the subqueries at each position.

5. The method according to claim 4, characterized in that, The step of enhancing the second supporting feature to obtain the enhanced third supporting feature includes: Construct a fractional Fourier transform model for the second supporting feature and establish the multiplier representation relation of the fractional derivative in the fractional Fourier transform domain; A fractional total variation optimization model is constructed for the second supporting feature to enhance the second supporting feature and obtain the enhanced third supporting feature. The fractional gradient magnitude in the regularization term of the fractional total variation optimization model is determined based on the established multiplier representation relation of the fractional derivative in the fractional Fourier transform domain.

6. The method according to claim 5, characterized in that, The construction of a fractional-order total variational optimization model for the second support feature to enhance the second support feature and obtain the enhanced third support feature includes: The support feature that minimizes the value of the fractional total variation optimization model is used as the fourth support feature for initial enhancement. Lightweight convolution is performed on the fourth support feature to adjust the distribution of high-frequency enhancement features, resulting in optimized support features; The optimized support feature is fused with the first support feature to obtain the enhanced third support feature.

7. The method according to claim 6, characterized in that, The supporting features that minimize the value of the fractional total variation optimization model include: Based on the perturbation direction test function, perturbation amplitude, and support feature function, a perturbation function is constructed; the perturbation function is substituted into the fractional-order total variation optimization model to obtain the perturbation energy function with respect to the perturbation amplitude. The first variation is obtained by differentiating the perturbation energy function with respect to the perturbation amplitude and taking a value at the perturbation amplitude of 0; the first variation is set to zero to construct the Euler-Lagrange equation. The gradient flow dynamic equation is constructed based on the Euler-Lagrange equation. The gradient flow dynamic equation is discretized in time using the explicit gradient descent method to iteratively solve for the support features until the support features that minimize the value of the fractional total variation optimization model are found.

8. The method according to claim 4, characterized in that, The process of combining the enhanced features of subqueries at each location to obtain the target query features includes: The enhanced features of the subqueries at each position are combined to obtain the enhanced third query feature; The third support feature is subjected to global average pooling to obtain the support description vector; Based on the support description vector and the multilayer perceptron, a channel-wise dynamic convolution kernel is generated; Based on the dynamic convolution kernel, the features of each channel in the third query feature are dynamically convolved and fine-tuned to obtain the fourth query feature; The target query feature is obtained by fusing the third query feature with the fourth query feature.

9. The method according to any one of claims 1 to 8, characterized in that, The target prediction results include a predicted center heatmap, a predicted center point offset, a predicted confidence level for candidate targets, and a predicted orientation angle for candidate targets. The determination of the loss value based on the target prediction result includes: Based on the difference between the predicted center heat map and the corresponding real center heat map, the center heat loss value is determined; Based on the difference between the predicted offset of the center point and the actual offset of the corresponding center point, the local offset loss value is determined; The confidence loss value is determined based on the difference between the predicted confidence of the candidate target and the corresponding true label; The direction angle loss value is determined based on the difference between the predicted direction angle and the corresponding true direction angle of the candidate target; The final loss value is determined based on the central heat loss value, local offset loss value, confidence loss value, and orientation angle loss value.

10. A training device for a target detection model based on a small number of samples, characterized in that, The device includes: The feature extraction module is used to extract the first query features from the query sample image using the current target detection model to be trained in each round of training with a small sample set. A canonical relevance projection module is used to determine a query projection matrix and a support projection matrix; wherein, the canonical relevance of the first query feature and the first support feature of the support sample map after projection optimization according to the corresponding query projection matrix and the support projection matrix is ​​maximized; the first query feature is projected into a second query feature based on the query projection matrix; and the first support feature is projected into a second support feature based on the support projection matrix. The feature enhancement module is used to enhance the second query feature based on the second supporting feature to obtain the target query feature; The prediction module is used to predict the target based on the target query features and obtain the target prediction result. The parameter update module is used to determine the loss value based on the target prediction result, and to update and adjust the parameters of the current target detection model based on the loss value.

Images