Radar image open set target detection method based on wildcard character learning

Abstract

The invention relates to a radar image open set target detection method based on wildcard character learning, and the method comprises the steps: obtaining a data set, and constructing a training data set which supports the detection of a known target and an unknown target; a wildcard character learning strategy is adopted, a model is driven in a self-supervised mode, and discriminative features of a known target and general characterization of an unknown target are learned; an improved YOLO detector is utilized to perform multi-scale feature extraction and fusion, and target detection in a radar image is realized through text-guided feature enhancement, open classification decision and adaptive confidence optimization. According to the embodiments of the invention, a complete and efficient open target detection solution is provided for radar image detection.

Description

Technical Field

[0001] This application relates to the field of computer vision, specifically to a radar image target detection method based on wildcard learning and open set discrimination. Background Technology

[0002] The main technical shortcomings of existing radar image target detection methods lie in their fundamental limitations in handling unknown targets. Traditional methods forcibly categorize all detected objects into a predefined target category system, ignoring the inherently non-fixed feature patterns and highly discrete distribution of unknown targets. When faced with novel or mutated targets, existing systems often misclassify them as known target categories or simply ignore them as background. This forced classification mechanism fails to accurately reflect the essential characteristics of unknown targets, leading to serious false negatives and false negatives. In terms of open set detection, existing methods lack self-supervised learning mechanisms for radar image features. They cannot effectively utilize the physical characteristics of radar imaging and the scattering characteristics of targets to autonomously discover and learn the feature representations of unknown targets without relying on extensive manual annotation. This significantly limits the adaptability and practicality of the system in real-world open environment perception, especially in critical application scenarios requiring rapid response to emerging targets. Summary of the Invention

[0003] This application provides a method for open set target detection in radar images based on wildcard learning, offering a complete and efficient solution for open target detection in radar images.

[0004] According to one of the solutions in this application, a method for open set target detection in radar images based on wildcard learning is provided, including: Obtain the dataset and construct a training dataset that supports the detection of both known and unknown targets; A wildcard learning strategy is adopted to drive the model in a self-supervised manner, learning the discriminative features of known targets and the general representation of unknown targets; By utilizing an improved YOLO detector for multi-scale feature extraction and fusion, and through text-guided feature enhancement, open classification decision and adaptive confidence optimization, target detection in radar images is achieved.

[0005] In some embodiments, prior to performing target detection in the radar image, the method further includes: Multimodal data fusion is performed based on radar image data acquisition. Based on open data partitioning methods, dataset format conversion and partitioning are performed.

[0006] In some embodiments, multimodal data fusion based on radar image data acquisition includes: Radar image data acquisition; Target category definition and labeling; Multimodal training samples were constructed by combining radar images and text descriptions; Radar image preprocessing.

[0007] In some embodiments, the dataset format conversion and partitioning based on open data partitioning methods includes: Region-text pair construction; Wildcard embedding initialization; Datasets are partitioned based on samples containing known categories and mixed samples containing known and unknown categories.

[0008] In some embodiments, the model training method includes: Adaptive decision learning; Based on the wildcard learning strategy, we perform training on wildcards of known categories and learning on wildcards of unknown categories. Region-text contrast loss is used to filter out predictions with high confidence and high positioning accuracy as reliable pseudo-labels; Obtain the overall loss function; Dynamic threshold scheduling is achieved by using a dynamic threshold that adapts to changes over time.

[0009] In some embodiments, the detection of a target in a radar image includes: Multi-scale features are extracted using an improved YOLO detector; Multi-scale feature fusion network; Open target classification; Unknown target filtering; Dynamic threshold adaptive adjustment; Post-processing of test results; New class discovery and vocabulary expansion.

[0010] In some embodiments, the multi-scale feature fusion network includes at least: The positioning information is fed back to the deep features through upsampling and downsampling paths; The feature fusion output is used for decision integration using a weighted fusion strategy. The text embedding is updated using an image pooling attention mechanism; Text-guided feature enhancement achieves feature modulation through cross-modal interaction.

[0011] In some embodiments, the open target classification includes at least: By combining visual-text alignment and IoU, multimodal two-head matching is achieved; For the detected targets, perform open classification.

[0012] In some embodiments, the post-processing of the detection results includes: Non-maximum suppression and confidence filtering are applied to the detection results, and low-quality detection boxes are removed based on dynamic thresholds; A cross-category suppression strategy is adopted to avoid overlap of known / unknown boxes.

[0013] In some embodiments, new class discovery and vocabulary expansion include: Unknown candidate convergence; Clustering based on regional characteristics; Semantic annotation.

[0014] This application presents various embodiments of a radar image open-set target detection method based on wildcard learning. At a minimum, it acquires a dataset and constructs a training dataset supporting the detection of both known and unknown targets. It employs a wildcard learning strategy, driving the model through self-supervision to learn discriminative features of known targets and general representations of unknown targets. It utilizes an improved YOLO detector for multi-scale feature extraction and fusion, achieving target detection in radar images through text-guided feature enhancement, open classification decision-making, and adaptive confidence optimization. This overcomes the limitations of traditional closed-set detection, enabling proactive identification and labeling of unknown targets through the wildcard learning mechanism, avoiding missed detections. A text embedding freezing strategy effectively addresses the catastrophic forgetting problem, ensuring the system's continuous learning capability. It supports flexible vocabulary expansion and offline deployment, reducing system complexity and operating costs. These embodiments provide a complete and efficient open-world target detection solution for radar images.

[0015] It should be understood that the foregoing general description and the following detailed description are exemplary and illustrative only, and are not intended to limit the scope of this application. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram of the overall structure of one embodiment of this application is shown; Figure 2 A detailed schematic diagram of the pooling portion of one embodiment of this application is shown; Figure 3 A detailed schematic diagram of the text guidance section of one embodiment of this application is shown; Figure 4This illustration shows a schematic diagram of wildcard and pseudo-tag generation according to an embodiment of this application; Figure 5 A schematic diagram of an open-world classification decision-making process according to an embodiment of this application is shown. Detailed Implementation

[0018] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0019] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0020] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0021] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0022] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.

[0023] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0024] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0025] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0026] Radar image target detection technology, as an important means of modern target perception, is widely used in important fields such as remote sensing, surveying, and exploration. Radar waves can penetrate obstructions such as smoke and dust, effectively detecting various hidden targets, such as surface facilities, vehicles, ships, and weather targets. Compared with traditional optical detection, radar equipment has advantages such as being unaffected by lighting conditions, high detection accuracy, and fast imaging speed, and has become the mainstream technology for open environment perception.

[0027] Currently, target detection in radar images mainly relies on traditional closed-set target detection methods. These methods are trained based on predefined categories of prohibited items, and deep learning models such as Faster R-CNN and the YOLO series have achieved significant results on standard datasets such as COCO and Pascal VOC.

[0028] Existing open-world object detection methods mainly include ORE (Open World Object Detection via Re-labeling), OWOD (Towards Open World Object Detection), and OW-DETR (Open World Detection Transformer).

[0029] Meanwhile, open-vocabulary object detection technologies have also developed rapidly. Methods such as YOLO-World integrate the CLIP text encoder into the YOLO framework, leveraging the cross-modal alignment capability of the vision-language model to detect categories that were not seen during training but can be described by text. These methods have expanded detection capabilities to some extent, enabling models to detect thousands or even tens of thousands of predefined categories.

[0030] The main technical shortcomings of existing radar image target detection methods lie in their fundamental limitations in handling unknown targets. Traditional methods forcibly categorize all detected objects into a predefined target category system, ignoring the inherently non-fixed feature patterns and highly discrete distribution of unknown targets. When faced with novel or mutated targets, existing systems often misclassify them as known targets or simply ignore them as background. This forced classification mechanism fails to accurately reflect the essential characteristics of unknown targets, leading to serious false negatives and false negatives. In terms of open set detection, existing methods lack self-supervised learning mechanisms for radar image features. They cannot effectively utilize the physical characteristics of radar imaging and the scattering characteristics of targets to autonomously discover and learn the feature representations of unknown targets without relying on extensive manual annotation. This greatly limits the adaptability and practicality of the system in real-world open environments, especially in critical application scenarios requiring rapid response to emerging targets.

[0031] Based on the background section described above, this application provides illustrative solutions to address the deficiencies in the prior art through embodiments, but these are not intended to limit the scope of patent protection claimed in this application.

[0032] As one of the solutions, comprehensive Figures 1 to 5 As shown, embodiments of this application provide a radar image open set target detection method based on wildcard learning, including: Obtain the dataset and construct a training dataset that supports the detection of both known and unknown targets; A wildcard learning strategy is adopted to drive the model in a self-supervised manner, learning the discriminative features of known targets and the general representation of unknown targets; By utilizing an improved YOLO detector for multi-scale feature extraction and fusion, and through text-guided feature enhancement, open classification decision and adaptive confidence optimization, target detection in radar images is achieved.

[0033] In light of the foregoing, this application aims to propose a radar image open set target detection method based on wildcard learning. Addressing the problems of existing technologies, it establishes a category system for open-world target detection and a multimodal data fusion mechanism. Data preprocessing is performed in conjunction with the physical characteristics of radar imaging to construct a training dataset supporting the detection of both known and unknown targets. A wildcard learning strategy is employed, driving the model in a self-supervised manner to simultaneously learn the discriminative features of known targets and the general representations of unknown targets, fundamentally building open-world perception capabilities. An improved YOLO detector is used for multi-scale feature extraction and fusion. Through text-guided feature enhancement, open-world classification decision-making, and adaptive confidence optimization mechanisms, accurate differentiation and detection of known targets, unknown targets, and background in radar images are achieved.

[0034] For example, with reference to the corresponding schematic diagram, the following steps S1 to S4 will be used as examples to further illustrate the radar image open set target detection method based on wildcard learning in this application.

[0035] In some implementations, the present application embodiments may include, prior to the target detection in the radar image, the method further comprising: Multimodal data fusion is performed based on radar image data acquisition. Based on open data partitioning methods, dataset format conversion and partitioning are performed.

[0036] In some implementation schemes, embodiments of this application may involve: multimodal data fusion based on radar image data acquisition, including: Radar image data acquisition; Target category definition and labeling; Multimodal training samples were constructed by combining radar images and text descriptions; Radar image preprocessing.

[0037] Step S1: Obtain the dataset.

[0038] Step S11: Radar image data acquisition, such as acquiring image data of millimeter-wave human body security inspection.

[0039] Step S12: Target category definition and labeling, such as manually labeling dangerous items in the image.

[0040] Step S13: Multimodal data fusion, such as combining radar images Construct multimodal training samples with text description T: (1) in, =x,y,w,h are the bounding box coordinates. This is the category label. (x,y) represents the coordinates of the center point of the bounding box, and w and h represent the width and height of the bounding box, respectively.

[0041] Step S13: Radar image preprocessing; for example, using bilateral filtering to remove radar imaging noise, normalizing radar wave scattering intensity and size standardizing processing.

[0042] In some implementations, the embodiments of this application may be: performing dataset format conversion and partitioning based on open data partitioning methods, including: region-text pair construction; wildcard embedding initialization; and partitioning the dataset based on samples containing known categories and mixed samples containing known and unknown categories.

[0043] Based on step S1 in the embodiment, this application may further include step S2.

[0044] Step S2: Dataset format conversion and partitioning.

[0045] Step S21: Constructing region-text pairs, such as converting traditional instance annotations. Convert to region-text pairs .in, For bounding box coordinates, For category labels, The generated text description is defined as follows: (2) in, It is the manually annotated text description corresponding to the i-th region. It is a condition used to check whether the above-mentioned manual description exists. It is the category label of the i-th region. is a set of known categories.

[0046] Step S22: Initialize wildcard embeddings, for example, initialize wildcard embeddings for unknown target detection: (3) Among them, Use a pre-trained CLIP text encoder. T represents the wildcard text feature embedding vector obtained through the text encoder, which is used to represent the semantic information of a certain category in the vector space.

[0047] Step S23: Dataset partitioning strategy. For example, adopt an open-world data partitioning method to partition the dataset D into: (4) Among them, only contains samples of known categories and and contains mixed samples of known and unknown categories.

[0048] In some embodiments, the embodiments of the present application may be: The model training method includes: adaptive decision learning; based on the wildcard learning strategy, perform known target wildcard training and unknown target wildcard learning; region-text contrast loss to filter out high-confidence and high-localization-precision prediction results as reliable pseudo labels; obtain the overall loss function; achieve dynamic threshold scheduling through a dynamically adaptive changing dynamic threshold.

[0049] Based on step S2 of the embodiments of the present application, step S3 may be further included.

[0050] Step S3: Model training.

[0051] Step S31: Adaptive decision learning. For example, use LoRA to perform parameter-efficient fine-tuning on the CLIP text encoder and freeze the original parameters : (5) Among them, is a low-rank decomposition matrix, . d is the dimension of the original weight matrix, r is the rank of the low-rank matrix, and r << d, which is used to control the number of fine-tuning parameters.

[0052] Step S32: Wildcard learning strategy, refer to Figure 4 for the generation of wildcards and pseudo labels shown in this embodiment.

[0053] Step S321: Known category wildcard training, for example: The CLIP text encoder is used to encode "object" to obtain wildcard embeddings, which are then used to uniformly learn "any object". During training, all ground truth bounding boxes are uniformly labeled as "object", and the remaining areas are treated as background.

[0054] For a known category, wildcards Training is performed using the following loss function: (6) in, is the feature embedding vector extracted from the i-th candidate region by the visual backbone network. For region similarity with known wildcards The cosine similarity score. This is the binary ground truth label for the i-th region, and its assignment rule is based on the IoU between the predicted region and the ground truth bounding box. (1 represents that the region is a foreground object, and 0 represents the background).

[0055] In practice, binary cross-entropy (including positive and negative samples) is used, as shown in the following formula: (7) in, This is the Sigmoid function, used to map similarity scores to a probability estimate of "belonging to the target category". Used to map similarity to the probability of "belonging to the object class"; The supervision label for the i-th region is assigned the same value as in equation (6); Let N be the binary cross-entropy loss function, and N be the number of candidate regions involved in the loss calculation.

[0056] Step S322: Unknown category wildcard learning, for example: from Derived from Unknown target category wildcard Training through self-supervised learning: (8) The region is selected based on the model's height, which is certain to be an object, but whose location does not match any of the known objects. It is the maximum intersection-union ratio (IoU) between the i-th predicted region and all ground truth bounding boxes. This is the IoU threshold. When the IoU between a region and the ground truth bounding box is lower than this value, the region is considered to be mismatched in position. This is the confidence threshold. When the objectivity score of a region is higher than this value, the region is considered to be highly certain to be an object.

[0057] Affine transformations in vector space: (9) in, Let W represent a general text embedding vector obtained through self-supervised learning, used to characterize an unknown target. W is a trainable parameter matrix used to perform linear transformations on object wildcards. Represents the bias vector, which is transformed by a linear transformation. Map to a subspace orthogonal to the known categories while preserving some common object features; Representing vectors L 2. Normalization is used to scale the output vector to a unit length.

[0058] Unknown category loss This method is used to train a model to identify unknown targets. Its core principle is to leverage knowledge of known categories to generate self-supervised soft pseudo-labels for unknown samples. The loss function is defined as follows: (10) in, For regional characteristics wildcards with unknowns The cosine similarity score; = For known wildcards Soft pseudo-labels are generated through Sigmoid function mapping. For potentially unknown regions filtered by the gating function Φ, the confidence level of the foreground objects can serve as a weak supervision signal that the region belongs to the unknown category. This is a self-supervised mechanism. This is a gating function whose function is to select high-potential candidate regions for unknown objects from all regions for training. The binary cross-entropy loss function is used to measure the unknown probability predicted by the model. With soft fake tags The differences between them.

[0059] Step S33: Region-text contrast loss.

[0060] Step S331: Design the region-text contrast loss function: (11) in, For the first i Feature embedding vectors of each region Embed the positive sample text that matches this region. Embed negative sample text that does not match the region. The negative sample set is typically defined as the set of samples in the current training batch excluding the positive samples. All other text embeddings besides those. This is the cosine similarity function. This is a temperature hyperparameter used to adjust the level of attention given to difficult negative samples. This refers to the batch size.

[0061] Step S332: Pseudo-tag generation and quality assessment, for example: From the model's predictions of unlabeled data, high-confidence and high-accuracy predictions are selected and transformed into high-quality pseudo-labels to drive the model's self-supervised training.

[0062] Pseudo-tag generation: (12) in, , The maximum IoU between the i-th region and all known category true bounding boxes.

[0063] Prediction results with high confidence and high positioning accuracy are selected as reliable pseudo-labels. The quality of each candidate pseudo-label is quantified using the following formula: (13) in, Let i be the maximum confidence level for region i to be predicted as any known class. Let be the confidence level that region i is predicted to be of an unknown category. This is the intersection-union ratio between the predicted bounding box and its best-matching ground truth bounding box. This ensures that the highest classification confidence score is used as the quality criterion regardless of whether the model classifies a region as known or unknown. It is used to adjust the level of attention given to difficult negative samples. The definition is as follows: (14) (15) in, This is the original similarity score between the i-th region and the j-th known category.

[0064] Final screening of pseudo-tags: (16) in, This is a quality score threshold for filtering pseudo-labels; only predictions above this threshold are considered reliable pseudo-labels.

[0065] Step S34: Overall loss function.

[0066] Step S341: Basic IoU Loss (17) in, The bounding box predicted by the model. The bounding box is the actual labeled area in the image. This indicates the calculation of the area of ​​the corresponding region. IoU ∈ [0,1] is the Intersection over Union (IoU); the larger the value, the higher the overlap between the two bounding boxes, and the more accurate the localization. This loss function, by directly optimizing the IoU metric, can effectively guide the model to generate predicted bounding boxes that better match the true location and size of the target. Its value range is... ∈[0,1], the loss is 0 when the two boxes completely overlap; the loss is 1 when the two boxes do not overlap. This loss is the basic component of subsequent more advanced localization losses.

[0067] Step S342: Overall Loss Function (18) in, For the binary cross-entropy loss of known categories, For classification loss of unknown categories, To account for the bounding box regression loss based on overlapping area, center point distance, and aspect ratio, is the weighting coefficient for bounding box regression loss, used to balance classification loss and localization loss. Wildcards for region features and general objects Similarity score, Represents the maximum IoU between the region and the object. It is the model's predicted score vector for known categories. It is a one-hot encoded supervisory signal of a known category. Gating function. The value is based on the objectivity score of the region. and location matching degree Dynamically output weights are used to adaptively adjust unknown loss terms. Contribution to the total loss.

[0068] Step S35: Dynamic threshold scheduling, the formula is as follows: (19) This is a dynamic threshold that adapts to change over time. Initially, a high threshold is used for strict filtering to promote stable model convergence; later, the threshold is gradually relaxed to identify difficult samples and improve the model's generalization ability. As the initial threshold, Here, t is the decay coefficient, and t is the current training step number. This is the preset total number of training steps.

[0069] In some implementations, the embodiments of this application can be used to achieve target detection in radar images, including: extracting multi-scale features using an improved YOLO detector; multi-scale feature fusion network; open target classification; unknown target filtering; dynamic threshold adaptive adjustment; post-processing of detection results; new class discovery and vocabulary expansion.

[0070] Based on step S3 in the embodiment, this application may further include step S4.

[0071] Step S4: Target detection.

[0072] Step S41: Radar image feature extraction, for example: Multi-scale features are extracted using an improved YOLO detector: (20) in, Let H be the feature map of layer I, with dimension (H). l W l , C), where H l and W l C represents the spatial dimensions of the feature map, and C represents the number of channels.

[0073] Step S42: Multi-scale feature fusion network, including at least: feeding localization information back to deep features through upsampling and downsampling paths; feature fusion output, using a weighted fusion strategy for decision integration; updating text embedding through image pooling attention mechanism; text-guided feature enhancement, achieving feature modulation through cross-modal interaction.

[0074] Step S421: Upsampling and downsampling paths: Injecting high-level semantic information layer by layer into low-level features: (twenty one) in, This is the original feature map of layer l. For higher-level feature maps, For upsampling operation, A 1×1 convolution is used for feature dimension alignment.

[0075] Construct a bottom-up enhancement path to feed back fine-grained localization information to deep features: (twenty two) in, The current layer features are output by the FPN path. This is a feature at a lower level in the PAN path. This is a downsampling operation. After the bidirectional path fusion described above, the resulting multi-scale enhanced feature maps will be fed into the subsequent detection head to simultaneously complete the target bounding box regression and open-world classification tasks.

[0076] Step S422: Feature fusion output, for example: The fused features are efficiently extracted and enhanced through a cross-stage partial connection layer, and the output is represented as follows: (twenty three) in, As a cross-stage partially connected layer, a balance between computational efficiency and feature representation capability is achieved by dividing the feature map into two parts and using different processing paths. The refined multi-scale features are then fed into the detection head to generate independent detection results at each scale.

[0077] To integrate multi-scale information and improve the robustness of the final detection, a weighted fusion strategy is adopted to integrate the detection results obtained at three scales (s=3). The fusion formula is as follows: (twenty four) in, For the first s The detection results at each scale typically include a class confidence vector and bounding box coordinates; Let be the fusion weight assigned to the s-th scale, and its value can be a preset fixed value or a learnable parameter.

[0078] Step S423: Image pooling attention mechanism, for example, combined with Figure 2 The following is a detailed explanation of the pooling part in this embodiment, designing image pooling attention update text embedding: (25) (26) in, W This is a matrix formed by concatenating the embedding vectors of all known category wildcards with the wildcards of unknown categories. The feature is a 3×3×3 region feature obtained by max pooling multi-scale features. This is a standard multi-head attention function. This operation enables text embeddings to adaptively adjust based on the overall content of the image. Max pooling is a function that extracts the most significant feature responses from the feature map and reduces dimensionality, which helps to enhance the model's robustness to small changes in the target location.

[0079] Step S424: Text-guided feature enhancement, such as combining Figure 3 The text guidance section of this embodiment is explained in detail. It utilizes text semantic information to dynamically enhance the regions in visual features related to the target, and achieves the following feature modulation through cross-modal interaction: First, calculate the feature map. The feature vector of each spatial location and the text embedding of all categories The maximum similarity is calculated, and a spatial attention map is generated using an activation function. : (27) Subsequently, the attention map is broadcast along the channel dimension, so that it is integrated with the feature map. Size matching, and element-wise multiplication: (28) in, F l The feature map to be enhanced is the result of multi-scale fusion. This is a text embedding matrix formed by concatenating wildcard embeddings of known categories and wildcard embeddings of unknown categories. For cosine similarity and other measurement functions, The activation function compresses the similarity to the [0, 1] interval as the attention weight. The operation selects the text information most relevant to the current feature. This represents a two-dimensional attention map. Copy along the channel dimension to make it the same as... Size (H) l W l , C) Matching.

[0080] Step S43: Open-world target classification, including at least: integrating visual-text alignment and IoU to achieve multimodal two-head matching; and performing open classification for detected targets.

[0081] Step S431: Multimodal two-head matching, for example: A multimodal dual-head matching mechanism is designed, considering both visual-text alignment and IoU. When the object IoU is low, text semantic matching is relied upon; when the text description is ambiguous, high IoU is relied upon. The formula is as follows: (29) Here, λ and γ are hyperparameters used to adjust the relative importance of semantic similarity and localization accuracy in the final matching score, respectively. When λ > γ, the model relies more on semantic information; when γ > λ, the model relies more on localization accuracy. To represent the localization accuracy (intersection over union ratio) of the i-th matching pair, the object-text similarity s is calculated using a stable method: (30) in, This represents the feature embedding of the i-th region. Indicates the first j Text embeddings of each category, , These are the learnable scaling factor and offset factor, typically scalars initialized to 1 and 0, respectively, and are optimized during model training using gradient descent. Their role is to adaptively adjust and stabilize the numerical distribution of cosine similarity, thereby improving the stability and convergence speed of model training. for L 2. Normalization. <.,> represents the vector dot product. L 2. The dot product after normalization is the cosine similarity.

[0082] Step S432: Open-world classification decision, for example, referencing Figure 5 This embodiment illustrates the open-world classification decision-making process, where detected targets are classified in the open world: (31) in, This is the confidence threshold for the known class; if the confidence level is higher than this value, the class is considered known. This is the confidence threshold for the unknown category; values ​​above this threshold are considered when the known class similarity is below a certain threshold. Then it is judged as unknown. To detect the maximum similarity between the target and a known category, The similarity between the target and the unknown wildcard.

[0083] Step S44: Unknown target filtering strategy, for example: During training, to avoid conflicts between predictions of unknown categories and predictions of known categories, an IoU filtering strategy is used to remove duplicate predictions: (32) in, For a known category prediction box, the confidence score is higher than a preset high threshold. The set, and Then it means A specific known class prediction box in the dataset. Used to adjust the level of attention given to difficult negative samples. It is a hyperparameter, usually set to a high value (e.g., 0.7), to ensure that the predicted bounding boxes in the ensemble are highly reliable known object detection results. For prediction boxes of unknown categories, the confidence score is higher than a preset low threshold. The set, and Then it means A specific known category prediction box in the [database / framework]. It is a hyperparameter, usually set to a low value (e.g., 0.01), used to filter out most of the background noise while retaining the prediction of potential unknown targets. =0.99 is the IoU threshold. When the predicted bounding box is unknown... With any known reliable prediction box When the IoU is higher than this threshold, It will be filtered. This is the set of predicted bounding boxes for unknown categories that are retained after filtering and do not overlap with known targets. This strategy ensures that when generating pseudo-labels for unknown categories, conflicts are avoided with high-confidence predictions for known categories, thus improving the quality of training data.

[0084] Step S45: Dynamic threshold adaptive adjustment, for example: To address the issue of overdetection or underdetection caused by changes in data distribution in open-world scenarios when using a fixed threshold, a dynamic threshold adaptive adjustment mechanism is proposed. This mechanism fine-tunes the base threshold in real time based on the confidence statistic of the current training batch, enabling the detection system to adapt to different environments. The adjustment formula is as follows: (33) in, Based on the threshold, The standard deviation of the confidence scores for all preset boxes in the current batch reflects the degree of disagreement in the model's judgments of this batch of data. The more dispersed the confidence score distribution, the better. The larger. This is the average confidence level of all preset boxes in the previous batch, reflecting the overall confidence level of the model's judgment on the current batch of data. and The adjustment coefficient is a hyperparameter that controls the weighting of the standard deviation and mean on the threshold. For example, it can be set to... Setting it to a negative value ensures that the threshold will be conservative when model predictions show significant discrepancies, preventing aggressive judgments under uncertainty; A positive value is used to ensure that the threshold increases appropriately as the overall confidence level increases. The calculated adaptive threshold will be used for classification decisions in the current batch.

[0085] Step S46: Post-processing of detection results, including at least: performing non-maximum suppression and confidence filtering on the detection results, removing low-quality detection boxes based on dynamic thresholds; and adopting a cross-class suppression strategy to avoid overlap of known / unknown boxes.

[0086] Non-maximum suppression and confidence filtering are applied to the detection results, based on a dynamic threshold. Eliminate low-quality test frames: (34) in, This is the original detection bounding box, which includes location, category, and confidence information. For detection box The confidence level is usually the maximum value in the category confidence vector. This is the dynamic confidence threshold.

[0087] A cross-class suppression strategy is employed to avoid overlap of known / unknown bounding boxes. (35) D represents the initial set of detection boxes to be processed. and For different detection boxes in set D. Represents the detection box The area of ​​the intersection region. Represents the detection box The area of ​​the union region. The ratio of the two is the intersection-union ratio (IoU), which measures the degree of spatial overlap between the two boxes. This is the IoU threshold. When the IoU between two boxes is below this threshold, they are considered to be two independent targets. and are the detection boxes, respectively. and The confidence score. > The suppression condition is: when the heights of the two bounding boxes overlap (IoU ≥ 1000). When retaining a box with higher confidence, keep the box with higher confidence. Suppressing boxes with low confidence .

[0088] Step S47: New class discovery and vocabulary expansion, including at least: unknown candidate aggregation; regional feature clustering; semantic annotation.

[0089] Step S471: Unknown candidate convergence, for example: During the model's inference process on unlabeled data, predictions that the model classifies as "unknown" with high confidence are collected to form a candidate set. Its mathematical definition is: (36) in, Let be the confidence probability that the i-th predicted box belongs to the unknown category. This is the confidence threshold for unknown categories, used to filter out unknown candidates with high confidence levels, ensuring the quality of the candidate set.

[0090] Step S472: Clustering of region features, for example: First, the regional characteristics are analyzed. L 2. Normalization, and use cosine distance as the similarity measure: (37) Then, a clustering algorithm was used to cluster the set U to obtain the cluster set. For each cluster Calculate its centroid (average feature) and average confidence level: (38) To ensure the quality of newly discovered categories, the clustering results are screened, and only clusters that simultaneously meet the following two conditions are considered reliable candidates for new categories: Scale requirements: >n min This ensures that there are sufficient samples to support the statistical significance of the class.

[0091] Confidence level conditions: >γ c This ensures that the overall confidence level of this category is high.

[0092] Step S473: Semantic annotation, for example: Automatic matching of candidate word sets L : (39) If there are no highly similar matches (below the threshold) The clusters were retained as "unnamed clusters" and manually labeled before being backfilled.

[0093] In some specific implementations, this application provides a radar image open set target detection system based on wildcard learning, including one or more processing modules configured to perform specific implementations of steps S1 to S4 described above.

[0094] The advantages of the embodiments of this disclosure compared to the prior art are at least reflected in the following aspects: The radar image target detection method and system based on wildcard learning and open set discrimination overcome the limitations of traditional closed set detection. It achieves proactive identification and labeling of unknown targets through a wildcard learning mechanism, avoiding dangerous missed detections; it employs a text embedding freezing strategy to effectively solve the catastrophic forgetting problem, ensuring the system's continuous learning capability; and it supports flexible vocabulary expansion and offline deployment, reducing system complexity and operating costs. This disclosure provides a complete and efficient open-world target detection solution for radar image target detection.

[0095] This application also provides a computer-readable storage medium storing computer-executable instructions thereon. When executed by a processor, the computer-executable instructions mainly implement the above-described radar image open set target detection method based on wildcard learning, wherein the configuration steps include at least the following: Obtain the dataset and construct a training dataset that supports the detection of both known and unknown targets; A wildcard learning strategy is adopted to drive the model in a self-supervised manner, learning the discriminative features of known targets and the general representation of unknown targets; By utilizing an improved YOLO detector for multi-scale feature extraction and fusion, and through text-guided feature enhancement, open classification decision and adaptive confidence optimization, target detection in radar images is achieved.

[0096] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A target detection method for open sets of radar images based on wildcard learning, including: Obtain the dataset and construct a training dataset that supports the detection of both known and unknown targets; A wildcard learning strategy is adopted to drive the model in a self-supervised manner, learning the discriminative features of known targets and the general representation of unknown targets; By utilizing an improved YOLO detector for multi-scale feature extraction and fusion, and through text-guided feature enhancement, open classification decision and adaptive confidence optimization, target detection in radar images is achieved.

2. The method according to claim 1, wherein, Before performing target detection in the radar image, the method further includes: Multimodal data fusion is performed based on radar image data acquisition. Based on open data partitioning methods, dataset format conversion and partitioning are performed.

3. The method according to claim 2, wherein, Based on radar image data acquisition, multimodal data fusion is performed, including: Radar image data acquisition; Target category definition and labeling; Multimodal training samples were constructed by combining radar images and text descriptions; Radar image preprocessing.

4. The method according to claim 3, wherein, Based on open data partitioning methods, dataset format conversion and partitioning are performed, including: Region-text pair construction; Wildcard embedding initialization; Datasets are partitioned based on samples containing known categories and mixed samples containing known and unknown categories.

5. The method according to claim 4, wherein, Model training methods include: Adaptive decision learning; Based on the wildcard learning strategy, we can train wildcards with known targets and learn wildcards with unknown targets. Region-text contrast loss is used to filter out predictions with high confidence and high positioning accuracy as reliable pseudo-labels; Obtain the overall loss function; Dynamic threshold scheduling is achieved by using a dynamic threshold that adapts to changes over time.

6. The method according to claim 5, wherein, To achieve target detection in radar images, including: Multi-scale features are extracted using an improved YOLO detector; Multi-scale feature fusion network; Open target classification; Unknown target filtering; Dynamic threshold adaptive adjustment; Post-processing of test results; New class discovery and vocabulary expansion.

7. The method according to claim 6, wherein, Multi-scale feature fusion networks, including at least: The positioning information is fed back to the deep features through upsampling and downsampling paths; The feature fusion output is used for decision integration using a weighted fusion strategy. The text embedding is updated using an image pooling attention mechanism; Text-guided feature enhancement achieves feature modulation through cross-modal interaction.

8. The method according to claim 7, wherein, Open target classification, including at least: By combining visual-text alignment and IoU, multimodal two-head matching is achieved; For the detected targets, perform open classification.

9. The method according to claim 8, wherein, Post-processing of test results includes: Non-maximum suppression and confidence filtering are applied to the detection results, and low-quality detection boxes are removed based on dynamic thresholds; A cross-category suppression strategy is adopted to avoid overlap of known / unknown boxes.

10. The method according to claim 9, wherein, New class discovery and vocabulary expansion include: Unknown candidate convergence; Clustering based on regional characteristics; Semantic annotation.

Images