Target detection methods, devices, electronic equipment and storage media

Abstract

The target detection method, apparatus, electronic device, and storage medium proposed in this application include: acquiring detection features of a detection image, inputting the detection features into a feature filtering submodule of a target detection model for feature filtering to obtain multiple query features, inputting the query features into a first feedforward network of the target detection model for query detection to obtain an initial detection result, inputting the multiple query features into a localization submodule for detection and localization to obtain a first corrected query feature, inputting the first corrected query feature into a deduplication submodule for feature processing to obtain a second corrected query feature, and obtaining a target detection result of the detection image based on the initial detection result, the first corrected bias result corresponding to the first corrected query feature, and the second corrected bias result corresponding to the second corrected query feature. This achieves adaptive target detection and alleviates the ambiguity of the detected target, thereby improving the flexibility and accuracy of target detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to target detection methods, apparatus, electronic devices and storage media. Background Technology

[0002] Object detection and image recognition are among the most important research areas in computer vision. Object detection refers to automatically detecting objects of a specific category in a given image and accurately locating their positions. Image recognition refers to classifying or labeling a given input image, mapping the image to a fixed category or label.

[0003] Among related technologies, compared to convolutional neural network-based detectors, transformer-based detectors (such as DETR and DINO) simplify the detection process, achieve end-to-end object detection, and achieve higher detection accuracy, making them the current mainstream deep learning detection models. However, transformer-based detectors detect a fixed number of predicted targets, which greatly limits the flexibility of the detection model. Summary of the Invention

[0004] This application provides a target detection method, apparatus, electronic device, and storage medium, which can improve the flexibility and applicability of the detection model.

[0005] To achieve the above objectives, a first aspect of this application proposes a target detection method, the method comprising:

[0006] The detection features of the detected image are obtained, and the detection features are input into the feature filtering submodule of the target detection model for feature filtering to obtain multiple query features;

[0007] The query features are input into the first feedforward network of the target detection model for query detection to obtain initial detection results;

[0008] Multiple query features are input into the localization submodule of the target detection model for detection and localization to obtain a first corrected query feature. The first corrected query feature is then input into the second feedforward network of the target detection model for query detection to obtain a first corrected bias result.

[0009] The first corrected query feature is input into the deduplication submodule of the target detection model for feature processing to obtain the second corrected query feature. The second corrected query feature is then input into the third feedforward network of the target detection model for query detection to obtain the second corrected bias result.

[0010] The target detection result of the detection image is obtained based on the initial detection result, the first corrected bias result, and the second corrected bias result.

[0011] In some embodiments, the detection features include multiple detection feature units, and the feature filtering submodule of the target detection model is used to filter the detection features to obtain multiple query features, including:

[0012] The multiple detection feature units are classified and identified to obtain the probability value of the classification result related to the predicted category;

[0013] From multiple detection feature units, select the detection feature units whose classification result probability value exceeds the classification score threshold as candidate detection feature units;

[0014] The classification result probability values ​​of all the candidate detection feature units are sorted by size, and at least one detection feature unit is selected from the candidate detection feature units as the query feature based on the sorting result.

[0015] In some embodiments, the feature filtering submodule includes a query feature capacity, and the step of selecting at least one detection feature unit from the candidate detection feature units as the query feature based on the ranking result includes:

[0016] Determine the number of feature units of the detection feature unit among the candidate detection feature units;

[0017] When the number of candidate units is less than the query feature capacity, all the detection feature units in the candidate detection feature units are used as the query feature;

[0018] When the number of candidate units is greater than the query feature capacity, a detection feature unit that matches the query feature capacity is selected from the candidate detection feature units according to the sorting of all the detection feature units in the candidate detection feature units, and is used as the query feature.

[0019] In some embodiments, the localization module includes a first cross-attention submodule, a first normalization submodule, a second normalization submodule, and a first multilayer perception submodule. The step of inputting multiple query features into the localization submodule of the target detection model for detection and localization to obtain a first corrected query feature includes:

[0020] The query features and the detection features are input into the first cross-attention submodule for data association to obtain the first localization detection features;

[0021] The first location detection feature and the query feature are input into the first standardization submodule for superposition processing to obtain the first location superposition feature;

[0022] The first positioning overlay feature is input into the first multi-layer perception submodule for information interaction to obtain the positioning interaction feature;

[0023] The location interaction feature and the first location overlay feature are input into the second standardization submodule for overlay processing to obtain the first corrected query feature.

[0024] In some embodiments, the deduplication module includes a second cross-attention submodule, a self-attention submodule, a third normalization submodule, a fourth normalization submodule, and a second multilayer perceptron submodule. The step of inputting the first corrected query feature into the deduplication submodule of the target detection model for feature processing to obtain the second corrected query feature includes:

[0025] The first modified query feature and the detection feature are input into the second cross-attention submodule for data association to obtain the second localization detection feature;

[0026] The second positioning detection feature and the first corrected query feature are input into the third standardization submodule for superposition processing to obtain the second positioning superposition feature;

[0027] The second positioning overlay feature is input into the self-attention submodule for data integration to obtain the positioning integrated feature;

[0028] The integrated positioning feature and the second overlay positioning feature are input into the third standardization submodule for overlay processing to obtain the integrated standard feature;

[0029] The integrated standard features are input into the second multi-layer perception submodule for information interaction to obtain the second corrected query features.

[0030] In some embodiments, the training steps of the object detection model include:

[0031] Obtain training image samples and corresponding test image samples;

[0032] The second image sample is input into the target detection model for data detection to obtain training detection results and training detection classification scores.

[0033] Based on the intersection-over-union ratio (IoU) between the training detection results and the test image samples, and the training detection classification score, sample labels are generated;

[0034] The training loss value is obtained based on the sample label and the training detection classification score, and the model parameters of the object detection model are updated based on the training loss value.

[0035] In some embodiments, obtaining the training loss value based on the sample label and the training detection classification score includes:

[0036] Based on the binary cross-entropy loss function, the first cross-entropy loss value of the sample label and the training detection classification score is obtained, and the second cross-entropy loss value of the training detection classification score is obtained.

[0037] Obtain the training detection target size of the training detection result, and obtain the training image size of the training image sample;

[0038] The bounding box parameter values ​​are obtained based on the ratio of the size of the training detection target to the size of the training image, and by performing an exponential operation based on the scaling hyperparameter.

[0039] The training loss value is obtained by summing the first cross-entropy loss value, the bounding box parameter value, and the second cross-entropy loss value.

[0040] To achieve the above objectives, a second aspect of this application provides a target detection device, the module of which includes:

[0041] The query feature acquisition module acquires the detection features of the detected image and inputs the detection features into the feature filtering submodule of the target detection model for feature filtering to obtain multiple query features;

[0042] The initial detection module is used to input the query features into the first feedforward network of the target detection model for query detection and obtain the initial detection result;

[0043] The first feature correction module is used to input multiple query features into the localization submodule of the target detection model for detection and localization to obtain a first corrected query feature, and input the first corrected query feature into the second feedforward network of the target detection model for query detection to obtain a first corrected bias result;

[0044] The second feature correction module inputs the first corrected query feature into the deduplication submodule of the target detection model for feature processing to obtain the second corrected query feature, and inputs the second corrected query feature into the third feedforward network of the target detection model for query detection to obtain the second corrected bias result.

[0045] The target detection result module is used to obtain the target detection result of the detection image based on the initial detection result, the first corrected bias result, and the second corrected bias result.

[0046] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the target detection method as described in the first aspect.

[0047] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the target detection method described in the first aspect.

[0048] The target detection method, apparatus, electronic device, and storage medium proposed in this application include: First, acquiring detection features of the detection image and inputting the detection features into a feature filtering submodule of a target detection model for feature filtering to obtain multiple query features; then, inputting the query features into a first feedforward network of the target detection model for query detection to obtain an initial detection result; next, inputting the multiple query features into a localization submodule of the target detection model for detection and localization to obtain a first corrected query feature, and inputting the first corrected query feature into a second feedforward network of the target detection model for query detection to obtain a first corrected bias result; then, inputting the first corrected query feature into a deduplication submodule of the target detection model for feature processing to obtain a second corrected query feature, and inputting the second corrected query feature into a third feedforward network of the target detection model for query detection to obtain a second corrected bias result; finally, obtaining the target detection result of the detection image based on the initial detection result, the first corrected bias result, and the second corrected bias result. This application embodiment utilizes a feature filtering submodule to filter the detection features of the detection image, adaptively selecting an appropriate number of query features, thereby obtaining an appropriate number of detection results using these appropriate query features, thus improving the flexibility and adaptability of target detection. In addition, the query features are sequentially input into the localization submodule for detection and localization and into the deduplication submodule for feature processing, so as to correct the query features multiple times, thereby effectively improving the accuracy of the target detection results of the obtained detection image.

[0049] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the structure of a target detection module provided in an embodiment of this application.

[0051] Figure 2This is a flowchart of a target detection method provided in another embodiment of this application.

[0052] Figure 3 yes Figure 2 The flowchart for step 201.

[0053] Figure 4 yes Figure 3 The flowchart for step 303.

[0054] Figure 5 This is a schematic diagram of an adaptive query feature filtering algorithm provided in another embodiment of this application.

[0055] Figure 6 yes Figure 2 The flowchart for step 203.

[0056] Figure 7 yes Figure 2 The flowchart for step 204.

[0057] Figure 8 This is a schematic diagram of a target detection process for an image, provided in another embodiment of this application.

[0058] Figure 9 This is a flowchart of the training process for an object detection model provided in another embodiment of this application.

[0059] Figure 10 yes Figure 9 The flowchart for step 904.

[0060] Figure 11 This is a performance simulation diagram of the first target detection method provided in another embodiment of this application.

[0061] Figure 12 This is a performance simulation diagram of the first target detection method provided in another embodiment of this application.

[0062] Figure 13 This is a performance simulation diagram of the second target detection method provided in another embodiment of this application.

[0063] Figure 14 This is a performance simulation diagram of the third target detection method provided in another embodiment of this application.

[0064] Figure 15 This is a performance simulation diagram of the fourth target detection method provided in another embodiment of this application.

[0065] Figure 16 This is a performance simulation diagram of a second target detection method provided in another embodiment of this application.

[0066] Figure 17This is a schematic diagram of the target detection device provided in another embodiment of this application.

[0067] Figure 18 This is a schematic diagram of the hardware structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0071] First, let's analyze some of the terms used in this application:

[0072] Transformer-based detectors are computer vision models that utilize the Transformer architecture to perform object detection tasks. This type of detector improves detection speed and efficiency without sacrificing accuracy by combining traditional convolutional neural networks (CNNs) with self-attention mechanisms. These detectors typically employ an encoder-decoder structure, where the encoder extracts features from the image, while the decoder predicts the location and class of the target. Due to their parallel processing capabilities and efficient capture of long-range dependencies, Transformer-based detectors demonstrate superior performance in multi-object detection in complex scenes.

[0073] The softmax function is a commonly used mathematical function, particularly prevalent in machine learning and deep learning, especially as an activation function for the output layer in classification tasks. Its main function is to transform a K-dimensional vector of true numbers into another K-dimensional vector of probability distributions, where each element is a probability value in the interval [0,1], and the sum of all elements is 1.

[0074] DETR (DEtection Transformer) is an object detection model based on the Transformer architecture. It changes the traditional object detection method based on convolutional neural networks (CNNs) by introducing an end-to-end Transformer model to handle object detection tasks. DETR-like models refer to models with similar design concepts and technical architectures to DETR, which typically use the Transformer as the main component to perform object detection tasks.

[0075] Intersection over Union (IoU) is a metric used in computer vision and image processing to evaluate the degree of overlap between two bounding boxes. IoU is primarily used in object detection tasks to measure the degree of matching between predicted bounding boxes and ground truth bounding boxes.

[0076] Binary Cross Entropy (BCE) is a commonly used loss function, especially suitable for binary classification problems. It measures the difference between the probability distribution predicted by the model and the actual labels. In machine learning and deep learning, loss functions are key components used to guide the direction of parameter updates during model training, and BCE is particularly suitable for problems where the output is a binary label (usually 0 or 1).

[0077] Object detection and image recognition are among the most important research areas in computer vision. Object detection refers to automatically detecting objects of a specific category in a given image and accurately locating their positions. Image recognition refers to classifying or labeling a given input image, mapping the image to a fixed category or label. Object detection and image recognition are closely related and complementary fundamental research tasks in computer vision, with wide applications in fields such as robot navigation, autonomous driving, intelligent monitoring, human-computer interaction, and image retrieval. Currently, the main object detection and recognition methods include traditional image processing methods and deep learning-based methods.

[0078] In related technologies, with the continuous development of computer technology and data processing technology, deep learning models have become the mainstream method and research hotspot in computer vision tasks. Compared with detectors based on convolutional neural networks, transformer-based detectors (such as DETR, DINO, etc.) simplify the detection process, achieve end-to-end object detection, and achieve higher detection accuracy, making them the current mainstream deep learning detection models. However, transformer-based detectors detect a fixed number of predicted targets, which greatly limits the flexibility of the detection model.

[0079] To improve the flexibility and applicability of the detection model, this embodiment utilizes a feature filtering submodule to filter the detection features of the detection image, adaptively selecting an appropriate number of query features. This allows for the acquisition of an appropriate number of detection results using these appropriate query features, thereby enhancing the flexibility and adaptability of target detection. Furthermore, the query features are sequentially input into the localization submodule for detection and localization, and then into the deduplication submodule for feature processing. This process refines the query features multiple times, effectively improving the accuracy of the target detection results obtained from the detection image.

[0080] To better describe the target detection method provided in this application, the target detection module applied to the target detection method is described first below. (Refer to...) Figure 1 This is a schematic diagram of the structure of a target detection module provided in an embodiment of this application. Figure 1 As shown, the target detection module is used to perform feature detection and recognition on the detection image. The target detection module includes a backbone network, a Transformer encoder, a feature selection submodule, a localization submodule, a deduplication submodule, and a feed-forward network (FFN) submodule.

[0081] The backbone network (i.e., the Backbone model) is used to perform preliminary feature extraction and representation on the input detection image. It can be any basic model related to vision processing, such as CNN model, transformer model, MLP model, vision state space model (Vision Mamba), etc.

[0082] The Transformer Encoder is used to enhance the features initially extracted by the backbone network. Then, the enhanced features are sequentially input into the feature filtering submodule, the localization submodule, the deduplication submodule, and the FFN submodule for relevant data processing to obtain the target detection results of the detected image, as described in the following steps.

[0083] Based on the target detection module described above, the target detection method, apparatus, electronic device, and storage medium provided in the embodiments of this application will be further described below. The target detection method provided in the embodiments of this application is applied to the target detection module or a controller connected to the target detection module.

[0084] The target detection method in the embodiments of this application will be described in detail below. (Refer to...) Figure 2 This is an optional flowchart of the target detection method provided in the embodiments of this application. Figure 2 The method may include, but is not limited to, steps 201 to 205. It is also understood that this embodiment... Figure 2 The order of steps 201 to 205 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0085] Step 201: Obtain the detection features of the detected image and input the detection features into the feature filtering submodule of the target detection model for feature filtering to obtain multiple query features.

[0086] Step 201 will be described in detail below.

[0087] In some embodiments, the object detection model responds to a detection image I∈R with height H and width W. H×W×3 After receiving the detection and recognition requests, the detection image is input into the backbone network to perform preliminary feature extraction to obtain the initial detection features of the detection image. Then, the initial detection features are input into the Transformer encoder, and through the attention operation in the Transformer encoder, the targets corresponding to multiple initial detection feature units in the initial detection features are separated and the features are enhanced to obtain the enhanced detection features.

[0088] It is understandable that a Transformer encoder typically includes an embedding layer component, a multi-head self-attention component, and so on. The initial detection features are then enhanced through the embedding layer and the multi-head self-attention component, as shown in the following example.

[0089] In the Transformer encoder, the embedding layer transforms each initial detection feature unit in the initial detection features into a fixed-length vector representation, which is then transformed into a corresponding initial query vector, key, and value. The similarity score between each query vector and all keys is then calculated using methods such as dot product or additive attention. The attention weights are then obtained by applying the softmax function. Next, the value vectors are weighted and summed according to the attention weights to obtain the enhanced feature representation of each initial detection feature unit, resulting in an enhanced detection feature comprising multiple detection feature units.

[0090] In related technologies, the transformer encoder-decoder architecture treats object detection as a set prediction problem. This type of model predicts objects based on a fixed-size set of N learnable object query vectors, with each query predicting a single object (box and class). These predicted objects are matched one-to-one with each human-labeled ground truth (GT) object; that is, each ground truth box matches the detection result of one query. However, this approach introduces a significant problem: the detection model can only detect a "limited number of objects." This is because the model's weights are influenced by the predefined, fixed-size N query parameters, limiting the model to predicting a maximum of N objects. It cannot handle input images containing more than N feature objects, severely restricting the model's flexibility and applicability.

[0091] Current DETK-like models can only predict a fixed number of detection results, mainly due to the limitation of a fixed number of query features. DETR models, such as DAB-DETR, DINO, and Stable-DINO, mainly include two types of queries: content query (CQ) and positional query (PQ). Content queries are statically initialized by the model into fixed-dimensional vectors [N,D], where N represents the number of queries and D represents the channel dimension of the query. Content queries are used to predict the bias and classification information of bounding boxes, obtained by inputting the query into an FFN network layer. Different queries should predict detection boxes for different targets; therefore, to increase the difference between queries, current methods have proposed positional queries. The dimension of positional queries is the same as that of CQs, and the difference between CQs is increased by directly adding PQs to CQs. Both types of queries require the model to predefine their number N, which greatly limits the model's detection capacity. Therefore, this invention, through experimental verification, found that PQs have little impact on the model's detection performance, with detection accuracy fluctuating within ±0.3% AP. Considering the requirements of free-quantity target detection, this application omits the PQ part.

[0092] In this embodiment, a novel self-adaptive decoder query (SADQ) is proposed to replace the original CQ part. This process is designed as a feature filtering submodule of the target detection model. This feature filtering submodule is used to filter the detection features of the detected image to adaptively select an appropriate number of query features to overcome the problem that "the detection model can only detect a limited number of targets". This will be described in further detail below.

[0093] Reference Figure 3 The detection features are input into the feature filtering submodule of the target detection model for feature filtering to obtain multiple query features, including the following steps 301 to 303.

[0094] Step 301: Classify and identify multiple detection feature units to obtain the probability value of the classification result related to the predicted category.

[0095] Step 302: Select detection feature units from multiple detection feature units whose classification result probability value exceeds the classification score threshold as candidate detection feature units.

[0096] Step 303: Sort the classification result probability values ​​of all candidate detection feature units by size, and select at least one detection feature unit from the candidate detection feature units as the query feature according to the sorting result.

[0097] Steps 301 to 303 are described in detail below.

[0098] In some embodiments, multiple detection feature units in detection feature E are classified and identified according to the predicted category C to obtain the probability value of the classification result related to the detection feature and the predicted category C. Where N is the number of detection feature units.

[0099] Then, obtain the pre-set classification score threshold S, and select the detection feature units whose classification result probability value exceeds the classification score threshold from multiple detection feature units as candidate detection feature units T. idx :(B,C) is shown in the following formula (1).

[0100]

[0101] Next, the classification probability values ​​of all candidate detection feature units are sorted by size. Based on the sorting results, at least one detection feature unit is selected from the candidate detection feature units as an appropriate number of query features. This will be described in further detail below.

[0102] Reference Figure 4 The process involves selecting at least one detection feature unit from the candidate detection feature units as a query feature based on the sorting results, including the following steps 401 to 403.

[0103] Step 401: Determine the number of feature units among the candidate detection feature units.

[0104] Step 402: When the number of candidate units is less than the query feature capacity, all detection feature units in the candidate detection feature units are used as query features.

[0105] Step 403: When the number of candidate units is greater than the query feature capacity, select a detection feature unit that matches the query feature capacity from the candidate detection feature units according to the sorting of all detection feature units in the candidate detection feature units, and use it as the query feature.

[0106] Steps 401 to 403 are described in detail below.

[0107] In some embodiments, the query feature capacity P of the feature filtering submodule is determined, and the candidate detection feature units T are determined. idx The number of feature units M in (B,C) that detect feature units.

[0108] When the number of candidate units is less than or equal to the query feature capacity, i.e., M≤P, all the candidate detection feature units are used as query features SADQ:(B,M,D) as shown in the following formula (2).

[0109] SADQ:(B,M,D)←index(E,T idx (2)

[0110] When the number of candidate units is greater than the query feature capacity, i.e. M>P, according to the sorting of all the candidate detection feature units, the detection feature unit that matches the query feature capacity is selected from the candidate detection feature units as the query feature SADQ:(B,M,D), as shown in the following formula (3).

[0111]

[0112] Reference Figure 5 This is a schematic diagram of an adaptive query feature filtering algorithm provided in an embodiment of this application. For example... Figure 5 As shown, the query feature capacity P and classification score threshold S are generated. Then, the input detection features E:(B,N,D) are processed by the FFN module to obtain the probability values ​​of the classification results related to the predicted category C. Next, the probability values ​​of the classification results will be... Detection feature units exceeding the classification score threshold S are selected as candidate detection feature units T. idx :(B,C), and based on the candidate detection feature unit T idx The number of features in (B,C) is compared with the query feature capacity P to further filter and obtain suitable query features SADQ:(B,M,D). It is understood that the query feature capacity P can be randomly generated, or it can be customized according to user needs, such as 1000, 2000, etc., or it can be set according to device parameters; and the higher the value of the query feature capacity P, the more detection feature units can be obtained, resulting in more accurate detection for the image.

[0113] Furthermore, during the model training phase, this application constructs a query cache pool with a capacity of P. When the number of queries M selected by a threshold exceeds the cache pool capacity P, the TopK method is used to select P queries from the M queries to remove redundant queries. In the early stages of model training, the classification scores obtained by the encoder's tokens are randomly distributed, which leads to a large number of redundant queries being selected. As training progresses, the classification scores of the selected queries increase under the influence of classification supervision information, while the classification scores of the unselected queries gradually decrease, thereby gradually removing redundant queries and achieving adaptive query selection.

[0114] Through steps 301 to 303 and steps 401 to 403, the detection feature units are initially filtered and screened using the classification result probability value and classification score of each unit in the detection feature to filter out less meaningful detection feature units. Next, the query feature capacity is further matched with the number of feature units of the initially filtered candidate detection feature units, and the detection feature units are filtered and screened again according to the matching results, so as to select feature units with high classification result probability values ​​as the adaptive number of query features, so as to effectively improve the flexibility and accuracy of detecting images when using the query features for subsequent detection.

[0115] Step 202: Input the query features into the first feedforward network of the target detection model to perform query detection and obtain the initial detection results.

[0116] Step 202 is described in detail below.

[0117] In some embodiments, after adaptively obtaining an appropriate number of query features (i.e., the adaptive decoder query vector SADQ), the query features SADQ are input into the first feedforward network (i.e., the first feedforward neural network submodule) of the target detection model for detection and recognition, thereby obtaining the initial detection result of the detection position corresponding to the query feature SADQ.

[0118] Step 203: Input multiple query features into the localization submodule of the target detection model for detection and localization to obtain the first corrected query features, and input the first corrected query features into the second feedforward network of the target detection model for query detection to obtain the first corrected bias result.

[0119] Step 203 will be described in detail below.

[0120] In some embodiments, to further improve the accuracy of the detection results of the obtained detection images, it is necessary to further refine the query features. Therefore, after obtaining the query features, the query features are also input into the localization submodule of the target detection model for detection and localization, so as to perform the first refinement of the query features and obtain the first refined query features. The special features of this localization submodule will be described in further detail below.

[0121] In some embodiments, the DETR-like model in related technologies employs an end-to-end (i.e., one query feature to one detection result) matching mode. However, this end-to-end detection model suffers from low training efficiency and slow convergence speed. Therefore, this solution introduces a one-to-many matching method for the localization submodule, which improves the training efficiency of the model by increasing the number of positive samples during the training process in the localization submodule.

[0122] In this embodiment, the localization module includes a first multi-scale deformable cross-attention module, a first layer-norm submodule, a second layer-norm submodule, and a first multi-layer perceptron (MLP) submodule. During the training process of the localization module, multiple duplicate ground truth detection boxes are set, and each duplicate ground truth detection box is matched with a single query feature. The first multi-scale deformable cross-attention module is used to focus multiple query features on the same duplicate ground truth detection box, i.e., a one-to-many matching method. This ensures that in the first corrected query features obtained after the localization module modifies the novelty features, multiple first corrected query feature units point to the same detection result. The process of obtaining the first corrected query features in practical applications will be further described below.

[0123] Reference Figure 6 The first corrected query feature is obtained by inputting multiple query features into the localization submodule of the target detection model for detection and localization, including the following steps 601 to 604.

[0124] Step 601: Input the query features and detection features into the first cross-attention submodule for data association to obtain the first localization detection features.

[0125] Step 602: Input the first localization detection feature and the query feature into the first standardization submodule for overlay processing to obtain the first localization overlay feature.

[0126] Step 603: Input the first localization overlay feature into the first multilayer perception submodule for information interaction to obtain localization interaction features.

[0127] Step 604: Input the location interaction features and the first location overlay features into the second standardization submodule for overlay processing to obtain the first corrected query features.

[0128] Steps 601 to 604 are described in detail below.

[0129] In some embodiments, the adaptive query features SADQq of the actual detected image are obtained. d Then, the matching vector (key) and information vector (value) of the query feature and the detection feature E are input together into the first multi-scale deformable cross-attention module for data association, so that multiple feature units in the query feature point to the same detection box, thereby obtaining the first localization detection feature CA(q). d E).

[0130] Next, the first localization detection feature CA(q) will be used. d ,E) and query feature q d The input is first standardized by layering the first standardization submodule (LayerNorm) and then standardized again to make the query feature q d and the first localization detection feature CA(q) d E) performs hybrid standardization to standardize and correct the query features, thereby stabilizing the data processing transformation process of the positioning submodule and improving the data stability of the positioning submodule, thus obtaining the first positioning superimposed feature LN(q) d +CA(q d ,E)).

[0131] Furthermore, the first localization overlay feature LN(q) is... d +CA(q d The input of E)) is used to interact with the first multilayer perceptron (MLP) submodule to facilitate the input of the first localization overlay feature LN(q) d +CA(q d The first localization detection feature CA(q) in E)) d ,E) and query feature q d Information exchange to obtain localization interaction features MLP(LN(q) d +CA(q d In this embodiment, the MLP includes at least two linear layers to better facilitate information interaction.

[0132] Finally, the interaction features of the localization MLP(LN(q) will be used. d +CA(q d ,E))) and the first localization superimposed feature LN(q d +CA(q d The input is overlaid into the second standardization submodule (LayerNorm) and then standardized to obtain the first modified query feature BLP(q). d E) is shown in the following formula (4).

[0133] BLP(q d ,E)=LN{MLP(LN(q d +CA(q d ,E)))+LN(q d +CA(q d ,E))} (4)

[0134] Through steps 601 to 604 above, the localization submodule obtained by the one-to-many training matching method is used to modify the query features so that multiple feature units in the query features point to the same detection box more often, thereby effectively improving the accuracy of target detection in the detection image. In addition, this one-to-many training matching method improves the training convergence and training efficiency of the target detection model.

[0135] In some embodiments, after obtaining the first modified query feature BLP(q) d After E), the first modified query feature BLP(q) will also be applied. d E) Input the second feedforward network (i.e., the second feedforward neural network submodule) in the target detection model to perform query detection and obtain the first corrected bias result B. ΔxΔyΔwΔh .

[0136] Step 204: Input the first corrected query feature into the deduplication submodule of the target detection model for feature processing to obtain the second corrected query feature, and input the second corrected query feature into the third feedforward network of the target detection model for query detection to obtain the second corrected bias result.

[0137] Step 204 is described in detail below.

[0138] In some embodiments, while this one-to-many training matching method can effectively improve the training convergence and detection accuracy of the object detection model, it also results in a large number of false positive queries, i.e., multiple repeated query features pointing to the same detection box, which contradicts the specific end-to-end matching model in the field of image detection.

[0139] Therefore, to address the false positive query problem, this embodiment of the application also includes a deduplication submodule in the target detection model. Then, the deduplication submodule is used to process the first corrected query features (i.e., a second correction) to remove duplicate query feature units pointing to the same result, thus obtaining the second corrected query features.

[0140] In this embodiment, the deduplication module includes a second multi-scale deformable cross-attention module, a multi-head self-attention module, a third normalization module (LayerNorm), a fourth normalization module (LayerNorm), and a second multi-layer perceptron (MLP) module. During the training of the deduplication module, through one-to-one matching, query feature units that match the ground truth (GT) box gradually achieve higher detection and classification scores under the influence of classification loss; conversely, query feature units that do not match the GT box gradually decrease in detection and classification scores. This process gradually filters out duplicate first-corrected query feature units with low detection and classification scores during training, ultimately retaining only one query feature unit matching one GT box.

[0141] Furthermore, since the query feature SADQ is updated sequentially by the localization submodule BLP and the deduplication submodule DP during the training of the object detection module, query features SADQ (i.e., queries) that have one-to-many matching queries in the localization submodule BLP may become matched one-to-one in the deduplication submodule DP. This can lead to contradictory supervision information and chaotic gradient updates, causing detection ambiguity to reappear. To address this issue, this embodiment employs an additional processing method: stopping the backpropagation of gradients from the deduplication submodule DP to the localization submodule BLP during training (Stop Gradient of Query, SGQ) to ensure consistent matching between different parts.

[0142] Based on this specially trained deduplication submodule, in practical applications, it can effectively correct multiple query feature units pointing to the same detection box in the first corrected query feature to obtain a second corrected query feature with only one query unit pointing to the same detection box. This will be described in more detail below.

[0143] Reference Figure 7The first corrected query feature is input into the deduplication submodule of the target detection model for feature processing to obtain the second corrected query feature, including the following steps 701 to 705.

[0144] Step 701: Input the first modified query feature and the detection feature into the second cross-attention submodule for data association to obtain the second localization detection feature.

[0145] Step 702: Input the second localization detection feature and the first corrected query feature into the third standardization submodule for superposition processing to obtain the second localization superposition feature.

[0146] Step 703: Input the second localization overlay feature into the self-attention submodule for data integration to obtain the localization integrated feature.

[0147] Step 704: Input the integrated positioning features and the superimposed positioning features into the third standardization submodule for superimposition processing to obtain the integrated standard features.

[0148] Step 705: Input the integrated standard features into the second multi-layer perception submodule for information interaction to obtain the second corrected query features.

[0149] Steps 701 to 705 are described in detail below.

[0150] In some embodiments, after obtaining the first modified detection feature BLP(q) d ,E)(i.e., Updated SADQq d After '), the first modified query feature BLP(q) is applied. d The matching vector (key) and information vector (value) of the query feature unit E and the detection feature E are input into the second cross-attention module (Multi-scale Deformable Cross-Attention Module) for data association. This allows for fine-tuning and optimization of the detection box position corresponding to the query feature unit using the cross-attention structure, further improving detection accuracy, thereby obtaining the second localization detection feature CA(q). d ',E).

[0151] Next, the second localization detection feature CA(q) will be used. d ',E) and the first modified query feature q d The input is superimposed from the third standardization submodule (LayerNorm) and then standardized to make the first modified query feature q... d ' and second localization detection feature CA(q) d ',E) Perform hybrid standardization to first modify query feature q dStandardization corrections are performed to stabilize the data processing transformation process of the deduplication submodule, improve the data stability of the deduplication submodule, and thus obtain the second localization overlay feature DP. CA (q d ',E)=LN(q d '+CA(q d ',E)).

[0152] Furthermore, the second localization feature DP is superimposed. CA (q d ',E) Input the multi-head self-attention submodule SA to eliminate duplicate detection feature units pointing to the same detection box in the first corrected detection features. The multi-head self-attention submodule SA includes a self-attention submodule (Multi-head Self-Attention Module) and a second multi-layer perceptron (MLP) submodule. The multi-head self-attention submodule SA is used to update the detection feature units in the second localization overlay features. It adopts a one-to-one matching method. In the deduplication submodule, the multi-head self-attention submodule SA performs a total of λ loop iterations to improve the deduplication effect of detection boxes.

[0153] The execution steps in the multi-head self-attention submodule (SA) specifically involve superimposing the second localization feature (DP). CA (q d The input data (e) is integrated into the Multi-head Self-Attention Module to obtain the localization integration feature SA(q). d '), and integrate the positioning feature SA(q) d ') and second localization superimposed feature DP CA (q d ',E) Input the third standardization submodule (LayerNorm) for overlay and then perform standardization processing to obtain the integrated standard feature LN[q d '+SA(q d Then, the integrated standard features are input into the second multilayer perceptron (MLP) submodule for information interaction. After a total of λ iterations, the second modified query feature DP is obtained. MSAB (q d ',E), as shown in the following formula (5).

[0154]

[0155] Where, q d ' (0) For the second localization overlay feature DP CA (q d ',E).

[0156] Through steps 701 to 605 above, the multi-head self-attention submodule SA (including a self-attention submodule and a second multilayer perception submodule) obtained by the one-to-one training matching method is used to perform data matching duplication elimination processing on the first modified query feature, so that each feature unit in the obtained second query feature points to a different detection box, thereby effectively solving the false positive query problem caused by the localization submodule based on the one-to-many matching method, and effectively improving the reliability of the target detection method; in addition, the second cross-attention submodule is used to fine-tune and optimize the position of the detection box corresponding to the query feature unit of the first modified query feature, further improving the detection accuracy of the target detection method.

[0157] In some embodiments, after obtaining the second modified query feature DP MSAB (q d After ',E), the second modified query feature DP will also be applied. MSAB (q d ',E) Input the third feedforward network (i.e., the second feedforward neural network submodule) in the target detection model to perform query detection, and obtain the second corrected bias result B. Δx'Δy'Δw'Δh' .

[0158] Step 205: Obtain the target detection result of the detection image based on the initial detection result, the first correction bias result, and the second correction bias result.

[0159] Step 205 is described in detail below.

[0160] In some embodiments, after obtaining the initial detection result B obtained from the query feature SADQ, x0y0w0h0 The first correction bias result B obtained from the first correction query feature ΔxΔyΔwΔh (i.e., BLP(q) d ,E)) and the second corrected bias result B obtained from the second corrected query feature Δx'Δy'Δw'Δh' (i.e., DP(q) d After ',E)), the initial detection result B is accumulated. x0y0w0h0 The first corrected bias result BLP(q) d E) and the second corrected bias result DP(q) d The accurate target detection result of the detected image can be obtained by formula (6) as shown below.

[0161]

[0162] Where T1 is the number of localization sub-modules in the object detection model, and T2 is the number of deduplication sub-modules in the object detection model.

[0163] Reference Figure 8 This is a schematic diagram of a target detection process for an image provided in an embodiment of this application. For example... Figure 8 As shown, after a detection image with dimensions H×W×3 is input into the object detection module, the backbone network first extracts initial features from the detection image. Then, the Transformer Encoder enhances the initial features to obtain detection features E. Next, the detection features E are input into the feature filtering submodule, where the Adaptive Free Query Selection Algorithm (AFQS) is used to adaptively select features, resulting in the self-adaptation decoder query (SADQ), i.e., the query features. The query features are then sequentially input into T1 localization submodules for a first correction process to obtain the first corrected detection features. The first corrected detection features are then sequentially input into T2 deduplication submodules for a second correction process to obtain the second corrected detection features. Finally, based on the initial detection results obtained from the query features SADQ, the first correction bias results obtained from the first corrected query features, and the second correction bias results obtained from the second corrected query features, the location detection results and category detection results of the object detection model are obtained. Based on the location detection results, detection boxes are generated on the detection image.

[0164] In some embodiments, during the training process of the target detection model using the above-described target detection method, there may be inconsistencies between the classification score predicted by the same detection feature unit query and the detection box regression result, such as: high classification confidence but low detection box accuracy; or low classification confidence but high detection box accuracy.

[0165] Based on this, in the training process of the object detection model, the Intersection over Union (IoU) between the predicted bounding box and the detected bounding box is introduced into the loss function. The training process of the object detection model will be further described below.

[0166] Reference Figure 9 The training steps for the object detection model include the following steps 901 to 904.

[0167] Step 901: Obtain training image samples and corresponding test image samples.

[0168] Step 902: Input the image samples into the target detection model for data detection to obtain the training detection results and training detection classification scores.

[0169] Step 903: Generate sample labels based on the cross-union ratio and training detection classification score between the training detection results and the test image samples.

[0170] Step 904: Obtain the training loss value based on the sample labels and training detection classification scores, and update the model parameters of the object detection model based on the training loss value.

[0171] Steps 901 to 904 are described in detail below.

[0172] In some embodiments, training image samples and corresponding test image samples for training each sub-module in the object detection model are first obtained. Next, the image samples are input into the object detection model, and data detection is performed based on the relevant steps in the object detection method described above to obtain the training detection result (i.e., the training detection box) and the training detection classification score p.

[0173] Then, based on the intersection-union ratio (IU) between the training detection results and the test image samples, and the training detection classification score, sample labels t are generated. This combines the IU and classification scores to overcome the inconsistency between the classification score and the bounding box regression results. In this embodiment, the method of generating sample labels t is not limited; it can be obtained by quantizing the IU and the training detection classification score into scores of the same order of magnitude and then summing them.

[0174] Next, the training loss value is obtained based on the sample label t and the training detection classification score p, and the model parameters corresponding to multiple sub-modules in the object detection model are updated based on the training loss value. The method for obtaining the training loss value will be further described below.

[0175] Reference Figure 10 The training loss value is obtained based on the sample label and the training detection classification score, including the following steps 1001 to 1004.

[0176] Step 1001: Based on the binary cross-entropy loss function, obtain the first cross-entropy loss value of the sample label and the training detection classification score, and obtain the second cross-entropy loss value of the training detection classification score.

[0177] Step 1002: Obtain the training detection target size of the training detection result, and obtain the training image size of the training image sample.

[0178] Step 1003: Based on the ratio of the size of the target to the size of the training image, and by performing an exponential operation based on the scaling hyperparameter, the bounding box parameter values ​​are obtained.

[0179] Step 1004: Accumulate the first cross-entropy loss value, the bounding box parameter value, and the second cross-entropy loss value to obtain the training loss value.

[0180] Steps 1001 to 1004 are described in detail below.

[0181] In some embodiments, the first cross-entropy loss value BCE(p,t) of the sample label t and the training detection classification score p is obtained based on the binary cross-entropy loss function BCE, and the second cross-entropy loss value BCE(p,0) of the training detection classification score is obtained.

[0182] Then, obtain the size of the training detection target in the training detection result (i.e., the training detection box), which includes the height h. i and width w i And obtain the training image size of the training image sample, which includes height H and width W.

[0183] Next, based on the ratio HW / (h) of the training detection target size to the training image size, i w i The square root of ) is obtained by performing an exponential operation α based on the scaling hyperparameter and adding one, resulting in the bounding box parameter values ​​as shown in the following formula (7).

[0184]

[0185] Finally, the first cross-entropy loss value, the bounding box parameter value, and the second cross-entropy loss value are summed to obtain the training loss value PoCoo as shown in the following formula (8).

[0186]

[0187] Where i and j represent different prediction target indices; h i and w i The height and width of the matched ground truth bounding box are represented; p and t represent the predicted classification score and the new label combined with Iou; the α value is between 0 and 1.

[0188] It is understood that in this embodiment, the loss function (8) corresponding to the training loss value PoCoo is the parameter update for the classification training of the object detection model, while the parameter update for the detection box training of the object detection model uses other related loss functions, such as the generalized intersection-over-union loss (GIou loss) and the L1 norm loss function (L1 loss).

[0189] Through steps 901 to 904 and steps 1001 to 1004, by introducing the sample labels obtained from the intersection-union ratio between the training detection results and the test image samples, as well as the training detection classification score, as one of the parameters of the loss function, the inconsistency between the classification score predicted by the same detection feature unit query and the detection box regression result is effectively overcome, thereby improving the training efficiency and detection accuracy of the object detection model during the training process. In addition, the size information corresponding to the size parameters between the training detection results and the test image samples is also combined as one of the parameters of the loss function, so that the object detection model can pay more attention to small targets during the training process, thereby increasing the detection accuracy of small targets in actual detection.

[0190] To improve the accuracy of the target detection method provided in this application embodiment, this embodiment also conducted simulation comparisons of the target detection method with several other related schemes, and carried out experiments using two publicly available target detection standard datasets, COCO2017 and WiderPerson.

[0191] The COCO2017 dataset is a widely used benchmark dataset for object detection. COCO2017 contains 118k training images and 5k validation images, covering 80 categories. WiderPerson is a large and diverse dataset for detecting people. It contains 13,382 images with a total of 399,786 annotations, averaging 29.87 annotations per image. Due to scene diversity and severe occlusion, this dataset presents a significant challenge for small object detection; it contains 8,000 training images and 1,000 validation images.

[0192] For the COCO2017 dataset, this application uses the standard mean precision metric to evaluate detection performance at different IoU thresholds and target scales, following the evaluation metrics of the COCO dataset. For the WiderPerson dataset, this application uses AP, Recall, and mean Minimum Recall (mMR) as evaluation metrics, which are widely used in the field of pedestrian detection.

[0193] Reference Figure 11 This is a performance simulation diagram of the first target detection method provided in the embodiments of this application. For example... Figure 11The diagram illustrates a comprehensive comparison between the object detection method (QFree-Det) provided in this application and various popular detectors across different training epochs and backbones. It can be seen that QFree-Det achieves the best performance in general object detection and small object detection in terms of AP and APs, respectively. For the ResNet50 backbone, the object detection model in this application outperforms the baseline model DINO by 1.5% AP and 2.3% APs at 12 epochs. In particular, QFree-Det (only 24 epochs) outperforms DINO (36 epochs), improving AP by 0.4% (51.3% vs 50.9%) and APs by 0.9% (35.5% vs 34.6%), respectively. These results significantly demonstrate the superior training efficiency and performance of the model presented in this invention. For Strip-MLP-T and Swin-T backbones (for which relatively few methods report results), the object detection model in this application achieved new state-of-the-art accuracies of 55.0% AP and 54.9% AP with parameters of 45M and 49M, respectively.

[0194] Reference Figure 12 This is a performance simulation diagram of the first target detection method provided in the embodiments of this application. (Refer to...) Figure 13 This is a performance simulation diagram of the second target detection method provided in the embodiments of this application. (Refer to...) Figure 14 This is a performance simulation diagram of the third target detection method provided in the embodiments of this application. (Refer to...) Figure 15 This is a performance simulation diagram of the fourth object detection method provided in this application. Recently, the Visual State Space Model (VSUM) has been proposed to address the quadratic complexity problem of attention mechanisms, and the latest MambaOut work indicates that Visual Mamba has great potential in long-sequence visual tasks (such as object detection). Therefore, this application uses VMamba-T as the backbone to test the accuracy of QFree-Det and DINO models. Figure 11The results show that QFree-Det achieves new state-of-the-art (SOTA) accuracies of 54.4% AP and 38.8% APs at 12 epochs, which are higher than DINO by 0.9% AP and 2.2% APs, respectively. Furthermore, compared to DiffusionDet, the only current model capable of predicting any number of targets, QFree-Det (24 epochs) significantly outperforms DiffusionDet (60 epochs), improving accuracy by +4.5% AP and +5.9% APs, fully demonstrating the superiority of the target detection model provided in this application. The accuracy comparison line graph is shown below. Figures 12 to 15 As shown in the image.

[0195] Reference Figure 16 This is a performance simulation diagram of the second object detection method provided in the embodiments of this application. To further evaluate the effectiveness of QFree-Det, this application conducted another ablation experiment on the challenging WiderPerson dataset. Figure 16 As shown, QFree-Det achieved superior performance compared to the baseline model DINO on four different backbones and surpassed other state-of-the-art models, further demonstrating its effectiveness.

[0196] The target detection method, apparatus, electronic device, and storage medium proposed in this application include the following steps: First, acquiring detection features of a detection image; classifying and recognizing multiple detection feature units to obtain classification result probability values ​​related to the predicted category; selecting detection feature units whose classification result probability values ​​exceed a classification score threshold from the multiple detection feature units as candidate detection feature units; sorting the classification result probability values ​​of all candidate detection feature units to determine the number of feature units in the candidate detection feature units; when the number of candidate units is less than the query feature capacity, using all detection feature units in the candidate detection feature units as query features; when the number of candidate units is greater than the query feature capacity, selecting detection feature units that match the query feature capacity from the candidate detection feature units according to the sorting of all detection feature units in the candidate detection feature units as query features; then, inputting the query features into the first feedforward network of the target detection model for query detection to obtain initial detection results; next, inputting the query features and detection features into the first cross-attention submodule for data association to obtain first localization detection features; and inputting the first localization detection features and query features into the first standardization submodule for superposition processing to obtain the first localization detection feature. The system performs a feature overlay process. First, the first localization overlay feature is input into the first multilayer perception submodule for information interaction to obtain localization interaction features. Then, the localization interaction features and the first localization overlay feature are input into the second standardization submodule for overlay processing to obtain a first corrected query feature. This first corrected query feature is then input into the second feedforward network of the target detection model for query detection to obtain a first corrected bias result. Next, the first corrected query feature and the detection feature are input into the second cross-attention submodule for data association to obtain a second localization detection feature. This second localization detection feature and the first corrected query feature are then input into the third standardization submodule for overlay processing to obtain a second localization overlay feature. This second localization overlay feature is then input into the self-attention submodule for data integration to obtain a localization integration feature. This localization integration feature and the second localization overlay feature are then input into the third standardization submodule for overlay processing to obtain an integrated standard feature. This integrated standard feature is then input into the second multilayer perception submodule for information interaction to obtain a second corrected query feature. This second corrected query feature is then input into the third feedforward network of the target detection model for query detection to obtain a second corrected bias result. Finally, based on the initial detection result, the first corrected bias result, and the second corrected bias result, the target detection result of the detected image is obtained.Furthermore, during the training process of the object detection model, training image samples and corresponding test image samples are acquired. These image samples are input into the object detection model for data detection, yielding training detection results and training detection classification scores. Based on the intersection-union ratio (IU) between the training detection results and the test image samples, and the training detection classification scores, sample labels are generated. Using the binary cross-entropy loss function, the first cross-entropy loss value for the sample labels and the training detection classification scores, as well as the second cross-entropy loss value for the training detection classification scores, are obtained. The training target size for the training detection results and the training image size for the training image samples are also acquired. Based on the ratio of the training target size to the training image size, and using an exponential operation based on the scaling hyperparameter, the bounding box parameter values ​​are obtained. The first cross-entropy loss value, the bounding box parameter values, and the second cross-entropy loss value are accumulated to obtain the training loss value. The model parameters of the object detection model are then updated based on the training loss value.

[0197] This application embodiment uses the classification result probability value and classification score of each unit in the detection feature to initially filter out the detection feature units, filtering out those that are less meaningful. Next, the query feature capacity is further matched with the number of feature units in the initially filtered candidate detection feature units, and the detection feature units are filtered again based on the matching results. This selects feature units with high classification result probability values ​​as an adaptive number of query features, effectively improving the flexibility and accuracy of image detection when using these query features for subsequent detection. Furthermore, a localization submodule obtained by a one-to-many training matching method is used to modify the query features, so that multiple feature units in the query features point to the same detection box more often, thereby effectively improving the accuracy of target detection in the image. This one-to-many training matching method also improves the training convergence and efficiency of the target detection model. Finally, a multi-head self-attention submodule SA (including a self-attention submodule and a second multilayer perception submodule) obtained by a one-to-one training matching method is used to perform data matching and duplicate elimination processing on the first modified query features, so that each feature unit in the resulting second query features points to a different detection box, effectively solving the problem of data duplication due to the one-to-many training matching method. The method addresses the false positive query problem in the localization submodule obtained by the "to many" matching method, effectively improving the reliability of the object detection method. Furthermore, it utilizes a second cross-attention submodule to fine-tune and optimize the position of the detection box corresponding to the first modified query feature unit, further enhancing the detection accuracy. Additionally, by introducing the sample label obtained from the intersection-union ratio between the training detection results and the test image samples, as well as the training detection classification score, as one of the parameters of the loss function, it effectively overcomes the inconsistency between the classification score predicted by the same detection feature unit query and the detection box regression result, thereby improving the training efficiency and detection accuracy of the object detection model during training. Moreover, it incorporates the size information corresponding to the size parameters between the training detection results and the test image samples as one of the parameters of the loss function, enabling the object detection model to pay more attention to small targets during training, thus increasing the detection accuracy of small targets in actual detection.

[0198] This application also provides a target detection device that can implement the above-described target detection method, see reference. Figure 17 The device 1700 includes:

[0199] The query feature acquisition module 1710 acquires the detection features of the detected image and inputs the detection features into the feature filtering submodule of the target detection model for feature filtering to obtain multiple query features;

[0200] The initial detection module 1720 is used to input query features into the first feedforward network of the target detection model for query detection and obtain initial detection results.

[0201] The first feature correction module 1730 is used to input multiple query features into the localization submodule of the target detection model for detection and localization, to obtain the first corrected query feature, and to input the first corrected query feature into the second feedforward network of the target detection model for query detection, to obtain the first corrected bias result;

[0202] The second feature correction module 1740 inputs the first corrected query feature into the deduplication submodule of the target detection model for feature processing to obtain the second corrected query feature, and inputs the second corrected query feature into the third feedforward network of the target detection model for query detection to obtain the second corrected bias result.

[0203] The target detection module 1750 is used to obtain the target detection result of the detection image based on the initial detection result, the first correction bias result, and the second correction bias result.

[0204] In some embodiments, the query feature acquisition module 1710 is further configured to:

[0205] Multiple detection feature units are classified and identified to obtain the probability value of the classification result related to the predicted category;

[0206] From multiple detection feature units, select the detection feature units whose classification result probability value exceeds the classification score threshold as candidate detection feature units;

[0207] The classification result probability values ​​of all candidate detection feature units are sorted by size, and at least one detection feature unit is selected from the candidate detection feature units as the query feature based on the sorting result.

[0208] In some embodiments, the query feature acquisition module 1710 is further configured to:

[0209] Determine the number of feature units among the candidate detection feature units;

[0210] When the number of candidate units is less than the query feature capacity, all detection feature units in the candidate detection feature units will be used as query features.

[0211] When the number of candidate units exceeds the query feature capacity, the candidate detection feature units are selected from the candidate detection feature units according to the sorting of all detection feature units, and the detection feature units that match the query feature capacity are selected as the query features.

[0212] In some embodiments, the first feature correction module 1730 is further configured to:

[0213] The query features and detection features are input into the first cross-attention submodule for data association to obtain the first localization detection features;

[0214] The first localization detection feature and the query feature are input into the first standardization submodule for overlay processing to obtain the first localization overlay feature;

[0215] The first positioning overlay feature is input into the first multilayer perception submodule for information interaction to obtain the positioning interaction feature;

[0216] The location interaction features and the first location overlay features are input into the second standardization submodule for overlay processing to obtain the first corrected query features.

[0217] In some embodiments, the second feature correction module 1740 is further configured to:

[0218] The first modified query feature and the detection feature are input into the second cross-attention submodule for data association to obtain the second localization detection feature;

[0219] The second localization detection feature and the first corrected query feature are input into the third standardization submodule for superposition processing to obtain the second localization superposition feature;

[0220] The second localization overlay feature is input into the self-attention submodule for data integration to obtain the localization integrated feature;

[0221] The integrated positioning features and the superimposed positioning features are input into the third standardization submodule for superposition processing to obtain the integrated standard features.

[0222] The integrated standard features are input into the second multi-layer perception submodule for information interaction to obtain the second modified query features.

[0223] In some embodiments, the target detection device further includes a training module 1760, which is used for:

[0224] Obtain training image samples and corresponding test image samples;

[0225] Image samples are input into the target detection model for data detection to obtain training detection results and training detection classification scores.

[0226] Based on the cross-union ratio between the training detection results and the test image samples, and the training detection classification score, sample labels are generated;

[0227] The training loss value is obtained based on the sample labels and the training detection classification score, and the model parameters of the object detection model are updated based on the training loss value.

[0228] In some embodiments, the training module 1760 is further configured to:

[0229] Based on the binary cross-entropy loss function, the first cross-entropy loss value of the sample label and the training detection classification score is obtained, as well as the second cross-entropy loss value of the training detection classification score is obtained.

[0230] Obtain the size of the training detection target for the training detection results, and obtain the size of the training image for the training image samples;

[0231] The bounding box parameter values ​​are obtained by using the ratio of the size of the target to the size of the training image and by performing an exponential operation based on the scaling hyperparameter.

[0232] The training loss value is obtained by summing the first cross-entropy loss value, the bounding box parameter value, and the second cross-entropy loss value.

[0233] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, the specific implementation of the target detection device is basically the same as the specific implementation of the target detection method described above, and will not be repeated here.

[0234] In this embodiment, the target detection device uses the classification result probability value and classification score of each unit in the detection features to initially filter and screen the detection feature units, filtering out those that are less meaningful. Next, the query feature capacity is further matched with the number of feature units in the initially filtered candidate detection feature units, and the detection feature units are filtered and screened again based on the matching results. This selects feature units with high classification result probability values ​​as an adaptive number of query features, effectively improving the flexibility and accuracy of image detection when using these query features for subsequent detection. Furthermore, the localization submodule obtained by the one-to-many training matching method is used to modify the query features, so that multiple feature units in the query features point more often to the same detection box, thereby effectively improving the accuracy of target detection in the image. This one-to-many training matching method also improves the training convergence and efficiency of the target detection model. Additionally, the one-to-many training matching method further enhances the accuracy of target detection. The multi-head self-attention submodule (SA) obtained by the one-to-many matching method (including a self-attention submodule and a second multilayer perceptron submodule) performs data matching duplication elimination processing on the first modified query features, so that each feature unit in the obtained second query features points to a different detection box. This effectively solves the false positive query problem caused by the localization submodule obtained by the one-to-many matching method, thus effectively improving the reliability of the target detection method. In addition, the second cross-attention submodule is used to fine-tune and optimize the position of the detection box corresponding to the query feature unit in the first modified query features, further improving the detection accuracy of the target detection method. Furthermore, by introducing the cross-union ratio between the training detection results and the test image samples, as well as the sample label obtained by the training detection classification score, as one of the parameters of the loss function, the problem of the classification score predicted by the same detection feature unit query is effectively overcome. This addresses the inconsistency between the score and the bounding box regression results, thereby improving the training efficiency and detection accuracy of the object detection model during training. Furthermore, it incorporates the size information corresponding to the size parameters between the training detection results and the test image samples as one of the parameters of the loss function, enabling the object detection model to pay more attention to small targets during training, thus increasing the detection accuracy of small targets in actual detection.

[0235] This application also provides an electronic device, including:

[0236] At least one memory;

[0237] At least one processor;

[0238] At least one program;

[0239] The program is stored in a memory, and the processor executes the at least one program to implement the target detection method described above in this application. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), in-vehicle computers, etc.

[0240] Please see Figure 18 , Figure 18 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0241] The processor 1801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0242] The memory 1802 can be implemented in the form of ROM (Read-Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 1802 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1802 and is called and executed by the processor 1801 using the target detection method of the embodiments of this application.

[0243] The input / output interface 1803 is used to implement information input and output;

[0244] The communication interface 1804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0245] Bus 1805 transmits information between various components of the device (e.g., processor 1801, memory 1802, input / output interface 1803, and communication interface 1804);

[0246] The processor 1801, memory 1802, input / output interface 1803 and communication interface 1804 are connected to each other within the device via bus 1805.

[0247] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described target detection method.

[0248] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0249] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0250] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0251] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0252] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0253] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0254] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0255] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or indirect coupling or communication connection between the apparatus or units, and may be electrical, mechanical, or other forms.

[0256] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0257] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0258] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0259] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A target detection method, characterized in that, include: The detection features of the detected image are obtained, and the detection features are input into the feature filtering submodule of the target detection model for feature filtering to obtain multiple query features; The query features are input into the first feedforward network of the target detection model for query detection to obtain initial detection results; Multiple query features are input into the localization submodule of the target detection model for detection and localization to obtain a first corrected query feature. The first corrected query feature is then input into the second feedforward network of the target detection model for query detection to obtain a first corrected bias result. The first corrected query feature is input into the deduplication submodule of the target detection model for feature processing to obtain the second corrected query feature. The second corrected query feature is then input into the third feedforward network of the target detection model for query detection to obtain the second corrected bias result. The target detection result of the detection image is obtained based on the initial detection result, the first corrected bias result, and the second corrected bias result; In the training process of the localization submodule, multiple duplicate detection ground truth boxes are set, and each duplicate detection ground truth box is matched with a single query feature. This results in multiple first modified query feature units pointing to the same detection result in the first modified query feature obtained after the localization submodule corrects the query feature. The deduplication submodule is used to remove duplicate first-correction query feature units that point to the same result.

2. The target detection method according to claim 1, characterized in that, The detection features include multiple detection feature units. The feature filtering submodule of the target detection model is used to filter the detection features to obtain multiple query features, including: The multiple detection feature units are classified and identified to obtain the probability value of the classification result related to the predicted category; From multiple detection feature units, select the detection feature units whose classification result probability value exceeds the classification score threshold as candidate detection feature units; The classification result probability values ​​of all the candidate detection feature units are sorted by size, and at least one detection feature unit is selected from the candidate detection feature units as the query feature based on the sorting result.

3. The target detection method according to claim 2, characterized in that, The feature filtering submodule includes a query feature capacity. The step of selecting at least one detection feature unit from the candidate detection feature units as the query feature based on the sorting result includes: Determine the number of feature units of the detection feature unit among the candidate detection feature units; When the number of feature units is less than the query feature capacity, all the detection feature units in the candidate detection feature units are used as the query feature; When the number of feature units is greater than the query feature capacity, a detection feature unit that matches the query feature capacity is selected from the candidate detection feature units according to the sorting of all the detection feature units in the candidate detection feature units, and is used as the query feature.

4. The target detection method according to claim 1, characterized in that, The localization submodule includes a first cross-attention submodule, a first normalization submodule, a second normalization submodule, and a first multilayer perception submodule. The localization submodule, which inputs multiple query features into the target detection model for detection and localization to obtain a first corrected query feature, includes: The query features and the detection features are input into the first cross-attention submodule for data association to obtain the first localization detection features; The first location detection feature and the query feature are input into the first standardization submodule for superposition processing to obtain the first location superposition feature; The first positioning overlay feature is input into the first multi-layer perception submodule for information interaction to obtain the positioning interaction feature; The location interaction feature and the first location overlay feature are input into the second standardization submodule for overlay processing to obtain the first corrected query feature.

5. The target detection method according to claim 1, characterized in that, The deduplication submodule includes a second cross-attention submodule, a self-attention submodule, a third normalization submodule, a fourth normalization submodule, and a second multilayer perceptron submodule. The step of inputting the first corrected query feature into the deduplication submodule of the target detection model for feature processing to obtain the second corrected query feature includes: The first modified query feature and the detection feature are input into the second cross-attention submodule for data association to obtain the second localization detection feature; The second positioning detection feature and the first corrected query feature are input into the third standardization submodule for superposition processing to obtain the second positioning superposition feature; The second positioning overlay feature is input into the self-attention submodule for data integration to obtain the positioning integrated feature; The integrated positioning feature and the second overlay positioning feature are input into the fourth standardization submodule for overlay processing to obtain the integrated standard feature; The integrated standard features are input into the second multi-layer perception submodule for information interaction to obtain the second corrected query features.

6. The target detection method according to claim 1, characterized in that, The training steps of the object detection model include: Obtain training image samples and corresponding test image samples; The training image samples are input into the target detection model for data detection to obtain training detection results and training detection classification scores. Based on the intersection-over-union ratio (IoU) between the training detection results and the test image samples, and the training detection classification score, sample labels are generated; The training loss value is obtained based on the sample label and the training detection classification score, and the model parameters of the object detection model are updated based on the training loss value.

7. The target detection method according to claim 6, characterized in that, The step of obtaining the training loss value based on the sample labels and the training detection classification score includes: Based on the binary cross-entropy loss function, the first cross-entropy loss value of the sample label and the training detection classification score is obtained, and the second cross-entropy loss value of the training detection classification score is obtained. Obtain the training detection target size of the training detection result, and obtain the training image size of the training image sample; The bounding box parameter values ​​are obtained based on the ratio of the size of the training detection target to the size of the training image, and by performing an exponential operation based on the scaling hyperparameter. The training loss value is obtained by summing the first cross-entropy loss value, the bounding box parameter value, and the second cross-entropy loss value.

8. A target detection device, characterized in that, The target detection device includes: The query feature acquisition module acquires the detection features of the detected image and inputs the detection features into the feature filtering submodule of the target detection model for feature filtering to obtain multiple query features; The initial detection module is used to input the query features into the first feedforward network of the target detection model for query detection and obtain the initial detection result; The first feature correction module is used to input multiple query features into the localization submodule of the target detection model for detection and localization to obtain a first corrected query feature, and input the first corrected query feature into the second feedforward network of the target detection model for query detection to obtain a first corrected bias result; The second feature correction module inputs the first corrected query feature into the deduplication submodule of the target detection model for feature processing to obtain the second corrected query feature, and inputs the second corrected query feature into the third feedforward network of the target detection model for query detection to obtain the second corrected bias result. The target detection result module is used to obtain the target detection result of the detection image based on the initial detection result, the first correction bias result, and the second correction bias result; In the training process of the localization submodule, multiple duplicate detection ground truth boxes are set, and each duplicate detection ground truth box is matched with a single query feature. This results in multiple first modified query feature units pointing to the same detection result in the first modified query feature obtained after the localization submodule corrects the query feature. The deduplication submodule is used to remove duplicate first-correction query feature units that point to the same result.

9. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the target detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the target detection method as described in any one of claims 1 to 7.

Images