Target detection method and model training method, device, electronic equipment and medium

Abstract

The present disclosure provides a target detection method and device, a model training method and device, electronic equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical field of computer vision, image processing, deep learning and the like. The specific implementation scheme is as follows: an image to be detected is acquired, feature extraction is performed on the image to be detected to acquire an image feature map of the image to be detected; an encoder of a pre-trained target detection network is used to encode the image feature map to acquire global attention features of the image to be detected; a first decoder of the target detection network is used to perform feature mapping on the global attention features to acquire regression features of the image to be detected; and a second decoder of the target detection network is used to perform feature mapping on the global attention features to acquire classification features of the image to be detected.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the fields of computer vision, image processing, and deep learning. Specifically, this disclosure relates to an object detection method, a model training method, an apparatus, an electronic device, and a medium. Background Technology

[0002] Object detection tasks typically involve two tasks: a classification task to determine the category of the target object and a regression task to determine the size and location information of the target object.

[0003] The features required for classification and regression tasks are often different, creating an inherent conflict. Summary of the Invention

[0004] This disclosure provides a target detection method, a model training method, an apparatus, an electronic device, and a medium.

[0005] According to a first aspect of this disclosure, a target detection method is provided, the method comprising:

[0006] Acquire the image to be detected, extract features from the image to be detected, and obtain the image feature map of the image to be detected;

[0007] The image feature map is encoded by the encoder of a pre-trained object detection network to obtain the global attention features of the image to be detected.

[0008] The first decoder of the object detection network performs feature mapping on the global attention features to obtain the regression features of the image to be detected; the second decoder of the object detection network performs feature mapping on the global attention features to obtain the classification features of the image to be detected.

[0009] The regression features are input into the regression prediction layer of the target detection network to obtain the position of the prediction box; the classification features are input into the classification prediction layer of the target detection network to obtain the category to which the target within the prediction box belongs.

[0010] According to a second aspect of this disclosure, a method for training an object detection model is provided, the method comprising:

[0011] Obtain the image to be trained, the position of the bounding box corresponding to the target in the image to be trained, and the category to which the target in the image to be trained belongs;

[0012] Feature extraction is performed on the image to be trained to obtain an image feature map of the image to be trained; the image feature map is encoded by the encoder of the object detection network to obtain the global attention features of the image to be trained;

[0013] The first decoder of the object detection network performs feature mapping on the global attention features to obtain the regression features of the image to be trained; the second decoder of the object detection network performs feature mapping on the global attention features to obtain the classification features of the image to be trained.

[0014] The regression loss and classification loss are determined based on the regression features and the classification features; the target detection network is trained based on the regression loss and the classification loss.

[0015] According to a third aspect of this disclosure, a target detection apparatus is provided, the apparatus comprising:

[0016] The backbone network module is used to acquire the image to be detected, extract features from the image to be detected, and obtain the image feature map of the image to be detected.

[0017] The encoder module is used to encode the image feature map through the encoder of a pre-trained object detection network to obtain the global attention features of the image to be detected.

[0018] The decoder module is used to perform feature mapping on the global attention features through the first decoder of the object detection network to obtain the regression features of the image to be detected; and to perform feature mapping on the global attention features through the second decoder of the object detection network to obtain the classification features of the image to be detected.

[0019] The prediction module is used to input the regression features into the regression prediction layer of the target detection network to obtain the position of the prediction box; and to input the classification features into the classification prediction layer of the target detection network to obtain the category to which the target in the prediction box belongs.

[0020] According to a fourth aspect of this disclosure, a training apparatus for an object detection model is provided, the apparatus comprising:

[0021] The data acquisition module is used to acquire the image to be trained, the position of the target box corresponding to the target in the image to be trained, and the category to which the target in the image to be trained belongs;

[0022] The feature training module is used to extract features from the image to be trained to obtain an image feature map of the image to be trained; and to encode the image feature map through the encoder of the object detection network to obtain the global attention features of the image to be trained.

[0023] The decoding training module is used to perform feature mapping on the global attention features through the first decoder of the object detection network to obtain the regression features of the image to be trained; and to perform feature mapping on the global attention features through the second decoder of the object detection network to obtain the classification features of the image to be trained.

[0024] The backpropagation module is used to determine regression loss and classification loss based on the regression features and the classification features; and to train the object detection network based on the regression loss and the classification loss.

[0025] According to a fifth aspect of this disclosure, an electronic device is provided, the electronic device comprising:

[0026] At least one processor; and

[0027] A memory communicatively connected to at least one of the aforementioned processors; wherein,

[0028] The memory stores instructions that can be executed by at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the target detection method and the training method for the target detection model.

[0029] According to a sixth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to execute the above-described target detection method and target detection model training method.

[0030] According to a seventh aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the above-described target detection method and target detection model training method.

[0031] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0032] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0033] Figure 1 This is a schematic diagram of a target detection method in related technologies;

[0034] Figure 2 This is a schematic diagram of a target detection method in related technologies;

[0035] Figure 3 This is a schematic flowchart of a target detection method provided in an embodiment of this disclosure;

[0036] Figure 4 This is a flowchart illustrating some steps of a target detection method provided in an embodiment of this disclosure;

[0037] Figure 5 This is a schematic diagram of a specific embodiment of a target detection method provided in this disclosure;

[0038] Figure 6 This disclosure provides a flowchart illustrating a training method for an object detection model.

[0039] Figure 7 This is a schematic diagram of the structure of a target detection device provided in an embodiment of this disclosure;

[0040] Figure 8 This is a schematic diagram of the structure of a training device for an object detection model provided in an embodiment of this disclosure;

[0041] Figure 9 This is a block diagram of an electronic device used to implement the target detection method and the target detection model training method of the embodiments of this disclosure. Detailed Implementation

[0042] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0043] In some related technologies, target detection tasks can be accomplished using neural networks based on CNN (Convolutional Neural Network). These mainly include One-Stage methods represented by YOLO and SSD (The Single Shot Detector), and Two-Stage methods represented by Faster_RCNN (Faster Region Convolutional Neural Network).

[0044] As an example, the classic structure of a CNN-based object detection network can be as follows: Figure 1As shown, the image to be detected is used as the input. The backbone network is input to extract the image features of the image to be detected, and the Neck structure is input to perform multi-scale feature fusion on the extracted features. The fused features are then input into the Classification Head and Regression Head respectively to ensure that the classification features and regression features do not interfere with each other, thereby alleviating the problem of coupling between classification features and regression features to a certain extent.

[0045] The training process of a CNN-based object detection network can also be as follows: Figure 1 As shown, the Classification Loss is obtained based on the difference between the category output by the Classification Head and the actual target category; the Regression Loss is obtained based on the difference between the predicted bounding box output by the Regression Head and the actual target bounding box; backpropagation is performed based on the Classification Loss and Regression Loss to train the Backbone, Neck, Classification Head, and Regression Head.

[0046] However, neural networks based on CNNs all require dense detection, and then NMS (non-maximum suppression) is used to filter out redundant targets.

[0047] In some related technologies, a neural network based on DETR (DEtection Transformer, the visual version of Transformer) can be used to complete the object detection task. It relies on the structure of encoder and decoder composed of transformers and uses Hungarian one-to-one matching loss to optimize the network, thereby eliminating the NMS step and simplifying the object detection process.

[0048] As an example, a DETR network can be like... Figure 2 As shown, the image to be detected is used as the Input, which is fed into the Backbone network to extract image features. These features are then fed into the Encoder, which obtains global attention features based on the extracted features. These global attention features are then fed into the Decoder, which maps them to the query feature vector. Figure 2 (A circle filled with diagonal lines) is used to obtain a query that contains both classification and regression features (i.e., a circle with diagonal lines in the middle). Figure 2The query, which contains classification features and regression features, is input into the Classification Head (classification network) and Regression Head (regression network) respectively to obtain the predicted bounding box and the category to which the target in the predicted bounding box belongs.

[0049] The training process of DETR can also be like... Figure 2 As shown, the Classification Loss is obtained based on the difference between the category output by the Classification Head and the actual target category; the Regression Loss is obtained based on the difference between the predicted bounding box output by the Regression Head and the actual target bounding box; backpropagation is performed based on the Classification Loss and Regression Loss to train the Backbone, Encoder, Decoder, Classification Head, and Regression Head.

[0050] Neural networks based on DETR, like those based on CNN, alleviate the problem of coupling between classification and regression features to some extent by processing features separately through classification and regression networks.

[0051] This approach does not completely decouple the categorical and regression features and does not solve the problem of coupling between them.

[0052] The target detection method, target detection model training method, target detection device, target detection model training device, electronic device, and computer-readable storage medium provided in this disclosure are intended to solve at least one of the above-mentioned technical problems of the prior art.

[0053] The target detection method and target detection model training method provided in this disclosure can be executed by electronic devices such as terminal devices or servers. Terminal devices can be in-vehicle devices, user equipment (UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0054] Figure 3 A schematic flowchart of a target detection method provided in an embodiment of this disclosure is shown, as follows: Figure 3As shown, the method mainly includes steps S310, S320, S330, and S340.

[0055] In step S310, the image to be detected is acquired, features are extracted from the image to be detected, and the image feature map of the image to be detected is obtained.

[0056] In step S320, the image feature map is encoded by the encoder of the pre-trained target detection network to obtain the global attention features of the image to be detected.

[0057] In step S330, the global attention features are mapped using the first decoder of the object detection network to obtain the regression features of the image to be detected; the global attention features are mapped using the second decoder of the object detection network to obtain the classification features of the image to be detected.

[0058] In step S340, regression features are input into the regression prediction layer of the object detection network to obtain the position of the prediction box; classification features are input into the classification prediction layer of the object detection network to obtain the category to which the object in the prediction box belongs.

[0059] For example, in step S310, the image to be detected is the image for which target detection is required. The image to be detected can be an image acquired online, such as the target image to be detected can be acquired online through web crawling technology, or the image to be detected can be an image acquired offline, or the image to be detected can be an image acquired in real time, or the image to be detected can be an artificially synthesized image, etc. This disclosure embodiment does not limit this.

[0060] It should be understood that the image to be detected can also be a video frame in a video. The image to be detected can be extracted from the video. The video can be called the video to be detected. The method of obtaining the video to be detected is similar to the method of obtaining the image to be detected, and will not be described in detail here.

[0061] In some possible implementations, traditional image processing methods can be used to extract features from the image to be detected and obtain the image feature map of the image to be detected;

[0062] In some possible implementations, in order to improve the accuracy and reliability of feature extraction results, neural networks can be used to extract features from the target image to obtain the image feature map of the image to be detected.

[0063] In some possible implementations, mainstream CNN networks can be used to extract features from the image to be detected, obtaining image feature maps. For example, the ResNet (residual network) series (such as ResNet 34, ResNet50, ResNet101, etc.) and the DarkNet (an open-source neural network framework written in C and CUDA) series (such as DarkNet19, DarkNet53) can be used.

[0064] In some possible implementations, the CNN network that extracts the image to be detected is used as the backbone of the object detection network and trained together with the other parts of the object detection network.

[0065] In some possible implementations, in order to balance the accuracy of feature extraction results with reducing resource consumption and detection speed, a suitable backbone can be selected for feature extraction of the image to be detected, depending on the application scenario.

[0066] In some specific implementations, the backbone network structure can be selected based on the application scenario, from lightweight structures (such as ResNet18, ResNet34, DarkNet19, etc.), medium-weight structures (such as ResNet50, ResNeXt50 (ResNeXt is a combination of ResNet and Inception (a type of convolutional neural network), DarkNet53, etc.), to heavyweight structures (such as ResNet101, ResNeXt152).

[0067] In step S320, in some possible implementations, the encoder of the object detection network can be an encoder based on the Transformer model or an encoder based on a variant of the Transformer model.

[0068] In some possible implementations, the encoder's network structure is consistent with the encoder structure of the DETR network in related technologies.

[0069] In some possible implementations, the encoder encodes the image feature map. This can be achieved by the encoder calculating the correlation between each feature point in the image feature map and other feature points in the image feature map, thereby concentrating the information of other feature points into that feature point and obtaining global attention features.

[0070] In some specific implementations, the image feature map can be reduced in dimensionality by performing a 1×1 convolution before being input into the encoder.

[0071] In some specific implementations, the encoder uses self-attention and multi-head self-attention mechanisms to perform global analysis on the image feature map, extract the interrelationships between different physical objects at different locations, and obtain global attention features.

[0072] In step S330, in some possible implementations, the first decoder and the second decoder can be decoders based on the Transformer model or decoders based on a variant of the Transformer model.

[0073] In some possible implementations, global attention features can be mapped to a predefined query feature vector.

[0074] In some possible implementations, the number of queries can be preset.

[0075] In some possible implementations, mapping global attention to a preset query can be achieved by determining content features (i.e., V value) and key features (i.e., K value) based on the global attention features output by the encoder, determining query features (i.e., Q value) based on the query feature vector, and performing cross-attention processing based on content features, key features, and query features to complete feature mapping and obtain regression features or classification features.

[0076] In some possible implementations, cross-attention processing based on content features, key-value features, and query features is performed to complete feature mapping. This can be achieved by determining attention weights based on query features and key-value features, and then determining attention features based on attention weights and content features, thus completing feature mapping.

[0077] In some possible implementations, the first decoder and the second decoder can be decoders with different structures and weights. In other possible implementations, the first decoder and the second decoder can also be decoders with the same structure but different weights.

[0078] In some specific implementations, the basic structures of the first decoder and the second decoder can be the same, both being the basic structures of the decoder of the Transformer model. Both the first decoder and the second decoder can be composed of multiple cascaded basic structures of the decoder of the Transformer model.

[0079] In some specific implementations, the number of basic structures of the Transformer model decoders included in the first and second decoders can be determined based on the attributes of the target in the image to be detected.

[0080] In some possible implementations, the target of the image to be detected is a relatively regular shape (such as a table, a book, or other shaped target). It is relatively easy to determine the size and position of the target, and the accuracy requirements for the obtained regression features are low. Therefore, a smaller number of basic structures of the Transformer model decoder can be used to obtain the regression features.

[0081] In some possible implementations, the target in the image to be detected may be a small target or a target with an irregular shape (such as a plant with a complex shape). Determining the size and position of the target can be difficult, and the requirements for the obtained regression features are high. Therefore, the basic structure of a large number of Transformer model decoders can be used to obtain regression features.

[0082] In some possible implementations, the target in the image to be detected may be a target with relatively obvious features (such as the significant difference between bicycles and cars, which are relatively easy to distinguish). It is relatively easy to determine the category of the target, and the requirements for the acquired classification features are low. Therefore, a smaller number of basic structures of the Transformer model decoder can be used to obtain regression features.

[0083] In some possible implementations, the target in the image to be detected may be a target with less obvious features (such as the feature differences between different types of cars are relatively small and difficult to distinguish), making it difficult to determine the category of the target and requiring high-quality classification features. Therefore, a large number of basic structures of the decoder of the Transformer model can be used to obtain regression features.

[0084] In some possible implementations, the regression query can be obtained by mapping the global attention features to a preset query through the first decoder.

[0085] In some possible implementations, a second decoder can be used to map global attention features to a preset query to obtain a classification query.

[0086] In some possible implementations, the first decoder and the second decoder are cascaded.

[0087] In some possible implementations, the cascade order of the first decoder and the second decoder is such that the output of the second decoder is the input of the first decoder; that is, the second decoder is connected to the first decoder.

[0088] In some possible implementations, the cascade order of the first decoder and the second decoder is such that the output of the first decoder is the input of the second decoder; that is, the first decoder is connected to the second decoder.

[0089] In other words, in some possible implementations, the global attention features can be mapped to a preset query through the first decoder to obtain the regression query; the global attention features can be mapped to the regression query through the second decoder to obtain the classification query.

[0090] In step S340, in some possible implementations, the target may include any target object such as a vehicle, a person, an object, or an animal, and the category to which the target belongs may include categories such as vehicle, person, animal, or object.

[0091] In some possible implementations, the regression prediction layer can be a network that performs regression prediction, and the classification prediction layer can be a network that performs category prediction.

[0092] In some possible implementations, the regression prediction layer and the classification prediction layer can be FFN (Feed-Forward Network), and the regression prediction layer and the classification prediction layer can include multiple FFNs in series.

[0093] Since classification and regression tasks often require different features, classification tasks need higher-level semantic information to ensure that the target can be distinguished into different categories, and the learned features are often concentrated in the core salient region of the object; while regression tasks are to locate the size and position of the object, and the learned features are often in the edge region of the object, with the foreground region near the background being more prominent. The learned features are often distributed around the object rather than in the center.

[0094] The object detection method of this disclosure, based on the use of a classification network (i.e., a classification prediction layer) and a regression network (regression prediction layer) to process features separately, alleviates the problem of coupling between classification features and regression features to a certain extent. It uses different decoders to obtain classification features and regression features, thereby achieving complete decoupling between classification query and regression query, as well as between classification features and regression features. This allows the object detection network to focus on the expression of both classification features and regression features simultaneously, enhancing the feature expression capability of the object detection network and thus improving the object detection effect.

[0095] The target detection method of this disclosure will be described in detail below.

[0096] As described above, in the target detection method of this embodiment, the first decoder and the second decoder are cascaded.

[0097] In some possible implementations, the cascade order of the first decoder and the second decoder is such that the output of the second decoder is the input of the first decoder; that is, the second decoder is connected to the first decoder.

[0098] In some possible implementations, the cascade order of the first decoder and the second decoder is such that the output of the first decoder is the input of the second decoder; that is, the first decoder is connected to the second decoder.

[0099] The first decoder and the second decoder are cascaded to form the decoder of the target detection network. On the one hand, this can reduce the difference between the network structure of the target detection network used in the embodiments of this disclosure and the network structure of DETR, making the target detection network used in the embodiments of this disclosure easier to implement and apply, and easier to migrate to the target detection network based on DETR.

[0100] In some specific implementations, the object detection network of this disclosure embodiment can be migrated to DN-DETR to solve the problem of slow convergence speed of the DETR network. Specifically, additional denoising tasks can be added to the object detection network used in this disclosure embodiment to achieve network optimization stability and accelerate network convergence speed.

[0101] In some specific implementations, the object detection network of this disclosure embodiment can be migrated to DAB-DETR to solve the problem that the concept of query in DETR is ambiguous, relying purely on the network to learn without clear semantic information. Specifically, based on the object detection network used in this disclosure embodiment, the query can be explicitly modeled into an anchor in the object detector, giving it clear semantic information.

[0102] The first and second decoders are cascaded to form the decoder of the object detection network. On the other hand, it can also reduce the number of preset queries (because only the first decoder needs to be set with preset queries), thereby reducing the resource consumption of the object detection network and speeding up the processing speed of the object detection network.

[0103] In some possible implementations, a regression query can be obtained by mapping global attention features to a preset query using a first decoder; or a classification query can be obtained by mapping global attention features to a regression query using a second decoder.

[0104] Figure 4 This illustration shows a flowchart of an object detection method provided in this disclosure, in which a first decoder maps global attention features to a preset query to obtain a regression query, and a second decoder maps global attention features to the regression query to obtain a classification query. Figure 4 As shown, the method mainly includes steps S410 and S420.

[0105] In step S410, the global attention features are mapped to a preset query feature vector through the first decoder of the object detection network to obtain the regression features of the image to be detected;

[0106] In step S420, the global attention features are mapped to regression features through the second decoder of the object detection network to obtain the classification features of the image to be detected.

[0107] In some possible implementations, in step S410, mapping global attention to a preset query can be achieved by determining content features (i.e., V value) and key features (i.e., K value) based on the global attention features output by the encoder, determining query features (i.e., Q value) based on the query feature vector, and performing cross-attention processing based on the content features, key features, and query features to complete the feature mapping and obtain regression features (i.e., regression query).

[0108] In some possible implementations, cross-attention processing based on content features, key-value features, and query features is performed to complete feature mapping. This can be achieved by determining attention weights based on query features and key-value features, and then determining attention features based on attention weights and content features, thus completing feature mapping.

[0109] In some possible implementations, in step S420, mapping global attention to regression features can be achieved by determining content features (i.e., V value) and key features (i.e., K value) based on the global attention features output by the encoder, determining query features (i.e., Q value) based on the regression feature vector, and performing cross-attention processing based on content features, key features, and query features to complete feature mapping and obtain classification features (i.e., classification query).

[0110] In some possible implementations, cross-attention processing based on content features, key-value features, and query features is performed to complete feature mapping. This can be achieved by determining attention weights based on query features and key-value features, and then determining attention features based on attention weights and content features, thus completing feature mapping.

[0111] Obtaining the regression query first, and then the classification query, is equivalent to continuing feature extraction on the regression query to obtain deeper semantic information. This aligns with the characteristics of regression tasks requiring shallow boundary information, while classification tasks require high-level semantic information. Therefore, object detection networks that obtain the regression query first and then the classification query have better feature representation capabilities and object detection performance.

[0112] Meanwhile, the regression query obviously contains more information than the preset query. For the second encoder, it is easier to obtain the semantic features required for classification than to perform feature mapping on the regression query based on the preset query. Therefore, its structure can be simpler.

[0113] In some possible implementations, the basic structures of the first decoder and the second decoder can be the same, both being the basic structures of the decoder of the Transformer model. Both the first decoder and the second decoder can be composed of multiple cascaded basic structures of the decoder of the Transformer model.

[0114] In some specific implementations, the number of basic structures of the Transformer model decoders included in the first and second decoders can be determined based on the attributes of the target in the image to be detected.

[0115] In some possible implementations, the target of the image to be detected is a relatively regular shape (such as a table, a book, or other shaped target). It is relatively easy to determine the size and position of the target, and the accuracy requirements for the obtained regression features are low. Therefore, a smaller number of basic structures of the Transformer model decoder can be used to obtain the regression features.

[0116] In some possible implementations, the target in the image to be detected may be a small target or a target with an irregular shape (such as a plant with a complex shape). Determining the size and position of the target can be difficult, and the requirements for the obtained regression features are high. Therefore, the basic structure of a large number of Transformer model decoders can be used to obtain regression features.

[0117] In some possible implementations, the target in the image to be detected may be a target with relatively obvious features (such as the significant difference between bicycles and cars, which are relatively easy to distinguish). It is relatively easy to determine the category of the target, and the requirements for the acquired classification features are low. Therefore, a smaller number of basic structures of the Transformer model decoder can be used to obtain regression features.

[0118] In some possible implementations, the target in the image to be detected may be a target with less obvious features (such as the feature differences between different types of cars are relatively small and difficult to distinguish), making it difficult to determine the category of the target and requiring high-quality classification features. Therefore, a large number of basic structures of the decoder of the Transformer model can be used to obtain regression features.

[0119] In some specific implementations, both the first decoder and the second decoder can include a basic structure of three cascaded Transformer models. That is, the decoder of the object detection network used in this embodiment has a total of six basic layers, and the decoders of other DETR models have the same number of basic layers. Therefore, the time required for object detection using the object detection method of this embodiment is the same as the detection time required by DETR, and it does not increase the detection time.

[0120] In some specific implementations, depending on the attributes of the target in the image to be detected, if the number of basic structures contained in the first decoder and the second decoder can be reduced, the detection speed can be accelerated and the detection time can be reduced.

[0121] The basic structure of the decoder of the Transformer model can be the basic structure of the decoder of any network model based on the Transformer model, which will not be elaborated here.

[0122] Figure 5 This illustration shows a schematic diagram of a target detection network in a specific embodiment of the target detection method provided in this disclosure, with reference to... Figure 5 The image to be detected is used as the input, fed into the backbone network to extract image features, and then fed into the encoder. Global attention features are obtained based on the extracted features and fed into the REG Decoder (the first decoder). These global attention features are then mapped to the query (query feature vector). Figure 5 (A circle filled with diagonal lines) is used to obtain the query containing regression features (i.e., a circle with diagonal lines in the middle). Figure 5 The query containing regression features is input into the CLA Decoder (i.e., the second decoder), which maps the global attention features to the query containing regression features, thus obtaining the query containing classification features (i.e., the query containing classification features). Figure 5 (A circle filled with vertical lines).

[0123] The prediction box is obtained by taking the regression feature query and inputting the Regression Head (i.e., the regression prediction layer).

[0124] Input the query containing classification features into the Classification Head (i.e., the classification prediction layer) to obtain the category to which the target in the prediction box belongs.

[0125] Figure 6 A flowchart illustrating the training method of the target detection model provided in this disclosure is shown, as follows: Figure 6As shown, the method mainly includes steps S610, S620, S630, and S640.

[0126] In step S610, the image to be trained, the position of the target box corresponding to the target in the image to be trained, and the category to which the target in the image to be trained are obtained;

[0127] In step S620, feature extraction is performed on the image to be trained to obtain the image feature map of the image to be trained; the image feature map is encoded by the encoder of the object detection network to obtain the global attention features of the image to be trained.

[0128] In step S630, the global attention features are mapped using the first decoder of the object detection network to obtain the regression features of the image to be trained; the global attention features are mapped using the second decoder of the object detection network to obtain the classification features of the image to be trained.

[0129] In step S640, regression loss and classification loss are determined based on regression features and classification features; the target detection network is trained based on regression loss and classification loss.

[0130] For example, in step S610, the image to be trained can be an image containing the target. The image to be trained can be an image collected online, such as a target image collected online through web crawling technology. Alternatively, the image to be trained can be an image collected offline, or it can be an image that has been artificially synthesized (e.g., processed by rotation, scaling, etc.). This disclosure does not limit the scope of the embodiments.

[0131] It should be understood that the training image can also be a single frame from a video. The training image can be extracted from the video, where the video can be referred to as the training image. The method of obtaining the training image is similar to that of obtaining the detection image, and will not be elaborated here.

[0132] In some possible implementations, the target in the image to be trained can include any object such as a vehicle, person, object, or animal, and the category to which the target belongs can include categories such as vehicle, person, animal, or object. The bounding box corresponding to the target can be the largest bounding rectangle containing the target.

[0133] In some possible implementations, in step S620, conventional image processing methods can be used to extract features from the image to be trained and obtain the image feature map of the image to be trained.

[0134] In some possible implementations, in order to improve the accuracy and reliability of feature extraction results, neural networks can be used to extract features from the training image to obtain the image feature map of the training image.

[0135] In some possible implementations, mainstream CNN networks can be used to extract features from the training images to obtain image feature maps. For example, the ResNet (residual network) series (such as ResNet 34, ResNet50, ResNet101, etc.) and the DarkNet (an open-source neural network framework written in C and CUDA) series (such as DarkNet19, DarkNet53) can be used.

[0136] In some possible implementations, the CNN network that extracts the images to be trained is used as the backbone of the object detection network and trained together with the other parts of the object detection network.

[0137] In some possible implementations, in order to balance the accuracy of feature extraction results with reducing resource consumption and detection speed, a suitable backbone can be selected for feature extraction of the training images based on the application scenario.

[0138] In some specific implementations, the backbone network structure can be selected based on the application scenario, from lightweight structures (such as ResNet18, ResNet34, DarkNet19, etc.), medium-weight structures (such as ResNet50, ResNeXt50 (ResNeXt is a combination of ResNet and Inception (a type of convolutional neural network), DarkNet53, etc.), to heavyweight structures (such as ResNet101, ResNeXt152).

[0139] In some possible implementations, the encoder of the object detection network can be an encoder based on the Transformer model or an encoder based on a variant of the Transformer model.

[0140] In some possible implementations, the encoder's network structure is consistent with the encoder structure of the DETR network in related technologies.

[0141] In some possible implementations, the encoder encodes the image feature map. This can be achieved by the encoder calculating the correlation between each feature point in the image feature map and other feature points in the image feature map, thereby concentrating the information of other feature points into that feature point and obtaining global attention features.

[0142] In some specific implementations, the image feature map can be reduced in dimensionality by performing a 1×1 convolution before being input into the encoder.

[0143] In some specific implementations, the encoder uses self-attention and multi-head self-attention mechanisms to perform global analysis on the image feature map, extract the interrelationships between different physical objects at different locations, and obtain global attention features.

[0144] In step S630, in some possible implementations, the first decoder and the second decoder can be decoders based on the Transformer model or decoders based on a variant of the Transformer model.

[0145] In some possible implementations, global attention features can be mapped to a predefined query feature vector.

[0146] In some possible implementations, the number of queries can be preset.

[0147] In some possible implementations, the first decoder and the second decoder can be decoders with different structures and weights.

[0148] In some possible implementations, the basic structures of the first decoder and the second decoder can be the same, both being the basic structures of the decoder of the Transformer model. Both the first decoder and the second decoder can be composed of multiple cascaded basic structures of the decoder of the Transformer model.

[0149] In some specific implementations, the number of basic structures of the Transformer model decoders included in the first and second decoders can be determined based on the attributes of the target in the image to be trained.

[0150] In some possible implementations, the target of the image to be trained is a relatively regular shape (such as a table, a book, or other objects with a defined shape). It is relatively easy to determine the size and position of the target, and the accuracy requirements for the obtained regression features are low. Therefore, a smaller number of basic structures of the Transformer model decoder can be used to obtain the regression features.

[0151] In some possible implementations, the target of the image to be trained may be a small target or a target with an irregular shape (such as a plant with a complex shape). It is difficult to determine the size and position of the target and the requirements for the obtained regression features are high. Therefore, the basic structure of the decoder of a large number of Transformer models can be used to obtain regression features.

[0152] In some possible implementations, the target of the image to be trained may be a target with obvious features (such as bicycles and cars, which have significant feature differences and are relatively easy to distinguish). It is relatively easy to determine the category of the target and the requirements for the obtained classification features are low. Therefore, a smaller number of basic structures of the decoder of the Transformer model can be used to obtain regression features.

[0153] In some possible implementations, the target of the image to be trained may be a target with less obvious features (such as the feature differences between different types of cars are relatively small and not easy to distinguish), making it difficult to determine the category of the target and requiring high-quality classification features. Therefore, a large number of basic structures of the decoder of the Transformer model can be used to obtain regression features.

[0154] In some specific implementations, both the first decoder and the second decoder can include a basic structure of three cascaded Transformer models. That is, the decoder of the object detection network used in this embodiment has a total of six basic layers, and the decoders of other DETR models have the same number of basic layers. Therefore, the time required for object detection using the object detection method of this embodiment is the same as the detection time required by DETR, and it does not increase the detection time.

[0155] In some specific implementations, depending on the attributes of the target in the image to be trained, if the number of basic structures contained in the first decoder and the second decoder can be reduced, the detection speed can be accelerated and the detection time can be reduced.

[0156] The basic structure of the decoder of the Transformer model can be the basic structure of the decoder of any network model based on the Transformer model, which will not be elaborated here.

[0157] In some possible implementations, the regression query can be obtained by mapping the global attention features to a preset query through the first decoder.

[0158] In some possible implementations, a second decoder can be used to map global attention features to a preset query to obtain a classification query.

[0159] In some possible implementations, the first decoder and the second decoder are cascaded.

[0160] The first decoder and the second decoder are cascaded to form the decoder of the target detection network. On the one hand, this can reduce the difference between the network structure of the target detection network used in the embodiments of this disclosure and the network structure of DETR, making the target detection network used in the embodiments of this disclosure easier to implement and apply, and easier to migrate to the target detection network based on DETR.

[0161] In some specific implementations, the object detection network of this disclosure embodiment can be migrated to DN-DETR to solve the problem of slow convergence speed of the DETR network. Specifically, an additional denoising task can be added to the object detection network used in this disclosure embodiment to achieve network optimization stability and accelerate the convergence speed of the network training process.

[0162] In some specific implementations, the object detection network of this disclosure embodiment can be migrated to DAB-DETR to solve the problem that the concept of query in DETR is ambiguous, relying purely on the network to learn without clear semantic information. Specifically, based on the object detection network used in this disclosure embodiment, the query can be explicitly modeled into an anchor in the object detector, giving it clear semantic information.

[0163] The first and second decoders are cascaded to form the decoder of the object detection network. On the other hand, it can also reduce the number of preset queries (because only the first decoder needs to be set with preset queries), thereby reducing the resource consumption of the object detection network and speeding up the training speed of the object detection network.

[0164] In some possible implementations, the cascade order of the first decoder and the second decoder is such that the output of the second decoder is the input of the first decoder; that is, the second decoder is connected to the first decoder.

[0165] In some possible implementations, the cascade order of the first decoder and the second decoder is such that the output of the first decoder is the input of the second decoder; that is, the first decoder is connected to the second decoder.

[0166] In other words, in some possible implementations, the global attention features can be mapped to a preset query through the first decoder to obtain the regression query; the global attention features can be mapped to the regression query through the second decoder to obtain the classification query.

[0167] In some possible implementations, the global attention features are mapped to a preset query feature vector through the first decoder of the object detection network to obtain the regression features of the image to be trained; the global attention features are mapped to the regression features through the second decoder of the object detection network to obtain the classification features of the image to be trained.

[0168] In some possible implementations, mapping global attention to a preset query can be achieved by determining content features (i.e., V value) and key features (i.e., K value) based on the global attention features output by the encoder, determining query features (i.e., Q value) based on the query feature vector, and performing cross-attention processing based on the content features, key features, and query features to complete the feature mapping and obtain regression features (i.e., regression query).

[0169] In some possible implementations, cross-attention processing based on content features, key-value features, and query features is performed to complete feature mapping. This can be achieved by determining attention weights based on query features and key-value features, and then determining attention features based on attention weights and content features, thus completing feature mapping.

[0170] In some possible implementations, mapping global attention to regression features can be achieved by determining content features (i.e., V value) and key features (i.e., K value) based on the global attention features output by the encoder, determining query features (i.e., Q value) based on the regression feature vector, and performing cross-attention processing based on content features, key features, and query features to complete feature mapping and obtain classification features (i.e., classification query).

[0171] In some possible implementations, cross-attention processing based on content features, key-value features, and query features is performed to complete feature mapping. This can be achieved by determining attention weights based on query features and key-value features, and then determining attention features based on attention weights and content features, thus completing feature mapping.

[0172] Obtaining the regression query first, and then the classification query, is equivalent to continuing feature extraction on the regression query to obtain deeper semantic information. This aligns with the characteristics of regression tasks requiring shallow boundary information, while classification tasks require high-level semantic information. Therefore, object detection networks that obtain the regression query first and then the classification query have better feature representation capabilities and object detection performance.

[0173] Meanwhile, the regression query obviously contains more information than the preset query. For the second encoder, it is easier to obtain the semantic features required for classification than to perform feature mapping on the regression query based on the preset query. Therefore, its structure can be simpler.

[0174] In some possible implementations, in step S640, the position of the predicted box is obtained by inputting the regression features into the regression prediction layer of the target detection network, and the regression loss is determined based on the position of the predicted box and the position of the target box.

[0175] By inputting classification features into the classification prediction layer of the object detection network, the category to which the object within the prediction box belongs is obtained, and the classification loss is determined based on the category to which the object in the training image belongs and the category to which the object within the prediction box belongs.

[0176] The regression prediction layer can be a network that performs regression prediction, and the classification prediction layer can be a network that performs category prediction.

[0177] In some possible implementations, the regression prediction layer and the classification prediction layer can be FFN (Feed-Forward Network), and the regression prediction layer and the classification prediction layer can include multiple FFNs in series.

[0178] Among some possible implementations, refer to Figure 5 The Regression Loss is determined based on the difference between the positions of the predicted bounding box and the target bounding box; the Classification Loss is determined based on the difference between the category of the target in the training image and the category of the target within the predicted bounding box.

[0179] Backpropagation is performed based on Classification Loss and Regression Loss to train the encoder, first decoder, second decoder, classification prediction layer, and regression prediction layer. In some possible implementations, the backbone network is also a component of the object detection network, in which case backpropagation is also used to train the backbone network.

[0180] That is to Figure 5 The system is trained using the following components: Backbone, Encoder, REG Decoder, CLA Decoder, Classification Head, and Regression Head.

[0181] Since classification and regression tasks often require different features, classification tasks need higher-level semantic information to ensure that the target can be distinguished into different categories, and the learned features are often concentrated in the core salient region of the object; while regression tasks are to locate the size and position of the object, and the learned features are often in the edge region of the object, with the foreground region near the background being more prominent. The learned features are often distributed around the object rather than in the center.

[0182] The training method of the object detection model in this embodiment alleviates the coupling problem between classification features and regression features to some extent by processing features through a classification network (i.e., classification prediction layer) and a regression network (regression prediction layer) respectively. It uses different decoders to obtain classification features and regression features, thereby achieving complete decoupling between classification query and regression query, as well as between classification features and regression features. This allows the object detection network to focus on the expression of both classification features and regression features simultaneously, enhancing the feature expression capability of the object detection network and thus improving the object detection effect of the object detection network.

[0183] Based on and Figure 3 The method shown follows the same principle. Figure 7 A schematic diagram of the structure of a target detection device provided in an embodiment of this disclosure is shown, as follows: Figure 7 As shown, the target detection device 70 may include:

[0184] The backbone network module 710 is used to acquire the image to be detected, extract features from the image to be detected, and obtain the image feature map of the image to be detected.

[0185] The encoder module 720 is used to encode the image feature map through the encoder of the pre-trained object detection network to obtain the global attention features of the image to be detected.

[0186] The decoder module 730 is used to perform feature mapping on the global attention features through the first decoder of the object detection network to obtain the regression features of the image to be detected; and to perform feature mapping on the global attention features through the second decoder of the object detection network to obtain the classification features of the image to be detected.

[0187] The prediction module 740 is used to input regression features into the regression prediction layer of the object detection network to obtain the position of the prediction box; and to input classification features into the classification prediction layer of the object detection network to obtain the category to which the object in the prediction box belongs.

[0188] In the target detection apparatus of this disclosure, while using a classification network (i.e., a classification prediction layer) and a regression network (regression prediction layer) to process features to a certain extent to alleviate the problem of coupling between classification features and regression features, different decoders are used to obtain classification features and regression features. This achieves complete decoupling of classification query and regression query, as well as complete decoupling of classification features and regression features. This allows the target detection network to focus on the expression of both classification features and regression features simultaneously, enhancing the feature expression capability of the target detection network and thus improving the target detection effect of the target detection network.

[0189] It is understood that the above-described modules of the target detection device in the embodiments of this disclosure have the ability to implement... Figure 3 The embodiments shown illustrate the functions of corresponding steps in the target detection method. These functions can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions. These modules can be software and / or hardware, and each module can be implemented individually or integrated from multiple modules. For a detailed description of the functions of each module in the aforementioned recall device, please refer to [link to relevant documentation]. Figure 3 The corresponding descriptions of the target detection methods in the embodiments shown are not repeated here.

[0190] Based on and Figure 6 The method shown follows the same principle. Figure 8 This diagram illustrates the structure of a training apparatus for an object detection model according to an embodiment of the present disclosure. Figure 8 As shown, the training device 80 for the object detection model may include:

[0191] The data acquisition module 810 is used to acquire the image to be trained, the position of the target box corresponding to the target in the image to be trained, and the category to which the target in the image to be trained belongs;

[0192] The feature training module 820 is used to extract features from the image to be trained and obtain the image feature map of the image to be trained; the encoder of the object detection network encodes the image feature map to obtain the global attention features of the image to be trained.

[0193] The decoding training module 830 is used to perform feature mapping on the global attention features through the first decoder of the object detection network to obtain the regression features of the image to be trained; and to perform feature mapping on the global attention features through the second decoder of the object detection network to obtain the classification features of the image to be trained.

[0194] The backpropagation module 840 is used to determine the regression loss and classification loss based on the regression features and classification features; and to train the object detection network based on the regression loss and classification loss.

[0195] In the training apparatus of the object detection model in this embodiment, the problem of coupling between classification features and regression features is alleviated to some extent by processing features through a classification network (i.e., classification prediction layer) and a regression network (regression prediction layer). Different decoders are used to obtain classification features and regression features, thereby achieving complete decoupling of classification query and regression query, as well as complete decoupling of classification features and regression features. This allows the model training network to focus on the expression of both classification features and regression features simultaneously, enhancing the feature expression capability of the model training network and thus improving the model training effect.

[0196] It is understood that the above-described modules of the training device for the target detection model in the embodiments of this disclosure have the ability to implement... Figure 6 The embodiments shown illustrate the functionality of the training method for the target detection model and its corresponding steps. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions. These modules can be software and / or hardware, and each module can be implemented individually or integrated from multiple modules. For a detailed description of the functions of each module in the recall device, please refer to [link to relevant documentation]. Figure 6 The corresponding description of the training method of the target detection model in the illustrated embodiment will not be repeated here.

[0197] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0198] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0199] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the target detection method and the target detection model training method provided in the embodiments of this disclosure.

[0200] Compared with existing technologies, this electronic device alleviates the coupling problem between classification and regression features to some extent by processing features separately through a classification network (i.e., a classification prediction layer) and a regression network (regression prediction layer). Furthermore, it uses different decoders to obtain classification and regression features, achieving complete decoupling between classification queries and regression queries, as well as between classification features and regression features. This allows the model training network to focus on the expression of both classification and regression features simultaneously, enhancing the feature representation capability of the model training network and thus improving the model training effect.

[0201] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the target detection method and the target detection model training method provided in the embodiments of this disclosure.

[0202] Compared with existing technologies, this readable storage medium alleviates the coupling problem between classification and regression features to some extent by processing features separately through a classification network (i.e., a classification prediction layer) and a regression network (regression prediction layer). Furthermore, it uses different decoders to obtain classification and regression features, achieving complete decoupling of classification queries and regression queries, as well as complete decoupling of classification features and regression features. This allows the model training network to focus on the expression of both classification and regression features simultaneously, enhancing the feature expression capability of the model training network and thus improving the model training effect.

[0203] The computer program product includes a computer program that, when executed by a processor, implements the target detection method and the target detection model training method provided in the embodiments of this disclosure.

[0204] Compared with existing technologies, this computer program product alleviates the coupling problem between classification and regression features to some extent by processing features separately through a classification network (i.e., classification prediction layer) and a regression network (regression prediction layer). Furthermore, it uses different decoders to obtain classification and regression features, achieving complete decoupling of classification queries and regression queries, as well as complete decoupling of classification features and regression features. This allows the model training network to focus on the expression of both classification and regression features simultaneously, enhancing the feature expression capability of the model training network and thus improving the model training effect.

[0205] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0206] like Figure 9As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0207] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0208] The computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as object detection methods and / or object detection model training methods. For example, in some embodiments, the object detection methods and object detection model training methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the object detection methods and object detection model training methods described above can be performed. Alternatively, in other embodiments, the computing unit 901 may be configured by any other suitable means (e.g., by means of firmware) to perform object detection methods and / or training methods for object detection models.

[0209] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0210] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0211] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0212] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0213] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0214] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0215] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0216] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A target detection method, comprising: Acquire the image to be detected, extract features from the image to be detected, and obtain the image feature map of the image to be detected; The image feature map is encoded by the encoder of a pre-trained object detection network to obtain the global attention features of the image to be detected. The first decoder of the target detection network determines the first content feature and the first key value feature based on the global attention feature, determines the first query feature based on the preset query feature vector, and performs cross-attention processing based on the first content feature, the first key value feature and the first query feature to complete feature mapping and obtain the regression feature of the image to be detected. The second decoder of the target detection network determines the second content feature and the second key value feature based on the global attention feature, determines the second query feature based on the regression feature vector, and performs cross-attention processing based on the second content feature, the second key value feature and the second query feature to complete feature mapping and obtain the classification feature of the image to be detected. The first decoder and the second decoder are cascaded, and the cascade order of the first decoder and the second decoder is such that the output of the first decoder is the input of the second decoder. The regression features are input into the regression prediction layer of the target detection network to obtain the position of the prediction box; the classification features are input into the classification prediction layer of the target detection network to obtain the category to which the target within the prediction box belongs.

2. The method according to claim 1, wherein, The structure of the first decoder is the same as that of the second decoder.

3. The method according to claim 1, wherein, The step of extracting features from the image to be detected to obtain an image feature map of the image to be detected includes: The target detection network extracts features from the image to be detected through its main branches, thereby obtaining the image feature map of the image to be detected.

4. A method for training an object detection model, comprising: Obtain the image to be trained, the position of the bounding box corresponding to the target in the image to be trained, and the category to which the target in the image to be trained belongs; Feature extraction is performed on the image to be trained to obtain the image feature map of the image to be trained; The image feature map is encoded by the encoder of the object detection network to obtain the global attention features of the image to be trained; The first decoder of the target detection network determines the first content feature and the first key-value feature based on the global attention feature, determines the first query feature based on the preset query feature vector, and performs cross-attention processing based on the first content feature, the first key-value feature and the first query feature to complete feature mapping and obtain the regression features of the image to be trained. The second decoder of the target detection network determines the second content feature and the second key value feature based on the global attention feature, determines the second query feature based on the regression feature vector, and performs cross-attention processing based on the second content feature, the second key value feature and the second query feature to complete feature mapping and obtain the classification features of the image to be trained. The first decoder and the second decoder are cascaded, and the cascade order of the first decoder and the second decoder is such that the output of the first decoder is the input of the second decoder. The regression loss and classification loss are determined based on the regression features and the classification features; the target detection network is trained based on the regression loss and the classification loss.

5. The method according to claim 4, wherein, The determination of regression loss and classification loss based on the regression features and the classification features includes: The regression features are input into the regression prediction layer of the target detection network to obtain the position of the prediction box; the classification features are input into the classification prediction layer of the target detection network to obtain the category to which the target in the prediction box belongs. The regression loss is determined based on the position of the predicted bounding box and the position of the target bounding box; the classification loss is determined based on the category of the target in the image to be trained and the category of the target within the predicted bounding box.

6. The method according to claim 4, wherein, The structure of the first decoder is the same as that of the second decoder.

7. The method according to claim 4, wherein, The target detection network also includes a backbone branch; The step of extracting features from the image to be trained to obtain an image feature map of the image to be trained includes: The target detection network is used to extract features from the image to be trained, thereby obtaining the image feature map of the image to be trained.

8. A target detection device, comprising: The backbone network module is used to acquire the image to be detected, extract features from the image to be detected, and obtain the image feature map of the image to be detected. The encoder module is used to encode the image feature map through the encoder of a pre-trained object detection network to obtain the global attention features of the image to be detected. The decoder module is used to determine a first content feature and a first key-value feature based on the global attention feature through the first decoder of the target detection network, determine a first query feature based on a preset query feature vector, and perform cross-attention processing based on the first content feature, the first key-value feature and the first query feature to complete feature mapping and obtain the regression features of the image to be detected. The second decoder of the target detection network determines the second content feature and the second key value feature based on the global attention feature, determines the second query feature based on the regression feature vector, and performs cross-attention processing based on the second content feature, the second key value feature and the second query feature to complete feature mapping and obtain the classification feature of the image to be detected. The first decoder and the second decoder are cascaded, and the cascade order of the first decoder and the second decoder is such that the output of the first decoder is the input of the second decoder. The prediction module is used to input the regression features into the regression prediction layer of the target detection network to obtain the position of the prediction box; and to input the classification features into the classification prediction layer of the target detection network to obtain the category to which the target in the prediction box belongs.

9. A training device for an object detection model, comprising: The data acquisition module is used to acquire the image to be trained, the position of the target box corresponding to the target in the image to be trained, and the category to which the target in the image to be trained belongs; The feature training module is used to extract features from the image to be trained and obtain the image feature map of the image to be trained. The image feature map is encoded by the encoder of the object detection network to obtain the global attention features of the image to be trained; The decoding training module is used to determine the first content feature and the first key-value feature based on the global attention feature through the first decoder of the target detection network, determine the first query feature based on the preset query feature vector, and perform cross-attention processing based on the first content feature, the first key-value feature and the first query feature to complete feature mapping and obtain the regression feature of the image to be trained. The second decoder of the target detection network determines the second content feature and the second key value feature based on the global attention feature, determines the second query feature based on the regression feature vector, and performs cross-attention processing based on the second content feature, the second key value feature and the second query feature to complete feature mapping and obtain the classification features of the image to be trained. The first decoder and the second decoder are cascaded, and the cascade order of the first decoder and the second decoder is such that the output of the first decoder is the input of the second decoder. The backpropagation module is used to determine regression loss and classification loss based on the regression features and the classification features; and to train the object detection network based on the regression loss and the classification loss.

10. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the target detection method of any one of claims 1-3 and the training method of the target detection model of any one of claims 4-7.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the target detection method according to any one of claims 1-3 and the training method of the target detection model according to any one of claims 4-7.

12. A computer program product comprising a computer program that, when executed by a processor, implements the target detection method according to any one of claims 1-3 and the training method for the target detection model according to any one of claims 4-7.

Images