Target detection method and device, electronic equipment and computer readable storage medium

Abstract

The present disclosure provides a target detection method and device, electronic equipment and computer readable storage medium, relates to the technical field of image processing, and particularly relates to the technical field of deep learning and target detection. The specific implementation scheme is as follows: multi-scale feature extraction is performed on a to-be-detected image to obtain multiple scale features corresponding to different scales, the multiple scale features include one deep feature and at least one shallow feature, and the scale corresponding to the shallow feature is smaller than the scale corresponding to the deep feature; cross-attention calculation is performed based on the multiple scale features to obtain a global semantic feature; the global semantic feature and the shallow feature are fused to obtain a detection feature; and a target class and a target position in the to-be-detected image are obtained according to the detection feature.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more particularly to the fields of deep learning and object detection technology. Specifically, this disclosure relates to an object detection method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Object detection is an important problem in the field of computer vision. In recent years, with the development of deep learning technology, object detection methods based on deep neural networks have become a research hotspot.

[0003] One of the major challenges in object detection is that images contain objects of different scales, and features of a single scale cannot cover objects of different sizes. Summary of the Invention

[0004] This disclosure provides a target detection method, apparatus, electronic device, and computer-readable storage medium.

[0005] According to a first aspect of this disclosure, a method for object detection is provided, the method comprising:

[0006] Multi-scale feature extraction is performed on the image to be detected to obtain multiple scale features corresponding to different scales. The multiple scale features include a deep feature and at least one shallow feature, and the scale corresponding to the shallow feature is smaller than the scale corresponding to the deep feature.

[0007] Based on multiple scale features, cross-attention calculation is performed to obtain global semantic features;

[0008] The global semantic features and the shallow features are fused to obtain the detection features;

[0009] The target category and target location in the image to be detected are obtained based on the detection features.

[0010] According to a second aspect of this disclosure, an apparatus for target detection is provided, the apparatus comprising:

[0011] The feature extraction module is used to perform multi-scale feature extraction on the image to be detected, and obtain multiple scale features corresponding to different scales. The multiple scale features include a deep feature and at least one shallow feature, and the scale corresponding to the shallow feature is smaller than the scale corresponding to the deep feature.

[0012] The attention module is used to perform cross-attention calculation based on multiple scale features to obtain global semantic features;

[0013] The feature fusion module is used to fuse the global semantic features with the shallow features to obtain detection features;

[0014] The detection module is used to obtain the target category and target location in the image to be detected based on the detection features.

[0015] According to a third aspect of this disclosure, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to at least one of the aforementioned processors; wherein,

[0018] The memory stores instructions that can be executed by at least one processor, which, when executed by at least one processor, enables the at least one processor to perform the target detection method.

[0019] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the above-described target detection method.

[0020] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method for target detection.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0023] Figure 1 This is a schematic flowchart of a target detection method provided in an embodiment of this disclosure;

[0024] Figure 2 This is a flowchart illustrating some steps of a target detection method provided in an embodiment of this disclosure;

[0025] Figure 3 This is a flowchart illustrating some steps of a target detection method provided in an embodiment of this disclosure;

[0026] Figure 4 This is a flowchart illustrating some steps of a target detection method provided in an embodiment of this disclosure;

[0027] Figure 5 This is a schematic diagram of a specific embodiment of a target detection method provided in this disclosure;

[0028] Figure 6This is a schematic diagram of the structure of a target detection device provided in an embodiment of this disclosure;

[0029] Figure 7 This is a block diagram of an electronic device used to implement the target detection method of the embodiments of this disclosure. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] In some related technologies, the Neck network is used to fuse features of different scales output by the Backbone network, and predictions are made on the fused multi-scale features.

[0032] In some related technologies, FPN (Feature Pyramid Networks) is used to fuse deep features with shallow features by progressively upsampling them, and then a Head is used on the feature maps of the three scales after fusion to detect the type and location of the target.

[0033] Deep features contain rich semantic information, while shallow features contain more object localization information. The propagation path from shallow features to deep features in FPN is too long, which causes the object localization information to be lost during the feature fusion process.

[0034] In some related technologies, PAN (Path Aggregation Network) shortens the information propagation path through bottom-up path enhancement, while using the precise positioning information of low-level features to locate targets at various scales.

[0035] However, because PAN adds feature propagation paths, its computational cost increases significantly.

[0036] In some related technologies, BiFPN (Bidirectional Feature Pyramid Network) reduces the amount of computation by removing nodes with single input edges, but at the same time, it leads to insufficient multi-scale feature fusion and difficulty in global modeling of features at different scales, thus limiting the network's feature extraction capabilities.

[0037] The target detection method, apparatus, electronic device, and computer-readable storage medium provided in this disclosure are intended to solve at least one of the above-mentioned technical problems of the prior art.

[0038] The target detection method provided in this disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0039] Figure 1 A schematic flowchart of a target detection method provided in an embodiment of this disclosure is shown, as follows: Figure 1 As shown, the method may include steps S110, S120, S130, and S140.

[0040] In step S110, multi-scale feature extraction is performed on the image to be detected to obtain multiple scale features corresponding to different scales. The multiple scale features include a deep feature and at least one shallow feature. The scale corresponding to the shallow feature is smaller than the scale corresponding to the deep feature.

[0041] In step S120, cross-attention calculation is performed based on multiple scale features to obtain global semantic features;

[0042] In step S130, global semantic features and shallow features are fused to obtain detection features;

[0043] In step S140, the target category and target location in the image to be detected are obtained based on the detection features.

[0044] For example, in step S110, multi-scale feature extraction of the image to be detected can be achieved by sampling the image at different scales.

[0045] In some possible implementations, multi-scale feature extraction can be performed on the image to be detected using a pre-trained backbone network.

[0046] The backbone network is a convolutional neural network that aggregates image features at a fine-grained level across different images.

[0047] In some possible implementations, the backbone can be a convolutional neural network capable of image feature extraction, such as VGGNet (Visual Geometry GroupNet), ResNet (Residual Network), or DenseNet (Dense Convolutional Network). This disclosure does not limit the backbone network.

[0048] In some possible implementations, multi-scale feature extraction can be used to obtain scale features at multiple different scales.

[0049] Among the scale features obtained at different scales, the feature with the largest corresponding scale is the deep feature; features at other scales are shallow features.

[0050] In some specific implementations, when using a pre-trained backbone network to extract multi-scale features from the image to be detected, scale features of different scales can be obtained by acquiring the outputs of different network layers. For example, feature maps of different sizes output by different network layers can be used as scale features of different scales, the feature map output by the deepest network layer can be used as deep features, and the feature maps output by other network layers can be used as shallow features.

[0051] In some possible implementations, after obtaining the outputs of different network layers, a 1*1 convolution is used to uniformly project the outputs to a lower dimension, thereby reducing computational overhead and realizing the transformation of the feature space, using the features after the 1*1 convolution as scale features of different scales.

[0052] In some possible implementations, in step S120, a pre-trained attention network can be used to perform cross-attention calculation.

[0053] Global semantic features can be obtained by performing cross-attention calculations among features at multiple scales.

[0054] In some possible implementations, cross-attention calculations can be performed on each scale feature and other scale features to obtain the attention features corresponding to that scale feature, and then the global semantic features can be obtained through the attention features corresponding to each scale feature.

[0055] In some possible implementations, for scale features, only feature points containing objects can contribute to the effectiveness of the acquired detection features by participating in the interaction and fusion of features at different scales. However, feature points in background regions without objects will not only fail to contribute to the effectiveness of the acquired detection features by participating in the interaction and fusion of features at different scales, but will also increase the computational cost. Therefore, before performing cross-attention calculation, pooling is performed on scale features to obtain feature points containing objects, and global semantic features are obtained by performing cross-attention calculation on feature points containing objects.

[0056] In some possible implementations, in step S130, global semantic features and deep features can be propagated to shallow features in a top-down manner, so that shallow features can be fused with global semantic features and deep features to make up for the lack of semantic information in shallow features and improve the target extraction capability of shallow features.

[0057] In some possible implementations, FPN can be used to fuse shallow features with global semantic features through progressive upsampling.

[0058] In some possible implementations, global semantic features and deep features can be propagated to shallow features through a top-down and bottom-up bidirectional fusion approach, enabling the shallow features to be fused with global semantic features and deep features. At the same time, shallow features are propagated to global semantic features and deep features, enabling the global semantic features and deep features to be fused with shallow features. This approach not only compensates for the lack of semantic information in shallow features and improves the target extraction capability of shallow features, but also compensates for the lack of localization information in deep features and improves the target extraction capability of deep features.

[0059] In some possible implementations, PAN can be used to progressively upsample global semantic features and fuse them with shallow features, or progressively downsample shallow features and fuse them with global semantic features and deep features.

[0060] In some possible implementations, in step S140, traditional machine learning classification methods such as SVM (Support Vector Machine) can be used to obtain the target category and target location in the image to be detected based on the detection features.

[0061] In some possible implementations, a deep learning-based classification network can be used to classify the target category and location in the image to be detected based on the detection data.

[0062] In some possible implementations, a pre-trained head can be used to predict detection features and obtain the target location and target category in the image to be detected.

[0063] The target location can be determined by the target bounding box output by Head.

[0064] In the target detection method provided in this disclosure embodiment, based on the attention mechanism, global modeling is performed by cross-attention calculation of scale features at multiple scales to obtain global semantic features. By fusing global semantic features with shallow features, the target extraction capability of the obtained detection features is improved, thereby improving the accuracy of the obtained target category and target location.

[0065] The target detection method provided in the embodiments of this disclosure will be described in detail below.

[0066] As mentioned above, in some possible implementations, cross-attention calculations can be performed on each scale feature and other scale features to obtain the attention feature corresponding to that scale feature, and the global semantic feature can be obtained through the attention feature corresponding to each scale feature.

[0067] Figure 2 This diagram illustrates a flowchart of an implementation method that performs cross-attention calculation on each scale feature and other scale features to obtain the attention feature corresponding to that scale feature, and then obtains the global semantic feature through the attention feature corresponding to each scale feature. Figure 2 As shown, steps S210, S220, and S230 may be included.

[0068] In step S210, multiple scale features are concatenated, the concatenated features are used as content features, the scale features are mapped into feature vectors, the feature vectors are used as query features, and the scale features are used as key-value features.

[0069] In step S220, cross-attention processing is performed based on content features, key features, and query features to obtain the attention features corresponding to the scale features;

[0070] In step S230, global semantic features are obtained based on attention features corresponding to multiple scale features.

[0071] In some possible implementations, in step S210, multiple scale features are spliced ​​together, which can be done by horizontally splicing multiple scale features as content features.

[0072] In some possible implementations, the scale feature can be mapped to a feature vector through a flattening operation. For example, if the scale feature size is 10*10, the scale feature can be transformed into a feature vector of size 100*1 through a flattening operation.

[0073] In some possible implementations, each scale feature is treated as a key-value feature and cross-attention is calculated.

[0074] In some possible implementations, in step S220, the content feature is used as the V value, the query feature as the Q value, and the key feature as the K value, and cross-attention calculation is performed to obtain the attention feature corresponding to the scale feature used as the key feature.

[0075] In some possible implementations, in step S230, global semantic features can be obtained by adding the attention features corresponding to each scale.

[0076] By using scale features at various scales as key-value features for attention calculation, and by interacting and fusing each scale feature with scale features at other scales, global interaction and fusion of scale features at different scales are achieved. This improves the global modeling capability of the acquired global semantic features, thereby enhancing the global modeling capability of the detection features acquired through global semantic features, and ultimately improving the accuracy of the acquired target category and target location.

[0077] As mentioned above, in some possible implementations, for scale features, only feature points containing objects can contribute to the effectiveness of the obtained detection features by participating in the interaction and fusion of features at different scales. However, feature points in background regions without objects will not only fail to contribute to the effectiveness of the obtained detection features by participating in the interaction and fusion of features at different scales, but will also increase the computational load. Therefore, before performing cross-attention calculation, pooling is performed on scale features to obtain feature points containing objects, and global semantic features are obtained by performing cross-attention calculation on feature points containing objects.

[0078] Figure 3 The diagram illustrates a specific implementation of a method to obtain global semantic features by performing pooling operations on scale features before cross-attention calculation, thereby obtaining feature points containing the object, and then performing cross-attention calculation on these object-containing feature points. Figure 3 As shown, steps S310 and S320 may be included.

[0079] In step S310, a pooling operation is performed on the shallow features to obtain pooled features, so that the resolution of the pooled features is consistent with the resolution of the deep features.

[0080] In step S320, cross-attention calculation is performed based on pooling features and deep features to obtain global semantic features.

[0081] In some possible implementations, in step S310, shallow features of different scales are filtered using windows with different side lengths to obtain pooling features with the same resolution as deep features, that is, the size of the obtained pooling features is the same as the size of the deep features.

[0082] In some possible implementations, the pooling operation can be max pooling. Since the activation values ​​of feature points containing objects are very likely to be greater than the activation values ​​of feature points not containing objects, max pooling can retain the feature point with the largest activation value in the window. Therefore, max pooling can retain feature points containing objects and delete feature points not containing objects.

[0083] In some possible implementations, in step S320, global modeling can be achieved by performing cross-attention calculation between pooled features and deep features to obtain global semantic features.

[0084] In some possible implementations, cross-attention calculations can be performed on each pooling feature with other pooling features and deep features to obtain the attention features corresponding to that pooling feature, and then the global semantic features can be obtained through the obtained attention features.

[0085] In some possible implementations, pooling features and deep features can be concatenated (e.g., horizontal concatenation) to serve as content features. In other possible implementations, a flattening operation can be used to map pooling features or deep features into feature vectors, which can then be used as query features; pooling features or deep features can also be used as key-value features for cross-attention computation.

[0086] In some possible implementations, content features are used as V values, query features as Q values, and key features as K values. Cross-attention calculation is performed to obtain the attention features corresponding to the pooling features or deep features that serve as key features.

[0087] In some possible implementations, global semantic features can be obtained by adding the attention features corresponding to each pooling feature and the attention features corresponding to the deep features.

[0088] By pooling the scale features, the number of feature points involved in the cross-attention calculation is greatly reduced, thus reducing the computational cost of the cross-attention calculation and improving the speed of the entire object detection process.

[0089] As mentioned above, in some possible implementations, FPN can be used to fuse shallow features by progressively upsampling global semantic features.

[0090] In some specific implementations, the global semantic features are copied and then upsampled to obtain the upsampled features as the detection features corresponding to the global semantic features. Then, a 1*1 convolution is used to reduce the dimensionality of the shallow features closest to the global semantic features, and the dimensionality-reduced features are obtained. The upsampled features and the dimensionality-reduced features are added together to obtain the first fusion feature, which is used as the detection feature corresponding to the shallow features closest to the global semantic features. The first fusion feature is upsampled to obtain the upsampled features, and a 1*1 convolution is used to reduce the dimensionality of the shallow features second closest to the global semantic features, and the dimensionality-reduced features are obtained. The upsampled features and the dimensionality-reduced features are added together to obtain the first fusion feature, which is used as the detection feature corresponding to the shallow features second closest to the global semantic features. This process is repeated to obtain the detection features corresponding to all shallow features.

[0091] As mentioned above, in some possible implementations, FPN can be used to fuse shallow features by progressively upsampling global semantic features.

[0092] Figure 4 This diagram illustrates a process for fusing shallow features with FPN through progressive upsampling of global semantic features. Figure 3 As shown, steps S410 and S420 may be included.

[0093] In step S410, the global semantic features are upsampled, and the global semantic features are fused with the shallow features to obtain the first fused feature corresponding to the shallow features.

[0094] In step S420, the first fusion feature corresponding to the shallow feature is downsampled, and the first fusion feature corresponding to the shallow feature is fused with the global semantic feature to obtain the detection feature.

[0095] In some possible implementations, in step S410, global semantic features and deep features are propagated to shallow features through a top-down feature fusion path, so that shallow features are fused with global semantic features and deep features.

[0096] In some specific implementations, the global semantic features are copied and then upsampled to obtain the upsampled features as the first fusion features corresponding to the global semantic features. Then, a 1*1 convolution is used to reduce the dimensionality of the shallow features closest to the global semantic features, and the dimensionality-reduced features are obtained. The upsampled features and the dimensionality-reduced features are added together to obtain the first fusion features corresponding to the shallow features closest to the global semantic features. The first fusion features corresponding to the shallow features closest to the global semantic features are upsampled to obtain the upsampled features. Then, a 1*1 convolution is used to reduce the dimensionality of the shallow features second closest to the global semantic features, and the dimensionality-reduced features are obtained. The upsampled features and the dimensionality-reduced features are added together to obtain the first fusion features corresponding to the shallow features second closest to the global semantic features. This process is repeated until the first fusion features corresponding to all shallow features are obtained.

[0097] In some possible implementations, in step S420, shallow features are propagated to global semantic features and deep features through a bottom-up feature fusion path, so that global semantic features, deep features and shallow features are fused together.

[0098] The system employs a method where the first fusion feature corresponding to the shallowest feature (i.e., the smallest scale feature) is copied and downsampled to obtain the downsampled feature as the detection feature corresponding to the shallowest feature. Then, a 1x1 convolution is used to reduce the dimensionality of the second shallowest feature (i.e., the second smallest scale feature), yielding the dimensionality-reduced feature. The downsampled feature and the dimensionality-reduced feature are then added together to obtain the detection feature corresponding to the second shallowest feature. This process is repeated until the detection feature corresponding to the global semantic feature is obtained.

[0099] By employing a top-down and bottom-up bidirectional fusion approach, global semantic features and deep features are propagated to shallow features, enabling the fusion of shallow features with global semantic features and deep features. Simultaneously, shallow features are propagated to global semantic features and deep features, enabling the fusion of global semantic features and deep features with shallow features. This approach not only compensates for the lack of semantic information in shallow features and improves their target extraction capabilities, but also compensates for the lack of localization information in deep features, thus enhancing their target extraction capabilities.

[0100] As mentioned above, multi-scale feature extraction can be performed on the image to be detected using a pre-trained backbone, cross-attention calculation can be performed using a pre-trained attention network, and the detection features can be predicted using a pre-trained head, thereby obtaining the target location and target category in the image to be detected.

[0101] In some possible implementations, shallow features are pooled through pooling network layers, which can be combined with attention networks to form a Neck.

[0102] In some possible implementations, the Backbone, Neck, and Head are sequentially arranged to form an object detection network. The trained Backbone, Neck, and Head are obtained by training the object detection network end-to-end using sample data.

[0103] In some possible implementations, an object detection network composed of a backbone, neck, and head can be used for arbitrary object detection, improving the accuracy of object detection. In specific implementations, the object detection network can be applied to areas such as face detection, pedestrian detection, vehicle detection, road detection in satellite images, obstacle detection in vehicle camera images, and lesion detection in medical images. Potential application scenarios include video processing, medical applications, security, and autonomous driving.

[0104] Compared to traditional machine learning methods, deep learning-based network models are obviously better at extracting image features. Therefore, using the backbone for feature extraction, the neck for cross-attention calculation, and the head for classification and regression can achieve better results.

[0105] Since most target detection methods in related technologies are based on the backbone, by forming a target detection network from the backbone, neck, and head, the target detection method provided in this disclosure can be easily combined with target detection methods in related technologies, thus facilitating the practical application of the target detection method provided in this disclosure.

[0106] The target detection method provided in this disclosure will be described in detail below with a specific embodiment.

[0107] Figure 5 A schematic diagram of a specific embodiment of the target detection method provided in this disclosure is shown, such as... Figure 5 As shown, taking a five-stage backbone as an example, its intermediate features are represented by C = {C i 1≤i≤5} indicates that the last three stage features are used as scale features, where C3 and C4 are shallow features (i.e., Figure 5The vertical circle in the middle), C5 is a deep feature (i.e. Figure 5 (The horizontal line circle) First, 1*1 convolution is used to uniformly project multi-scale features to a lower dimension, reducing computational overhead while realizing the transformation of the feature space.

[0108] A pooling layer (specifically, a max pooling layer) is used to pool features C3 and C4 to filter feature points. Specifically, C3 and C4 are filtered using windows with side lengths of 4 and 2 respectively, and the feature point with the highest activation value in the window is retained. The pooled features are denoted as {C'3, C'4, C5}.

[0109] After flattening {C'3, C'4, C5}, they are used as query features and input into the attention network. The features at the three scales are concatenated and used as the value (content feature). Cross-attention calculations are then performed with C'3, C'4, and C5 to obtain the attention features corresponding to C'3, C'4, and C5. This is then expressed using the formula... Obtain global semantic features (i.e.) Figure 5 (Horizontal and vertical circles). Among them... The attention feature corresponding to C'3, The attention features corresponding to C'4 The attention feature corresponding to C5.

[0110] Through Top-down (i.e.) Figure 5 The propagation path corresponding to the left-middle diagonal circle) and Bottom-up (i.e. Figure 5 The propagation path corresponding to the right-hand diagonal circle) is a bidirectional enhancement path that fully integrates global semantic features F5 and shallow features C3 and C4 to obtain the detection features corresponding to C'3, C'4, and C5 (i.e., Figure 5 (Right-center diagonal circle).

[0111] Experiments using the end-to-end object detection model DINO on the MS COCO2017 dataset demonstrate that the object detection method proposed in this disclosure improves accuracy. With the same number of training epochs and lower resolution, it still surpasses the original model in accuracy, achieving 50.5 mAP (mean accuracy), while significantly reducing training time. Furthermore, under the same input size, the inference time used by the object detection method proposed in this disclosure is reduced by 85% compared to other models; on T4, inference using TensorRT (an inference library) requires only 10.3 ms.

[0112] Based on and Figure 1 The method shown follows the same principle. Figure 6 A schematic diagram of the structure of a target detection device provided in an embodiment of this disclosure is shown, as follows: Figure 6As shown, the target detection device 60 may include:

[0113] The feature extraction module 610 is used to perform multi-scale feature extraction on the image to be detected, and to obtain multiple scale features corresponding to different scales. The multiple scale features include at least one shallow feature and one deep feature, and the scale corresponding to the shallow feature is smaller than the scale corresponding to the deep feature.

[0114] Attention module 620 is used to perform cross-attention calculation based on features at multiple scales to obtain global semantic features;

[0115] The feature fusion module 630 is used to fuse global semantic features with shallow features to obtain detection features;

[0116] The detection module 640 is used to obtain the target category and target location in the image to be detected based on the detection features.

[0117] In the target detection device of this disclosure embodiment, based on the attention mechanism, global modeling is performed by cross-attention calculation of scale features at multiple scales to obtain global semantic features. By fusing global semantic features with shallow features, the target extraction capability of the obtained detection features is improved, thereby improving the accuracy of the obtained target category and target location.

[0118] It is understood that the above-described modules of the target detection device in the embodiments of this disclosure have the ability to implement... Figure 1 The embodiments shown illustrate the functions of corresponding steps in the target detection method. These functions can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions. These modules can be software and / or hardware, and each module can be implemented individually or integrated from multiple modules. For a detailed description of the functions of each module in the aforementioned target detection device, please refer to [link to relevant documentation]. Figure 1 The corresponding descriptions of the target detection methods in the embodiments shown are not repeated here.

[0119] In some possible implementations, the attention module 620 includes: a feature acquisition unit, used to concatenate multiple scale features, use the concatenated features as content features, map the scale features into feature vectors, use the feature vectors as query features, and use the scale features as key-value features; an attention calculation unit, used to perform cross-attention processing based on content features, key-value features, and query features to obtain attention features corresponding to the scale features; and a feature calculation unit, used to obtain global semantic features based on the attention features corresponding to multiple scale features.

[0120] In some possible implementations, the attention module 620 includes: a pooling unit for performing pooling operations on shallow features to obtain pooled features, such that the resolution of the pooled features is consistent with the resolution of the deep features; and performing cross-attention calculation based on the pooled features and the deep features to obtain global semantic features.

[0121] In some possible implementations, the pooling operation is a max pooling operation.

[0122] In some possible implementations, the feature fusion module 630 includes: a top-down unit for upsampling global semantic features, fusing global semantic features with shallow features, and obtaining a first fused feature corresponding to the shallow features; and a bottom-up unit for downsampling the first fused feature corresponding to the shallow features, fusing the first fused feature corresponding to the shallow features with global semantic features, and obtaining detection features.

[0123] In some possible implementations, the feature extraction module 610 is used to extract multi-scale features from the image to be detected using a pre-trained backbone network; the attention module 620 is used to perform cross-attention calculation based on multiple scale features using a pre-trained attention network to obtain global semantic features; and the detection module 640 is used to input the detection features into a pre-trained detection head to obtain the target category and target location in the image to be detected.

[0124] In some possible implementations, the backbone network, attention network, and detection head are sequentially assembled to form the target detection network; the backbone network, attention network, and detection head are obtained by pre-training the target detection network end-to-end.

[0125] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0126] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0127] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the target detection method provided in the embodiments of this disclosure.

[0128] Compared with existing technologies, this electronic device is based on an attention mechanism. It performs global modeling by cross-attention calculation of scale features at multiple scales to obtain global semantic features. By fusing global semantic features with shallow features, it improves the target extraction capability of the obtained detection features, thereby improving the accuracy of the obtained target category and target location.

[0129] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the target detection method provided in the embodiments of this disclosure.

[0130] Compared with existing technologies, this readable storage medium is based on an attention mechanism. It performs global modeling by cross-attention calculation of scale features at multiple scales to obtain global semantic features. By fusing global semantic features with shallow features, it improves the target extraction capability of the obtained detection features, thereby improving the accuracy of the obtained target category and target location.

[0131] The computer program product includes a computer program that, when executed by a processor, implements the target detection method as provided in the embodiments of this disclosure.

[0132] Compared with existing technologies, this computer program product is based on an attention mechanism. It performs global modeling by cross-attention calculation of scale features at multiple scales to obtain global semantic features. By fusing global semantic features with shallow features, it improves the target extraction capability of the obtained detection features, thereby improving the accuracy of the obtained target category and target location.

[0133] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0134] like Figure 7As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0135] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0136] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as object detection methods. For example, in some embodiments, the object detection method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the object detection method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the object detection method described above by any other suitable means (e.g., by means of firmware).

[0137] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0139] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0141] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0142] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0143] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0144] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for object detection, comprising: Multi-scale feature extraction is performed on the image to be detected to obtain multiple scale features corresponding to different scales. The multiple scale features include a deep feature and at least one shallow feature, and the scale corresponding to the shallow feature is smaller than the scale corresponding to the deep feature. The shallow features are pooled to obtain pooled features, which are then used to filter out feature points containing objects, and the resolution of the pooled features is made consistent with the resolution of the deep features. Multiple scale features are concatenated, and the concatenated features are used as content features. The deep features and at least one pooling feature are mapped into feature vectors, and the feature vectors are used as query features. The corresponding scale features are used as key-value features. Cross-attention processing is performed based on the content features, the key-value features, and the query features to obtain the attention features corresponding to the scale features; Global semantic features are obtained based on attention features corresponding to multiple scale features; The global semantic features and the shallow features are fused to obtain the detection features; The target category and target location in the image to be detected are obtained based on the detection features.

2. The method according to claim 1, wherein, The pooling operation is a max pooling operation.

3. The method according to claim 1, wherein, The step of fusing the global semantic features with the shallow features to obtain detection features includes: The global semantic features are upsampled, and the global semantic features are fused with the shallow features to obtain the first fused feature corresponding to the shallow features; The first fusion feature corresponding to the shallow feature is downsampled, and the first fusion feature corresponding to the shallow feature is fused with the global semantic feature to obtain the detection feature.

4. The method according to claim 1, wherein, The multi-scale feature extraction of the image to be detected includes: Multi-scale feature extraction is performed on the image to be detected using a pre-trained backbone network; The step of performing cross-attention calculation based on multiple scale features to obtain global semantic features includes: Based on multiple scale features, a pre-trained attention network is used to perform cross-attention calculation to obtain global semantic features; The step of obtaining the target category and target location in the image to be detected based on the detection features includes: The detection features are input into a pre-trained detection head to obtain the target category and target location in the image to be detected.

5. The method according to claim 4, wherein, The backbone network, the attention network, and the detection head are sequentially combined to form a target detection network; the backbone network, the attention network, and the detection head are obtained by pre-training the target detection network end-to-end.

6. A target detection apparatus, comprising: The feature extraction module is used to perform multi-scale feature extraction on the image to be detected, and to obtain multiple scale features corresponding to different scales. The multiple scale features include a deep feature and at least one shallow feature, and the scale corresponding to the shallow feature is smaller than the scale corresponding to the deep feature. Attention module, including: A pooling unit is used to perform pooling operations on the shallow features to obtain pooled features, thereby filtering out feature points containing objects, and making the resolution of the pooled features consistent with the resolution of the deep features. The feature acquisition unit is used to concatenate multiple scale features, use the concatenated features as content features, map the deep features and at least one pooling feature into feature vectors respectively, use the feature vectors as query features, and use the corresponding scale features as key-value features. An attention calculation unit is used to perform cross-attention processing based on the content features, the key-value features, and the query features to obtain the attention features corresponding to the scale features; The feature calculation unit is used to obtain global semantic features based on the attention features corresponding to multiple scale features; The feature fusion module is used to fuse the global semantic features with the shallow features to obtain detection features; The detection module is used to obtain the target category and target location in the image to be detected based on the detection features.

7. The apparatus according to claim 6, wherein, The pooling operation is a max pooling operation.

8. The apparatus according to claim 6, wherein, The feature fusion module includes: The top-down unit is used to upsample the global semantic features, fuse the global semantic features with the shallow features, and obtain the first fused feature corresponding to the shallow features; The bottom-up unit is used to downsample the first fusion feature corresponding to the shallow feature, and fuse the first fusion feature corresponding to the shallow feature with the global semantic feature to obtain the detection feature.

9. The apparatus according to claim 6, wherein, The feature extraction module is used to perform multi-scale feature extraction on the image to be detected using a pre-trained backbone network; The attention module is used to perform cross-attention calculation using a pre-trained attention network based on multiple scale features to obtain global semantic features; The detection module is used to input the detection features into a pre-trained detection head to obtain the target category and target location in the image to be detected.

10. The apparatus according to claim 9, wherein, The backbone network, the attention network, and the detection head are sequentially combined to form a target detection network; the backbone network, the attention network, and the detection head are obtained by pre-training the target detection network end-to-end.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images