An image target detection method

Abstract

The application discloses an image target detection method, which comprises the following steps: inputting a to-be-detected image into a convolutional neural network model to obtain an extracted image feature vector; inputting the image feature vector into a region proposal network to obtain a first candidate region feature and a corresponding position coordinate matrix; obtaining an initial candidate region set through the first candidate region feature and the position coordinate matrix; obtaining a category label set corresponding to the to-be-detected image, and obtaining a transformed second candidate region feature and a transformed label feature based on the category label set, the candidate region set and a self-attention layer; performing feature fusion on the second candidate region feature and the label feature by using a cross-attention layer to obtain a target feature; and inputting the target feature and the candidate region set into a full connection layer to obtain the final position corresponding to the candidate region set and the category of the object contained in each candidate region set. The application improves the image detection precision.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image detection, and relates to but is not limited to an image target detection method. BACKGROUND

[0002] Target detection is one of the most basic problems in the field of computer vision, and the purpose is to detect which objects in a picture or video and determine the specific position of the objects. Although an image target detection model constructed through deep learning technology has been widely applied in intelligent driving, smart city and other scenarios. However, in the related art, the target detection model still has the problem of low detection accuracy.

[0003] Therefore, how to quickly improve the image detection accuracy has become a problem to be solved. SUMMARY

[0004] Therefore, the embodiment of the present application provides an image target detection method, which at least solves the problem of low image detection accuracy in the related art.

[0005] According to a first aspect of the embodiment of the present application, an image target detection method is provided, comprising:

[0006] inputting a to-be-detected image into a convolutional neural network model to obtain an extracted image feature vector;

[0007] inputting the image feature vector into a region proposal network to obtain a first candidate region feature and a corresponding position coordinate matrix; and obtaining an initial candidate region set through the first candidate region feature and the position coordinate matrix;

[0008] obtaining a class label set corresponding to the to-be-detected image, and based on the class label set, the candidate region set and a self-attention layer, obtaining a transformed second candidate region feature and a transformed label feature;

[0009] performing feature fusion on the second candidate region feature and the label feature by using a cross-attention layer to obtain a target feature; and inputting the target feature and the candidate region set into a fully connected layer to obtain the final position corresponding to the candidate region set and the class of the object contained in each of the candidate region set.

[0010] According to a second aspect of the embodiment of the present application, an electronic device is provided, comprising a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction makes the processor execute the operation corresponding to the method of the first aspect.

[0011] According to a third aspect of the embodiments of the present application, a computer storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the method according to the first aspect.

[0012] The scheme provided by the embodiments of the present application inputs the image to be detected into a convolutional neural network model to obtain an extracted image feature vector; inputs the image feature vector into a region proposal network to obtain a first candidate region feature and a corresponding position coordinate matrix; and obtains an initial candidate region set through the first candidate region feature and the position coordinate matrix; obtains a category label set corresponding to the image to be detected, and obtains a transformed second candidate region feature and a transformed label feature based on the category label set, the candidate region set and a self-attention layer; performs feature fusion on the second candidate region feature and the label feature by using a cross-attention layer to obtain a target feature; and inputs the target feature and the candidate region set into a full connection layer to obtain the final position corresponding to the candidate region set and the category of the object contained in each candidate region set. In this process, the image feature vector is extracted by the convolutional neural network, which can capture high-level semantic information in the image, the region proposal network is used to generate the candidate region and the position coordinate matrix from the image feature vector, which helps to accurately locate the potential target position, reduces false detection and missed detection, the self-attention layer is introduced to better capture the relevance between different candidate regions, and the candidate region feature is adjusted according to the category label set, thereby improving the classification accuracy. The cross-attention layer can further fuse the second candidate region feature and the label feature to realize more accurate feature representation. The target feature and the candidate region set obtained through the above steps are processed by the full connection layer, which can output the accurate position of each candidate region and the category of the object contained therein, so that the final detection result is more accurate and reliable. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0014] Figure 1 Flowchart of an image target detection method provided by the embodiments of the present application Figure 1 ;

[0015] Figure 2 Inference process diagram of a self-attention layer provided by the embodiments of the present application

[0016] Figure 3A reasoning process diagram of a cross-attention layer provided by an embodiment of the present application is provided.

[0017] Figure 4 A flowchart of an image target detection method provided by an embodiment of the present application is provided. Figure 2

[0018] Figure 5 A flowchart of an image target detection method provided by an embodiment of the present application is provided. Figure 3

[0019] Figure 6 A visual comparison result diagram of a baseline model (Baseline) and a model (Baseline+IRR) after adding an IRR framework on a VOC 2007 test set provided by an embodiment of the present application is provided.

[0020] Figure 7 A structural diagram of an electronic device provided by an embodiment of the present application is provided. DETAILED DESCRIPTION

[0021] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application but not all the embodiments of the present application. The following embodiments are used to illustrate the present application but not to limit the scope of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0022] In the following description, “some embodiments” are described, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

[0023] It should be noted that the terms “first\second\third” involved in the embodiments of the present application are only to distinguish similar objects and do not represent a specific order of the objects. It can be understood that “first\second\third” can be interchanged with a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0024] ​​Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which embodiments of the present application belong. It should also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless specifically so defined herein.

[0025] Figure 1 Flowchart of an image target detection method provided by embodiments of the present application Figure 1 The image target detection method provided by embodiments of the present application can be executed by an electronic device, which can be a computer, a server, etc.

[0026] As shown in Figure 1 The image target detection method comprises the following steps.

[0027] S101, inputting a to-be-detected image into a convolutional neural network model to obtain an extracted image feature vector.

[0028] In embodiments of the present application, the to-be-detected image contains multiple objects. An "object" in an image generally refers to a specific entity or a part of interest contained in the image, which can be a person, an animal, a vehicle, a building, etc. any thing with clear boundaries and features. The to-be-detected image is input into the convolutional neural network model. The convolutional neural network can effectively extract multi-level feature representations from the to-be-detected image through a series of convolution operations, pooling operations and nonlinear activation functions, and obtain the extracted image feature vector.

[0029] S102, inputting the image feature vector into a region proposal network to obtain a first candidate region feature and a position coordinate matrix of the candidate region; and obtaining an initial candidate region set through the first candidate region feature and the position coordinate matrix.

[0030] In embodiments of the present application, a candidate region refers to a rectangular region that may contain an object of interest in a detected image. These regions are not the final determined object positions, but potential object positions generated in advance based on certain algorithms or strategies. The candidate regions do not necessarily correspond to the actual object bounding box accurately, but they provide a starting point so that the subsequent target detection algorithm can focus more on these regions for accurate classification and bounding box adjustment. The role of the region proposal network (RPN) is to generate candidate regions on the image, while outputting the prediction value of each candidate region. These prediction values are used to represent whether the candidate region contains an object on the image. Specifically, the RPN usually outputs two main information:

[0031] Candidate region location: These are the coordinates of regions on the image that may contain the target object (usually the coordinates of the bounding box, including the coordinates of the top left and bottom right corners).

[0032] Predicted values ​​(confidence scores): These values ​​represent the probability that each candidate box contains the target object. For example, a predicted value close to 1 indicates that the candidate region is very likely to contain a target object, while a predicted value close to 0 indicates that the candidate box is very likely not to contain a target object.

[0033] Specifically, the image feature vector is input into the region proposal network to obtain the first candidate region features and location coordinate matrix; the initial candidate region set is obtained through the candidate region features and location coordinate matrix.

[0034] S103. Obtain the set of category labels corresponding to the image to be detected, and based on the set of category labels, the set of candidate regions and the self-attention layer, obtain the transformed second candidate region features and the transformed label features.

[0035] In embodiments of the present invention, category labels typically refer to category information associated with target objects in an image. These category labels are used to indicate the category to which each target object in the image belongs. For example, in an object detection task, if an image contains different types of objects (such as cars, pedestrians, bicycles, etc.), each object will have a corresponding category label. In object detection tasks, category labels typically refer to the category of each target object in the image. For example, if an image contains a car and a pedestrian, then the category labels for these two target objects might be "car" and "pedestrian." These category labels are used during training to guide the model in learning how to distinguish between different categories of targets. When training the object detection model, category labels are used to calculate the loss function. The model's output (predicted category and location) is compared with the true labels and bounding boxes to calculate classification loss and regression loss. By minimizing these losses, the model can learn how to more accurately identify and locate target objects in an image. During the inference (testing) phase, the model outputs the predicted category and confidence score for each candidate box, which are generated based on the label information learned by the model during the training phase. The category label set refers to a set of all possible category labels organized for model training and inference. For example, if a dataset has 20 different categories, then the label set will contain labels for those 20 categories.

[0036] Furthermore, after obtaining the set of category labels corresponding to the image to be detected, the set of category labels and the set of candidate regions are input into the self-attention layer to obtain the transformed second candidate region features and the transformed label features.

[0037] S104, fusing the second candidate region feature and the label feature by using a cross-attention layer to obtain a target feature; and inputting the target feature and the candidate region set into a fully connected layer to obtain the final position corresponding to the candidate region set and the category of the object contained in each candidate region set.

[0038] In the embodiments of the present application, the cross-attention layer is used to learn the mutual dependence between the region set and the label set. In the cross-attention layer, the feature representation of the candidate region set will interact with the feature representation of the category label set to capture the global dependence between the candidate region and the category label. For example, the cross-attention layer will learn the relationship between each candidate region and each category label, thereby updating the feature representation of each candidate region. The second candidate region feature and the label feature are input into the cross-attention layer for feature fusion to obtain the target feature, and the target feature and the candidate region set are input into the fully connected layer for feature integration to obtain the final position corresponding to the candidate region set and the category of the object contained in each candidate region set.

[0039] It can be understood that in the embodiments of the present application, the image to be detected is input into the convolutional neural network model to obtain the extracted image feature vector; the image feature vector is input into the region proposal network to obtain the first candidate region feature and the corresponding position coordinate matrix; and the initial candidate region set is obtained through the first candidate region feature and the position coordinate matrix; the category label set corresponding to the image to be detected is obtained, and based on the category label set, the candidate region set and the self-attention layer, the transformed second candidate region feature and the transformed label feature are obtained; the second candidate region feature and the label feature are fused by using the cross-attention layer to obtain the target feature; and the target feature and the candidate region set are input into the fully connected layer to obtain the final position corresponding to the candidate region set and the category of the object contained in each candidate region set. In this process, the image feature vector is extracted by the convolutional neural network, which can capture high-level semantic information in the image, and the region proposal network is used to generate candidate regions and their position coordinate matrices from the image feature vector, which helps to accurately locate the potential target position, reduces false detection and missed detection, and the introduction of the self-attention layer can better capture the relevance between different candidate regions and adjust the candidate region feature according to the category label set, thereby improving the classification accuracy. The cross-attention layer can further fuse the second candidate region feature and the label feature to achieve more accurate feature representation. The target feature and the candidate region set obtained through the above steps are processed by the fully connected layer, which can output the accurate position of each candidate region and the category of the object contained therein, so that the final detection result is more accurate and reliable.

[0040] In some embodiments of the present application, S103 can be implemented by S1031 to S1033, which are described as follows.

[0041] S1031, processing the category label set by using a global vector word representation algorithm to obtain a corresponding category word vector.

[0042] S1032, converting the category word vector into a feature representation space corresponding to the image feature vector by using a conversion function to obtain a converted category word vector.

[0043] S1033, obtaining a transformed second candidate region feature and a transformed label feature based on the converted category word vector, the candidate region set and a self-attention layer.

[0044] In some embodiments of the present application, the category label is converted by using a global vector word representation algorithm (GloVe algorithm). Assuming that there are three categories, each category is directly input into the GloVe algorithm to obtain a corresponding category word vector. A learnable conversion function is used to adjust the word vector to adapt it to the specific task requirements and ensure that it is located in the same representation space as other types of data features (such as image feature vectors) to obtain a converted category word vector.

[0045] Further, the converted category word vector and the candidate region set are input into the self-attention layer to obtain a transformed second candidate region feature and a transformed label feature.

[0046] In some embodiments of the present application, S1033 can be implemented by S301 to S302, which are described as follows.

[0047] S301, inputting the category word vector into the self-attention layer to obtain a first query vector, a first key vector and a first value vector; and obtaining a transformed label feature based on the first query vector, the first key vector and the first value vector.

[0048] In some embodiments of the present application, the first query vector, the first key vector and the first value vector are obtained by linear transformation when the category word vector is input into the self-attention layer, and the specific transformation is as follows:

[0049] ;

[0050] wherein, is the first query vector, is the first key vector, is the first value vector, , , are transformation matrices respectively, F is the category word vector, i is the i-th element of the input category word vector. i

[0051] ​​Further, the attention score calculation, the application of the softmax function, and the weighted summation are performed based on the first query vector, the first key vector, and the first value vector to obtain the transformed label feature. The attention score calculation and the application of the softmax function are as follows:

[0052] ;

[0053] In the above formula, is a correlation coefficient matrix, and d is a dimension of the first key vector.

[0054] The weighted summation is performed, that is, the learned correlation coefficient matrix The feature information in the set is aggregated to obtain the transformed label feature.

[0055] S302, input the candidate region set into the self-attention layer to obtain a second query vector, a second key vector, and a second value vector, and obtain a transformed second candidate region feature based on the second query vector, the second key vector, and the second value vector.

[0056] In some embodiments of the present application, the candidate region set is input into the self-attention layer to obtain a second query vector, a second key vector, and a second value vector, and the attention score calculation, the application of the softmax function, and the weighted summation are performed based on the second query vector, the second key vector, and the second value vector to obtain a transformed second candidate region feature. S302 and S301 are similar.

[0057] As shown in Figure 2 , the inference process of the self-attention layer provided in the embodiments of the present application is as follows. Figure 2 1 represents the number of heads of the multi-head attention mechanism used in the self-attention module. In the feedforward network (FFN), the information of multiple feature spaces is aggregated and calculated through a residual to obtain the final enhanced region feature, as follows: N

[0058] ;

[0059] In the above formula, F represents an input feature set input into the self-attention module or the cross-attention module (the specific meaning is described above), represents an updated feature set after the attention mechanism processing, is an update function, which is responsible for updating the feature representation according to the original input feature F and the feature processed by the attention mechanism. This function can involve a multi-layer perceptron (MLP, Multi-Layer Perceptron) or other types of neural networks for learning the nonlinear combination of features, ​is a learnable weight parameter matrix used to transform the collected feature information into the same dimensional space as the original input feature set F, F denotes the updated feature set is assigned to F, meaning that the original input feature set is replaced by the updated feature set.

[0060] The above formula describes how to update the feature representation by combining the original input features and the features processed by the attention mechanism in the self-attention module or the cross-attention module. This process involves feature aggregation, transformation and updating, aiming to enhance the representation ability of the features by capturing the mutual dependence within and between the sets.

[0061] In some embodiments of the present application, the feature fusion of the second candidate region feature and the label feature in S104 using the cross-attention layer to obtain the target feature can be implemented through S1041 to S1043, which is described as follows.

[0062] S1041, the cross-attention layer is used to fuse the second candidate region feature and the label feature to obtain the fused feature.

[0063] As shown in Figure 3 , Figure 3 is a reasoning process diagram of a cross-attention layer provided by an embodiment of the present application. In Figure 3 , the key vector and the value vector are determined by the label feature, and the query vector is obtained by the second candidate region feature. The attention score calculation, the application of the softmax function and the weighted summation are performed by the query vector, the key vector and the value vector. Since there are multiple heads, each head will generate its own context vector. In order to integrate these information from different perspectives, the outputs of each head are spliced together, and then in the feedforward neural network (FFN), the feature fusion is performed by using the residual network to obtain the fused feature.

[0064] S1042, when the fused feature meets the preset condition or the iteration number reaches the preset iteration condition, the fused feature is taken as the target feature.

[0065] In some embodiments of the present application, the second candidate region feature and the label feature are input into the cross-attention layer to perform the transformation of the query vector, the key vector and the value vector, the attention score calculation, the application of the softmax function and the weighted summation and other operations to fuse the features, and obtain the fused feature. When the fused feature meets the preset condition or the iteration number reaches the maximum iteration number, the fused feature is taken as the target feature.

[0066] S1043, when the fused feature does not meet the preset condition or the iteration number does not reach the preset iteration condition, input the fused feature to the self-attention layer again until the target feature is obtained.

[0067] In some embodiments of the application, when the fused feature does not meet the preset condition or the iteration number does not reach the maximum iteration number, the fused feature is fed back to the self-attention layer for feature updating and processing operation, and the feature output by the self-attention layer is input to the cross-attention layer again until the feature output by the cross-attention layer meets the preset condition or the iteration number reaches the preset iteration condition, and the target feature is obtained.

[0068] In some embodiments of the application, S1041 can be implemented by S1041A, which is described as follows.

[0069] S1041A, input the second candidate region feature and the label feature into the cross-attention layer to obtain a third query vector, a third key vector and a third value vector; and perform feature fusion through the third query vector, the third key vector and the third value vector to obtain a fused feature.

[0070] In some embodiments of the application, the second candidate region feature and the label feature are input to the cross-attention layer, the third query vector, the third key vector and the third value vector are determined by the second candidate region feature and the label feature, and the fused feature is obtained by attention score calculation, application of a softmax function and weighted summation based on the third query vector, the third key vector and the third value vector.

[0071] In some embodiments of the application, the input of the target feature and the candidate region set to the fully connected layer in S104 to obtain the final position corresponding to the candidate region set and the class of the object contained in each of the candidate region set can be implemented by S201, which is described as follows.

[0072] S401, add the target feature and the candidate region set to obtain an added feature, and input the added feature to a fully connected layer to obtain a final position and a class.

[0073] In some embodiments of the application, the target feature and the candidate region set are added to obtain an added feature, and the added feature is input to several fully connected layers, which are finally branched into two outputs: one is a classification score (used to determine which class the object belongs to), and the other is a bounding box regression (used to refine the position and size of the object).

[0074] As shown in Figure 4 , a flowchart of an image target detection method provided by an embodiment of the application is shown in Figure 4 , and Figure 2 ,Figure 4 In this process, the image to be detected is input into a convolutional neural network model to obtain extracted image feature vectors. These feature vectors are then input into a region proposal network to obtain first candidate region features and their corresponding position coordinate matrices. An initial candidate region set is obtained using these first candidate region features and the position coordinate matrices. Input category labels, such as cls_a, cls_b, and cls_c, are converted into vector representations using a GloVe model. The category label vectors and the candidate region set are then processed by a self-attention layer to obtain transformed second candidate region features and transformed label features. The second candidate region features and transformed label features are then input into a cross-attention layer for feature fusion. If the fused features meet preset conditions or the number of iterations reaches a preset iteration condition, the fused features are used as the target features. If the fused features do not meet the preset conditions or the number of iterations does not reach the preset iteration condition, the fused features are input into the self-attention layer until the target features are obtained. The target features and the candidate set are added together and input into a fully connected layer to obtain the final positions corresponding to the candidate region set and the categories of the objects contained in each candidate region set.

[0075] like Figure 5 As shown, Figure 5 A flowchart illustrating an image target detection method provided in an embodiment of the present invention. Figure 3 ,Right now Figure 5 This is an IRR framework that combines self-attention layers and cross-attention layers. Figure 5 In this model, an image is segmented into multiple candidate regions (the four green circles shown in the figure). These regions may represent potential object locations, and features are extracted from each region using a method such as a convolutional neural network. Pre-trained word embeddings (such as GloVe) are used to initialize category labels, forming initial category feature representations (the blue circles in the figure). The candidate region set and category label set are input into a self-attention layer to obtain transformed second candidate region features and transformed label features. These transformed second candidate region features and transformed label features are then input into a cross-attention layer for feature fusion. The model iterates multiple times, gradually optimizing the relationship between region proposals and category labels until convergence is achieved, obtaining the target features. Finally, based on these target features, the final object classification result is obtained, representing the specific category corresponding to each region and a precise adjustment of the object's location, ensuring more accurate object bounding boxes.

[0076] In an embodiment of the present invention, the following example 1 is provided:

[0077] Taking Pascal VOC typical samples as examples, several typical feature extraction methods based on Bayesian network are selected to test the recognition performance of the added interaction relationship reasoning framework on different types of detection methods. Table 1 below shows the comparison experimental results between the detection model added with the interaction relationship reasoning framework and the baseline model on the VOC 2007 test set.

[0078] Table 1 Performance comparison of baseline+IRR and baseline on Pascal VOC test set

[0079]

[0080] Wherein, Faster R-CNN is a fast region convolutional neural network, Faster R-CNN+IRR is a fast region convolutional neural network+interaction relationship reasoning framework, Faster R-CNN is a fast region convolutional neural network, Faster R-CNN+IRR is a fast region convolutional neural network+interaction relationship reasoning framework, ResNet50-FPN is a 50-layer residual network-pyramid network, ResNet101-FPN is a 101-layer residual network-pyramid network, the baseline model is a model without introducing the IRR framework, and Backbone is a backbone network.

[0081] After the IRR framework is added, the mAP index of the baseline model is increased by 1.6% and 1.1% respectively. The results show that the image target detection method can enhance the classification and positioning ability of the algorithm. It is verified that learning the reliable relationship structure between the region and the label and inside the region is beneficial to enhancing the representation ability of the feature and improving the performance of the detection algorithm.

[0082] For the MS-COCO2017 data set, the application selects several mainstream region-based target detection algorithms as baseline models, including Faster R-CNN, Mask R-CNN, DCNv2 with mask and DCNv2 with cascade mask. The latter two modes are added with DCNv2 in Mask R-CNN and Cascade Mask R-CNN, and the detection performance is stronger. ResNet101-FPN is selected as the backbone network for experiments. As can be seen from Table 1, the detection algorithm added with the IRR framework is basically better than the baseline model. In terms of evaluation indexes, the AP, Faster R-CNN and Mask R-CNN detection algorithms are increased by 1.7% and 1.5% respectively. It is also improved from 43.5% and 47.3% to 44.4% and 47.9% on the two more advanced dcnv2-based models.

[0083] In addition, in order to further verify the effectiveness of the IRR framework, the present application also carries out a visual comparison experiment, as shown in Figure 6 In the experiment, ResNet50-FPN is selected as the backbone network, and Faster R-CNN detector is selected as the baseline model. The IRR framework can effectively enhance the detection ability of the detection algorithm. For example, in the first figure in Figure 6 , based on the baseline model, the model with the IRR framework improves the classification confidence value of the "motorbike" label by learning the relationship structure of the target in the image, so that the detection algorithm filters the detection frame according to the confidence threshold. The detection frame labeled as "motorbike" is effectively retained, thereby improving the detection ability of the algorithm. For the second figure, the IRR framework effectively reduces the redundant detection frame labeled as "boat" through the attention mechanism, thereby realizing the correct detection of the detection algorithm. Figure 6 (Figure 9) shows the improvement effect of the IRR (Interactive Relationship Reasoning) framework in the target detection algorithm. Figure 6 The visual comparison results of the baseline model (Baseline) and the model with the IRR framework (Baseline+IRR) on the VOC 2007 test set are shown in Figure 6 . The first figure shows that the classification confidence value of the "motorbike" label is improved after the IRR framework is added to the baseline model. The baseline model may not accurately identify or have a low confidence detection frame. After the IRR framework is added, the model can more accurately identify the "motorbike" label by learning the relationship structure of the target in the image, and filter the detection frame according to the confidence threshold, effectively retaining the detection frame labeled as "motorbike", thereby improving the detection ability of the algorithm. The second figure shows that the IRR framework effectively reduces the redundant detection frame through the attention mechanism. In the baseline model, there may be multiple redundant detection frames labeled as "boat". The IRR framework reduces these redundant detection frames by aggregating the feature information of the region set, thereby realizing the correct detection of the detection algorithm. Figure 6 The visual results in Figure 6 verify the actual effect of the IRR framework in the target detection algorithm, especially the ability to improve detection accuracy and reduce false positives. The IRR framework learns the correlation coefficient matrix within the candidate region, within the label, and between the region and the label through the self-attention module and the cross-attention module, thereby effectively capturing the global long-range dependency in the image, enhancing the feature representation ability, and further improving the performance of the detection algorithm.

[0084] The present application improves the traditional O(n3) algorithm to a more efficient O(n2) algorithm solution, providing improved performance without sacrificing space complexity.

[0085] On this basis, two models are used for target detection respectively, and their performances are tested to verify the effectiveness of target detection. First, only an independent attention model is introduced in target detection to verify its role in target detection. On this basis, the IRR model containing only the cross-attention model is used to verify the cross-attention in target detection. Finally, a comprehensive IRR analysis method is constructed, and on this basis, the influence of the combined detector on the detection effect is tested. Table 2 is the comparison of ablation performance by VOC 2007 test set. Baseline: baseline model, Baseline+IRR: baseline+IRR model, Baseline+IRR (containing self-attention module): baseline+IRR (containing self-attention module), Baseline+IRR (containing cross-attention module): baseline+IRR (containing cross-attention module), Baseline+IRR (containing self-attention and cross-attention module): baseline+IRR (containing self-attention and cross-attention module). Among them, Baseline+IRR (containing self-attention and cross-attention module) is an image target detection method proposed in the embodiment of the application.

[0086] Table 2 Comparison of ablation performance of different module IRR frameworks in VOC 2007 test set

[0087]

[0088] The existing detection algorithm based on the graph convolutional neural network is based on the fixed graph structure constructed by artificial, and the unreliable edge building relationship in the graph can cause the relationship reasoning ability on the degraded graph to be affected, thereby affecting the performance of the detection algorithm. First, the framework initializes the candidate region features and the prior label word vector extracted as two independent sets, then designs a self-attention module to learn the mutual dependence relationship in the set, and realizes the self-reconstruction in the set. The cross-attention module realizes the mutual reconstruction between the region set and the label set. Finally, the similarity of the two updated sets is used to calculate the predicted classification value, and the updated region set is fused to predict the regression value of the position with the original feature.

[0089] As shown in Figure 7 , a structural schematic diagram of an electronic device according to an embodiment of the application, and the specific implementation of the electronic device is not limited in the embodiment of the application.

[0090] As shown in Figure 7 , the electronic device can include a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

[0091] Wherein:

[0092] The processor 502, the communication interface 504, and the memory 506 complete communication with each other through a communication bus 508.

[0093] The communication interface 504 is configured to communicate with other electronic devices or servers.

[0094] The processor 502 is configured to execute the program 510, and specifically can execute the related steps in the above method embodiments.

[0095] Specifically, the program 510 can include program code, and the program code includes computer operation instructions.

[0096] The processor 502 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device can be processors of the same type, such as one or more CPUs; or can be processors of different types, such as one or more CPUs and one or more ASICs.

[0097] The memory 506 is configured to store the program 510. The memory 506 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0098] The program 510 can be specifically used to cause the processor 502 to execute the operations corresponding to the methods described in the above method embodiments.

[0099] The specific implementation of each step in the program 510 can refer to the corresponding description in the corresponding steps and units in the above method embodiments, and will not be described here. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device and the module described above can refer to the corresponding process description in the foregoing method embodiments, and will not be described here.

[0100] It should be noted that, according to the needs of implementation, each component / step described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or part of the operations of the components / steps can be combined into a new component / step, to achieve the purpose of the embodiments of the present application.

[0101] The above-described methods according to embodiments of the application can be implemented in hardware, firmware, or software, or any combination thereof, and can be stored in or implemented with the aid of one or more data storage media (e.g., removable storage media like CD-ROM, floppy disks, hard disks, optical disks, or memory such as ROM, RAM, flash memory, etc.), or implemented with the aid of one or more processors of a processing system. The present application is not limited by these implementations, merely using such software, hardware, firmware, or any combination thereof. It is therefore contemplated that activities of the above-described methods according to the embodiments of the present application can be implemented in software, firmware, hardware, or any combination thereof. The methods of the present application can also be stored on and / or transmitted over as one or more computer programs, software packages, or computer readable media of instructions configured to carry out the methods described herein. The software, firmware, or computer readable media of instructions can be stored on computer readable media, which can be any available media or memory that can be accessed by a computer. By way of example, and not limitation, such computer readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, functional computer programs, software, firmware, routines, sub-routines, programs, instructions, codes, code segments, or computer readable media can direct a computer or other programmable data processing devices to function in a

[0102] Those skilled in the art can appreciate that the units and method steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, or in a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present application.

[0103] The above embodiments are only used to illustrate the present application, but not to limit the present application. Those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present application. Therefore, all equivalent technical solutions should be within the scope of the present application, and the patent protection scope of the present application should be defined by the claims.

Claims

1. An image target detection method, characterized by, The application relates to an image target detection method and device. The image to be detected is input into a convolutional neural network model to obtain an extracted image feature vector; The image feature vector is input into a region proposal network to obtain first candidate region features and a corresponding position coordinate matrix; and an initial candidate region set is obtained through the first candidate region features and the position coordinate matrix; A class label set corresponding to the image to be detected is obtained, and a transformed second candidate region feature and a transformed label feature are obtained based on the class label set, the candidate region set and a self-attention layer; the class label set contains labels of all different classes in a data set, the data set is a public data set and is used for training and testing of an image target detection model; Feature fusion of the second candidate region feature and the label feature is performed through a cross-attention layer to obtain a target feature; and the target feature and the candidate region set are input into a full connection layer to obtain final positions corresponding to the candidate region set and classes of objects contained in the candidate region set; The method comprises the following steps: The class label set is processed through a global vector word representation algorithm to obtain corresponding class word vectors; The class word vectors are converted into a feature representation space corresponding to the image feature vector through a conversion function to obtain converted class word vectors; The converted class word vectors are input into the self-attention layer to obtain a first query vector, a first key vector and a first value vector; and the transformed label feature is obtained based on the first query vector, the first key vector and the first value vector; The candidate region set is input into the self-attention layer to obtain a second query vector, a second key vector and a second value vector; and the transformed second candidate region feature is obtained based on the second query vector, the second key vector and the second value vector.

2. The method of claim 1, wherein, The method comprises the following steps: The second candidate region feature and the label feature are fused through the cross-attention layer to obtain fused features; When the fused features meet a preset condition or the number of iterations reaches a preset iteration condition, the fused features are taken as the target feature; When the fused features do not meet the preset condition or the number of iterations does not reach the preset iteration condition, the fused features are input into the self-attention layer again until the target feature is obtained.

3. The method of claim 2, wherein, The method comprises the following steps: The second candidate region feature and the label feature are input into the cross-attention layer to obtain a third query vector, a third key vector and a third value vector; and the features are fused through the third query vector, the third key vector and the third value vector to obtain the fused features.

4. The method of claim 1, wherein, The inputting the target feature and the candidate region set into a full connection layer to obtain a final position corresponding to the candidate region set and a category of an object contained in each of the candidate region set comprises: Adding the target feature and the candidate region set to obtain an added feature, and inputting the added feature into the full connection layer to obtain the final position and the category.

Images