Methods, apparatus, storage media, and electronic devices for determining the detection frame

Abstract

This invention provides a method, apparatus, storage medium, and electronic device for determining detection boxes. The method includes: inputting a target image and a first set of supporting images corresponding to the target image into a target detection model; performing a matching operation on a first feature and a second feature using a feature rematching network included in the target detection model; determining a first detection box for a target object included in the target image based on the matching operation; and filtering the first detection box using a target re-identification network to determine the target object's target detection box. This invention solves the problem in related technologies where methods for determining detection boxes cannot simultaneously guarantee high recall and precision, achieving a balance between high recall and precision.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of target detection based on deep learning, and more specifically, to a method, apparatus, storage medium, and electronic device for determining a detection box. Background Technology

[0002] Object detection is a fundamental and challenging task in computer vision, a hot research topic in academia and a popular application area in industry, widely used in intelligent transportation, security monitoring, and industrial inspection. Due to its speed, high accuracy, and mature iteration, deep learning-based object detection technology is the preferred choice for most object detection techniques. However, deep learning-based methods require collecting a large number of images containing the target category and annotating the bounding boxes before using the annotated image data to train the object detection model. The data collection and annotation process is extremely resource-intensive and time-consuming, and the trained model can only detect annotated and trained target categories, failing to detect new categories, resulting in poor generalizability. Few-shot object detection algorithms aim to extract features from a small number of new target samples to detect new categories, achieving more general object detection. The challenge lies in the limited number of new target samples, how to extract core features from them, and how to utilize these features for effective and accurate object detection.

[0003] In related technologies, a two-stage Faster-RCNN architecture is typically used as the basic framework. After extracting features from the input query image and support set images, the support set image features are enhanced to strengthen the query image features in a cross-image feature fusion module. This is then fed into an improved RPN module to generate ROI feature vectors. A feature aggregation module then filters candidate boxes and performs spatial alignment between the support set vectors and ROI feature vectors. Finally, the processed ROIs and support vectors are fed into a classifier for classification, ultimately outputting the target type and the accurate location of the detection box. However, the attention mechanism included in the cross-image feature fusion module used to enhance the classification and localization performance of few-shot target detection is not hardware-friendly and difficult to deploy on various industrial chips. The Faster-RCNN used as the backbone network in this design has a relatively high target detection time. Furthermore, the support image set used in this design consists of multiple images; while feature fusion from multiple images can effectively improve the model's recall, it also leads to increased false positives and reduced accuracy.

[0004] Therefore, it can be seen that the methods for determining the detection box in related technologies have the problem of not being able to simultaneously guarantee high recall and precision.

[0005] There is currently no effective solution to the aforementioned problems in the relevant technologies. Summary of the Invention

[0006] This invention provides a method, apparatus, storage medium, and electronic device for determining a detection frame, to at least solve the problem in related technologies that methods for determining a detection frame are difficult to simultaneously guarantee high recall and precision.

[0007] According to an embodiment of the present invention, a method for determining a detection box is provided, comprising: inputting an image to be detected and a first set of support images corresponding to the image to be detected into a target detection model; performing a matching operation on a first feature and a second feature using a feature rematching network included in the target detection model, wherein the first feature is a feature of the image to be detected extracted by a backbone network included in the target detection model, and the second feature is a feature of the first set of support images extracted by the backbone network; determining a first detection box of a target object included in the image to be detected based on the matching operation; and filtering the first detection box using a target re-identification network to determine a target detection box of the target object.

[0008] According to another embodiment of the present invention, a detection box determination device is provided, comprising: a matching module, configured to input an image to be detected and a first support image set corresponding to the image to be detected into a target detection model, and to perform a matching operation on a first feature and a second feature using a feature rematching network included in the target detection model, wherein the first feature is a feature of the image to be detected extracted by a backbone network included in the target detection model, and the second feature is a feature of the first support image set extracted by the backbone network; a first determination module, configured to determine a first detection box of a target object included in the image to be detected based on the matching operation; and a second determination module, configured to filter the first detection box using a target re-identification network to determine a target detection box of the target object.

[0009] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0010] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0011] This invention inputs the image to be detected and a first set of supporting images corresponding to the image to be detected into a target detection model. The feature rematching network included in the target detection model performs a matching operation on the first features of the image to be detected extracted by the backbone network and the second features of the first set of supporting images extracted by the backbone network. Based on the matching operation, a second detection box for the target object included in the image to be detected is determined. A target re-identification network is then used to filter the first detection box, thus determining the target detection box for the target object. Since the target detection box for the target object included in the image to be detected can be determined using both the target detection network and the target re-identification network, and the target detection network includes a trained feature rematching network, feature rematching and target re-identification can significantly improve the recall and effectiveness of small-sample target detection. Therefore, it can solve the problem in related technologies where methods for determining detection boxes cannot simultaneously guarantee high recall and precision, achieving a simultaneous guarantee of high recall and precision. Attached Figure Description

[0012] Figure 1 This is a hardware structure block diagram of a mobile terminal for a method of determining a detection frame according to an embodiment of the present invention.

[0013] Figure 2 This is a flowchart of a method for determining a detection frame according to an embodiment of the present invention;

[0014] Figure 3 This is a structural block diagram of a detection frame determination device according to an embodiment of the present invention. Detailed Implementation

[0015] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0017] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a method of determining a detection frame according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0018] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the detection frame determination method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0019] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0020] This embodiment provides a method for determining a detection box. Figure 2 This is a flowchart of a method for determining a detection frame according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0021] Step S202: Input the image to be detected and the first support image set corresponding to the image to be detected into the target detection model, and use the feature rematching network included in the target detection model to perform a matching operation on the first feature and the image, wherein the first feature is the feature of the image to be detected extracted by the backbone network included in the target detection model, and the second feature is the feature of the first support image set extracted by the backbone network.

[0022] Step S204: Determine a first detection box for the target object included in the image to be detected based on the matching operation;

[0023] Step S206: Filter the first detection box using the target re-identification network to determine the target detection box of the target object.

[0024] In the above embodiments, the object detection model can be a model obtained by training an initial detection model using training data. The object detection model may include a backbone network and a feature rematching network. After obtaining the object detection network, it can be used to identify the image to be detected. The image to be detected is input into the object detection model, and the backbone network of the object detection model can extract the first feature of the image to be detected. The first support image set is input into the object detection model, and the backbone network extracts the second feature of each first support image in the first support image set. The feature rematching network included in the object detection model performs feature rematching on the first and second features, i.e., a matching operation, to determine the first detection box of the target object included in the image to be detected. Then, the pre-trained object re-identification network is used to filter the first detection box to obtain the target detection box of the target object.

[0025] In the above embodiments, for the image Q to be detected and the category c to be detected in the image, N first support images with the category c target and the target bounding box coordinates can be prepared as a first support image set S. The image to be detected and the first support image set S are input into the target detection network to extract the backbone features SF. 4_avg2 The first set of supporting images S is input into the target re-identification network, and N feature vectors SR = [N, 512] are extracted. For each image Q to be detected, after the backbone features are extracted by the small sample detection network, SF4 and SF5 are directly read from memory or hard disk and then input into the RPN (Region Proposal Network) and Feature Rematching Network. The result of feature rematching is taken as the target detection result, i.e., the first detection box. The pre-trained target re-identification network is then used to filter the first detection box to obtain the target detection box of the target object.

[0026] The entity executing the above steps can be a processor, but is not limited to this.

[0027] This invention inputs the image to be detected and a first set of supporting images corresponding to the image to be detected into a target detection model. The feature rematching network included in the target detection model performs a matching operation on the first features of the image to be detected extracted by the backbone network and the second features of the first set of supporting images extracted by the backbone network. Based on the matching operation, a second detection box for the target object included in the image to be detected is determined. A target re-identification network is then used to filter the first detection box, thus determining the target detection box for the target object. Since the target detection box for the target object included in the image to be detected can be determined using both the target detection network and the target re-identification network, and the target detection network includes a trained feature rematching network, feature rematching and target re-identification can significantly improve the recall and effectiveness of small-sample target detection. Therefore, it can solve the problem in related technologies where methods for determining detection boxes cannot simultaneously guarantee high recall and precision, achieving a simultaneous guarantee of high recall and precision.

[0028] In an exemplary embodiment, before inputting the image to be detected and a first set of support images corresponding to the image to be detected into the object detection model, the method further includes: acquiring a query image and a second set of support images corresponding to the query image, wherein the objects included in each image in the second set of support images are of the same category as the objects included in the query image; and training an initial detection model using the query image and the second set of support images to obtain the object detection model. In this embodiment, the second set of support images S can be randomly selected from a small sample object detection dataset. p = [N, 3, W1, H1], query image Q = [1, 3, W2, H2], where N is the number of second support images included in the second support image set, and the target category of all N second support images is the same as the target category to be detected in the query image. The object category includes the type of object; for example, if the object in the query image is a soccer ball, the objects in the images in the second support image set are also soccer balls; if the object in the query image is an apple, the objects in the images in the second support image set are also apples. Both the images in the second support image set and the query image need to include the target's bounding box coordinates.

[0029] In the above embodiments, the initial detection model can be trained using the query image and the second support image set. The initial detection model includes an initial backbone network and an initial feature rematching network. The initial backbone network can extract the third features of the query image and the third features of the second support image set. The initial feature rematching network can predict the query detection boxes of objects included in the query image based on the third features. The initial backbone network can be an improved version of an existing network, for example, an improved version of the deep residual network ResNet-50.

[0030] In an exemplary embodiment, training an initial detection model using the query image and the second support image set to obtain a target detection model includes: acquiring a third support image set corresponding to the query image, wherein the type of objects included in the third support image set is the same as the type of objects included in the query image; and training the initial detection model using the query image, the second support image set, and the third support image set to obtain the target detection model. In this embodiment, to enhance the model's ability to distinguish between background and interference, another N support images can be randomly selected from the small sample target detection dataset as a negative sample support image set S. n That is, the third supporting image set, S n The N images have the same target category and are similar to S. p This means that the target categories of the query images are not completely identical, but belong to categories with similar appearances. Specifically, each image in the third support image set contains objects of the same type as the objects in the query image; for example, basketballs and soccer balls belong to the same ball category, and peaches and plums belong to the same fruit category. Simultaneously, the initial detection model is trained using the query image, the second support image set, and the third support image set to obtain the target detection model, improving its ability to distinguish between background and interference objects.

[0031] In an exemplary embodiment, training the initial detection model using the query image, the second support image set, and the third support image set to obtain the target detection model includes: for any target image in the query image, the second support image set, and the third support image set, performing the following operations to obtain a third feature corresponding to each target image: inputting the target image into the initial backbone network included in the initial detection model to obtain the third feature output by the Nth structural block of the initial backbone network; determining a query detection box for an object included in the query image based on the third feature and the initial region candidate network included in the initial detection model; extracting detection box features corresponding to the position of the query detection box from the query image; correcting the query detection box based on the third feature corresponding to the second support image set, the third feature corresponding to the third support image set, the detection box features, and the initial feature rematching network included in the initial detection model to obtain the second detection box, and determining a first confidence level for the second detection box; determining a target loss value for the initial detection model based on the second detection box and the first confidence level corresponding to the second detection box; and updating the network parameters of the initial detection model based on the target loss value to obtain the target detection model. In this embodiment, the second supporting image set S can be... p The query image Q and the input image Q are respectively fed into the initial backbone network, such as ResNet-50. In the 4th block of ResNet-50 (i.e., N is 4, where N is only an example, N can also be a positive integer such as 3 or 5, and this invention does not limit this), the features are obtained, which are the third features SF of the second support image set. p4 = [N,1024,w1,h1] and the third feature QF of the query image p4 ==[1,1024,w2,h2]. Similarly, for the third supporting image set S n The third support image set S can be extracted using the initial backbone network. n The third feature SF n4 The initial region candidate network (RPN) and the third feature, included in the initial network model, are used to determine the query detection box. Based on the location of the query detection box, the detection box features are determined from the query image. Specifically, the ROI feature vector, i.e., the detection box feature QF, can be extracted from the feature vector QF4 of the query image Q based on the coordinates of these M predicted boxes. 4_roi= [M, 1024, 14, 14]. The query detection box is corrected based on the third feature, the detection box features, and the initial feature rematching network. The corrected detection box is then determined as the second detection box. The initial feature rematching network can correct the query detection box to obtain the second detection box, improving the accuracy of determining the target detection box.

[0032] In the above embodiments, the second detection box and the query detection box can be multiple detection boxes. After determining the second detection box, an activation process can be used to determine the first confidence level of each second detection box. Based on the second detection box and its corresponding first confidence level, the target loss value of the initial detection model is determined. The network parameters of the initial network model are then iteratively updated based on the target loss value to obtain the target detection model. The activation process can include sigmoid activation.

[0033] In an exemplary embodiment, determining the query detection box of the object included in the query image based on the third feature and the initial region candidate network included in the initial detection model includes: performing global max pooling and mean fusion operations sequentially on the third feature corresponding to the second supporting image set to obtain a fourth feature; performing a broadcast dot product between the third feature corresponding to the query image and the fourth feature to obtain a fifth feature; performing global max pooling and mean fusion operations sequentially on the third feature corresponding to the third supporting image set to obtain a sixth feature; performing a broadcast dot product between the third feature corresponding to the query image and the sixth feature to obtain a seventh feature; inputting the fifth feature into the initial region candidate network to obtain a first classification feature and a first localization feature, and inputting the seventh feature into the initial region subsequent network to obtain a second classification feature and a second localization feature; and determining the query detection box based on the first classification feature, the first localization feature, the second classification feature, and the second localization feature. In this embodiment, the third feature SF of the second supporting image set can be... p4 Perform global max pooling on [N, 1024, w1, h1] to obtain SF. 4_avg1 =[N,1024,1,1], then for SF 4_avg1 =[N,1024,1,1] undergoes mean fusion to obtain the fourth feature SF 4_avg2 = [1,1024,1,1], and then perform a broadcast dot product between the fourth feature and the third feature of the query image to obtain the aggregated feature, i.e., the fifth feature F. p1 = [1, 1024, w2, h2]. By using broadcast dot product instead of the Transformer attention mechanism, a hardware-friendly attention mechanism can be implemented. The attention mechanism is simple, effective, and easy to deploy.

[0034] In the above embodiments, the same operation can be performed on the third feature of the third supporting image set, performing global max pooling and mean fusion on the third feature to obtain the sixth feature, and then broadcasting the third feature of the query image and the sixth feature to obtain the seventh feature F. n1 .

[0035] In the above embodiments, the aggregation feature, i.e., the fifth feature F, can be... p1 The data is fed into the RPN network (i.e., the initial region candidate network) to obtain the first classification feature C. rpn and the first localization feature L rpn The aggregated features, namely the seventh feature F, will be used. n1 The data is fed into the RPN network to obtain the second classification feature and the second location feature. The query detection box is determined based on the first classification feature, the first location feature, the second classification feature, and the second location feature.

[0036] In an exemplary embodiment, determining the query detection box based on the first classification feature, the first positioning feature, the second classification feature, and the second positioning feature includes: determining a first positioning prediction box for an object included in the query image based on the first positioning feature; determining a first foreground prediction box and a first background prediction box from the first positioning prediction box; determining a second background prediction box for the object included in the query image based on the second positioning feature; selecting a second positioning prediction box and a third background prediction box from the first foreground prediction box, the first background prediction box, and the second background prediction box; determining a second confidence level of the second positioning prediction box and a third confidence level of the third background prediction box based on the first classification feature and the second classification feature; performing non-maximum suppression on the second positioning prediction box and the third background prediction box based on the second confidence level and the third confidence level to obtain a third positioning prediction box; and determining the third positioning prediction box as the query detection box. In this embodiment, the first positioning feature L can be used in conjunction with a preset anchor frame size. rpn The initial location coordinate prediction box, i.e., the first location prediction box, is obtained. Based on the intersection-union ratio (IUU) between the first location prediction box and the target box in the query image Q, prediction boxes greater than a first threshold are designated as first foreground prediction boxes. The first threshold can be 0.6 (it should be noted that this value is only an example; the first threshold can also be 0.5, 0.7, etc., and this invention does not limit this). Prediction boxes less than a second threshold (the second threshold can be 0.3; this value is only an example; the first threshold can also be 0.2, 0.4, etc., and this invention does not limit this) are designated as first background prediction boxes. Additionally, sigmoid activation can be applied to the first classification feature to obtain the confidence score of each prediction box. Similarly, the aggregated feature F... n1The data is fed into the RPN network, and all predicted bounding boxes obtained are treated as second background predicted bounding boxes. A portion of the foreground predicted bounding boxes and a portion of the background predicted bounding boxes are randomly selected, and non-maximum suppression is applied to these boxes. Finally, M predicted bounding boxes remain, which are the third localization predicted bounding boxes. These third localization predicted bounding boxes are determined as query detection boxes.

[0037] In an exemplary embodiment, the process of refining the query detection box based on the third feature corresponding to the second supporting image set, the third feature corresponding to the third supporting image set, the detection box feature, and the initial feature rematching network included in the initial detection model to obtain the second detection box includes: extracting an eighth feature from the third feature corresponding to the second supporting image set based on the first bounding box coordinates of each image in the second supporting image set; extracting a ninth feature from the third feature corresponding to the third supporting image set based on the second bounding box coordinates of each image in the third supporting image set; performing the following operations for any one of the eighth feature, the ninth feature, and the detection box feature to obtain a tenth feature corresponding to each feature: inputting the first target feature into the Mth structural block of the initial backbone network to obtain the tenth feature; performing a mean fusion operation on the tenth feature corresponding to the eighth feature to obtain an eleventh feature, and performing a mean fusion operation on the tenth feature corresponding to the ninth feature to obtain a twelfth feature; and inputting the tenth feature corresponding to the eighth feature, the eleventh feature, and the twelfth feature into the initial feature rematching network to refine the query detection box and obtain the second detection box. In this embodiment, for the second supporting image set S p The third feature SF p4 Using the coordinates of its bounding box, i.e., the coordinates of the first bounding box, we extract its ROI feature vector, i.e., the eighth feature SF. p4_roi = [N, 1024, 14, 14], for the negative sample support image set, i.e., the third support image set S n The third feature SF n4 The same method is used to obtain the ROI feature vector, namely the ninth feature SF. n4_roi = [N, 1024, 14, 14], then QF 4_roi and SF p4_roi and SF n4_roi These are fed into the initial backbone network, such as the Mth block of ResNet-50, to obtain the tenth feature QF5 = [M, 2048, 7, 7] corresponding to the eighth feature and the tenth feature SF corresponding to the ninth feature. p5 = [N,2048,7,7] and the tenth feature SF corresponding to the detection box features. n5=[N,2048,7,7], then for SF p5 and SF n5 The mean fusion was performed separately to obtain the eleventh feature SF of the feature vector. p5_avg =[1,1024,7,7] and the twelfth feature SF n5_avg =[1,1024,7,7], finally QF5 and SF p5 and SF n5 The data is fed into the initial feature rematching network to correct the query detection box. Here, M is a positive integer, such as 5. This value is merely an example; M can also be 4, 6, etc., and this invention does not impose any limitations on this.

[0038] In an exemplary embodiment, inputting the tenth feature, the eleventh feature, and the twelfth feature corresponding to the eighth feature into the initial feature rematching network to correct the query detection box and obtain the second detection box includes: performing global mean pooling on the tenth feature corresponding to the eighth feature to obtain the thirteenth feature; performing the following operations on any one of the eleventh and twelfth features as a second target feature to obtain the second detection box: performing global mean pooling on the second target feature to obtain the fourteenth feature; performing channel merging on the thirteenth and fourteenth features to obtain the fifteenth feature; sequentially inputting the fifteenth feature into a first number of fully connected layers to obtain a third classification feature; and then performing the following operations on the query detection box: performing global mean pooling on the second target feature to obtain the fourteenth feature; performing channel merging on the thirteenth and fourteenth features to obtain the fifteenth feature; and sequentially inputting the fifteenth feature into a first number of fully connected layers to obtain the ... finally performing the following operations on the query detection box: performing global mean pooling on the second target feature to obtain the fourteenth feature; performing channel merging on the thirteenth and fourteenth features to obtain the fifteenth feature; and finally inputting the query detection box into the initial feature rematching network to correct the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to obtain the query detection box to The thirteenth and fourteenth features are sequentially subjected to convolution and global mean pooling operations to obtain the sixteenth feature. The sixteenth feature is then input into a fully connected layer to obtain the fourth classification feature. The fifteenth feature is sequentially subjected to A×A convolution, B×B pooling downsampling, C×C convolution, D×D convolution, and E×E pooling downsampling operations to obtain the seventeenth feature, where A, B, C, D, and E are all positive integers. The seventeenth feature is then sequentially input into a second number of fully connected layers to obtain the fifth classification feature and the third localization feature. Based on the third classification feature, the fourth classification feature, the fifth classification feature, and the third localization feature, the query detection box is corrected to obtain the second detection box. In this embodiment, the initial feature rematching network may include a global matching module, a local matching module, and a block matching module. The final feature vector, i.e., the tenth, eleventh, and twelfth features corresponding to the eighth feature, can be fed into the initial feature rematching network to obtain the second detection box.

[0039] The global matching module has the following structure: it matches QF5 and SF respectively. p5_avgGlobal mean pooling is performed to merge the features obtained from the two methods, and then the result is passed through three fully connected layers to obtain the third classification feature C1. It should be noted that the first number can be 3, or it can be 2, 4, etc.; this invention does not impose any restrictions on this.

[0040] The structure of the local matching module is as follows: QF5 and SF are matched respectively. p5_avg Perform a convolution operation, which can be a 1x1 convolution, and multiply the features obtained from the two convolutions element by element. Then, perform global mean pooling on the obtained features, and then pass them through a fully connected layer to finally obtain the fourth classification feature C2.

[0041] The block matching module has the following structure: First, it matches QF5 and SF... p5_avg Channel merging is performed, and then the merged features are successively processed through A×A convolution, B×B pooling downsampling, C×C convolution, D×D convolution, and E×E pooling downsampling, and then passed through a second number of fully connected layers to obtain the fifth classification feature C2 and the third localization feature L. Here, A can be 1, B can be 3, C can be 3, D can be 1, E can be 3, and the second number can be 2. It should be noted that the above values ​​for A and E and the second number are merely illustrative and are not intended to limit the scope of this invention.

[0042] For QF5 and SF n5_avg Using the same procedure, the final classification and localization features of the negative samples can be obtained. The query detection box is then modified based on these features to obtain the second detection box.

[0043] In an exemplary embodiment, correcting the query detection box based on the third classification feature, the fourth classification feature, the fifth classification feature, and the third localization feature to obtain the second detection box includes: determining the offset of the query detection box based on the third localization feature; correcting the query detection box based on the offset to obtain the third detection box; merging the third classification feature, the fourth classification feature, and the fifth classification feature to obtain the target classification feature; determining the fourth confidence level of the third detection box based on the target classification feature; and applying non-maximum suppression to the third detection box based on the fourth confidence level to obtain the second detection box. In this embodiment, the third localization feature, i.e., feature L, can be determined as the final localization feature, and feature C = C1 + C2 + C3 can be determined as the final classification feature, i.e., the target classification feature. By correcting M query detection boxes using feature L, the final prediction box, i.e., the third detection box, can be obtained. By applying sigmoid activation to feature C to obtain the fourth confidence level of each third detection box, and finally applying non-maximum suppression to the final third detection box, the final target detection result, i.e., the second detection box, can be obtained.

[0044] For QF5 and SF n5_avg The same operation was used to obtain the final negative sample target detection results.

[0045] In an exemplary embodiment, determining the target loss value of the initial detection model based on the second detection box and the first confidence level corresponding to the second detection box includes: determining the foreground prediction box and the background prediction box determined by the initial detection model; determining a first loss value between the foreground prediction box and the third labeled box of the query image; determining a fifth confidence level of the foreground prediction box and a sixth confidence level of the background prediction box based on the first confidence level; determining a second loss value between the fifth confidence level and a first constant, and determining a third loss value between the sixth confidence level and a second constant; determining a fourth loss value between the second detection box and the third labeled box; and determining the target loss value based on the first loss value, the second loss value, the third loss value, and the fourth loss value. In this embodiment, the target loss value of the initial detection model can be determined based on the second detection box, the foreground detection box, and the background detection box determined by the initial detection model. The loss between the foreground detection box and the true target box of the query image is calculated separately. The foreground and background predicted boxes within the second detection box can be determined. For the coordinates of the foreground predicted box, the L2 loss between it and the ground truth target box (the third labeled box) is calculated. No loss is calculated for the coordinates of the background predicted box. For the confidence score of the foreground predicted box, the target category is defined as 1, and the cross-entropy loss between it and 1 is calculated. For the confidence score of the background predicted box, the background category is defined as 0, and the cross-entropy loss between it and 0 is calculated. Similarly, for the second detection box, the L2 loss and the cross-entropy loss of the coordinate box confidence are calculated. The first constant can be 1, and the second constant can be 0. After obtaining the first, second, third, and fourth loss values, the sum of these four losses can be determined as the target loss value. The weights corresponding to each of the first, second, third, and fourth loss values ​​can also be determined, and the weighted sum is determined as the target loss value. After determining the target loss value, the backpropagation algorithm can be used to backpropagate the gradient of the loss, thereby training the network.

[0046] In an exemplary embodiment, before training an initial detection model using the query image and the second support set images to obtain the target detection model, the method further includes: training a first initial network model using the target image dataset to obtain a first network model; removing the fully connected layers included in the first network model to obtain a second network model; determining the second network model as a teacher model; training the second initial network model using the target image dataset to obtain a third network model; determining the third network model as a student model, wherein the number of parameters in the second network model is greater than the number of parameters in the third network model; and determining the model including the teacher model and the student model as the backbone network of the initial detection model. In this embodiment, since the dataset for few-shot target detection is small, the pre-training of the backbone network is important for the generalization ability of the backbone network after being transferred to the few-shot target detection task. The first initial network can be a ResNet-50 deep residual network. A ResNet-50 deep residual network can be used as the feature extraction backbone network. The ResNet-50 network was trained using the large-scale image dataset ImageNet-21k containing 20,000 object classes. After training, the fully connected layers were removed and the network was used as the backbone for feature extraction.

[0047] In the above embodiment, the second initial network model is a ResNet-34 model. Because the ResNet-50 network has a large number of parameters, the few-shot detection network is time-consuming. Therefore, a few-shot detection model containing a ResNet-50 backbone network can be used as the teacher model, and a few-shot detection model containing a ResNet-34 structure (replacing the ResNet-50 structure) can be used as the student model. Network distillation training is then performed. Compared to the teacher model, the distilled student model has only a slight decrease in accuracy, but the number of model parameters and computation time are reduced by nearly half. The distillation model can reduce the time consumption of few-shot object detection and improve the object detection speed.

[0048] In an exemplary embodiment, filtering the first detection boxes using a target re-identification network to determine the target detection box of the target object includes: determining a sub-image to be detected corresponding to each of the first detection boxes from the image to be detected; inputting each sub-image to be detected into the target re-identification network to obtain an eighteenth feature; for each eighteenth feature, performing the following operations to determine the seventh confidence level of the detection box corresponding to each eighteenth feature: determining the target similarity between the eighteenth feature and the nineteenth feature of each image in the first supporting image set, and determining the maximum similarity included in the target similarity as the seventh confidence level; determining the detection box corresponding to the confidence level greater than a predetermined threshold included in the seventh confidence level as the target detection box. In this embodiment, determining the sub-image to be detected corresponding to each of the first detection boxes from the image to be detected includes: using the sub-image to be detected extracted according to the coordinate frame of the first detection box as the sub-image to be detected of the target to be matched. The first detection box can have K frames. These K sub-images are input into the target re-identification network to obtain feature vectors, specifically the eighteenth feature KR = [K, 512]. For each of the K feature vectors, KR... i =[1,512], all of which are compared with the N feature vectors SR of the first supporting image set. i =[1,512] Calculate the cosine similarity and take the maximum value as KR. i The similarity with SR is used as the vector KR. i The confidence level of the corresponding predicted bounding box. By setting a certain threshold for this confidence level, we can retain reliable predicted bounding boxes and filter out unreliable ones, thus obtaining the final target detection result, i.e., the target detection box.

[0049] In an exemplary embodiment, before filtering the first detection box using the target re-identification network to determine the target detection box of the target object, the method further includes: acquiring a first dataset, a second dataset, and a third dataset, wherein each image in the first dataset includes the object to be identified, each image in the second dataset includes the object to be identified, the images in the second dataset are captured under different conditions than the images included in the first dataset, and each image in the third dataset includes objects of the same category as the object to be identified; extracting a twentieth feature from the first dataset using an initial re-identification network, extracting a twentieth-first feature from the second dataset using the initial re-identification network, and extracting a twentieth-second feature from the third dataset using the initial re-identification network; determining a triplet loss value among the twentieth, twentieth-first, and twentieth-second features; iteratively updating the network parameters of the initial re-identification network based on the triplet loss value to obtain the target re-identification network. In this embodiment, the RPN network part obtains the preliminary target detection results, and the feature re-matching part significantly enhances the preliminary detection results. To further enhance the target detection effect, a target re-identification network can be constructed for further false positive sample filtering. The object re-identification network can use ResNet-18, which has a small number of parameters, as the backbone network and adopt the Re-ID dataset. An image of any object, Obj1 = [1, 3, W1, H1], is selected from the dataset as the target to be matched. Then, objects with the same ID are selected from the dataset, i.e., images of the same object under different background, lighting, and angle conditions, such as Obj2 = [1, 3, W1, H1], and objects of the same category but with different IDs, such as images of cups with identical shapes but different colors, Obj3 = [1, 3, W1, H1]. For Obj1, Obj2, and Obj3, the network is input to extract features SR1 = [N, 512], SR2 = [N, 512], and SR3 = [N, 512], respectively. Then, a Triplet loss is calculated on the three features to train the network.

[0050] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0051] This embodiment also provides a detection frame determination device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0052] Figure 3 This is a structural block diagram of a detection frame determination device according to an embodiment of the present invention, as shown below. Figure 3 As shown, the device includes:

[0053] The matching module 32 is used to input the image to be detected and the first support image set corresponding to the image to be detected into the target detection model, and use the feature rematching network included in the target detection model to perform matching operations on the first feature and the second feature, wherein the first feature is the feature of the image to be detected extracted by the backbone network included in the target detection model, and the second feature is the feature of the first support image set extracted by the backbone network.

[0054] The first determining module 34 is used to determine a first detection box of the target object included in the image to be detected based on the matching operation;

[0055] The second determining module 36 is used to filter the first detection box using a target re-identification network to determine the target detection box of the target object.

[0056] In an exemplary embodiment, the apparatus may be used to obtain a query image and a second support image set corresponding to the query image before inputting the image to be detected and a first support image set corresponding to the image to be detected into the target detection model, wherein the objects included in each image in the second support image set are of the same category as the objects included in the query image; and to train an initial detection model using the query image and the second support image set to obtain the target detection model.

[0057] In an exemplary embodiment, the apparatus can train an initial detection model using the query image and the second support image set to obtain a target detection model by: acquiring a third support image set corresponding to the query image, wherein the type of objects included in the third support image set is the same as the type of objects included in the query image; and training the initial detection model using the query image, the second support image set, and the third support image set to obtain the target detection model.

[0058] In an exemplary embodiment, the apparatus can train the initial detection model using the query image, the second support image set, and the third support image set to obtain the target detection model in the following manner: For any target image in the query image, the second support image set, and the third support image set, the following operations are performed to obtain a third feature corresponding to each target image: The target image is input into the initial backbone network included in the initial detection model to obtain the third feature output by the Nth structural block of the initial backbone network; Based on the third feature and the initial region candidate network included in the initial detection model, the target image is determined to be a target image. The process involves: identifying query detection boxes for objects included in a query image; extracting detection box features corresponding to the positions of the query detection boxes from the query image; refining the query detection boxes using a rematching network based on the third features corresponding to the second support image set, the third features corresponding to the third support image set, the detection box features, and the initial features included in the initial detection model, to obtain a second detection box and determine a first confidence level for the second detection box; determining the target loss value of the initial detection model based on the second detection box and the first confidence level corresponding to the second detection box; and updating the network parameters of the initial detection model based on the target loss value to obtain the target detection model.

[0059] In an exemplary embodiment, the apparatus can determine a query detection box for an object included in a query image based on the third feature and an initial region candidate network included in the initial detection model in the following manner: Performing global max pooling and mean fusion operations sequentially on the third feature corresponding to the second supporting image set to obtain a fourth feature; performing a broadcast dot product between the third feature corresponding to the query image and the fourth feature to obtain a fifth feature; performing global max pooling and mean fusion operations sequentially on the third feature corresponding to the third supporting image set to obtain a sixth feature; performing a broadcast dot product between the third feature corresponding to the query image and the sixth feature to obtain a seventh feature; inputting the fifth feature into the initial region candidate network to obtain a first classification feature and a first localization feature, and inputting the seventh feature into the initial region subsequent network to obtain a second classification feature and a second localization feature; and determining the query detection box based on the first classification feature, the first localization feature, the second classification feature, and the second localization feature.

[0060] In an exemplary embodiment, the apparatus can determine the query detection box based on the first classification feature, the first positioning feature, the second classification feature, and the second positioning feature in the following manner: determining a first positioning prediction box for an object included in the query image based on the first positioning feature; determining a first foreground prediction box and a first background prediction box from the first positioning prediction box; determining a second background prediction box for an object included in the query image based on the second positioning feature; selecting a second positioning prediction box and a third background prediction box from the first foreground prediction box, the first background prediction box, and the second background prediction box; determining a second confidence level of the second positioning prediction box and a third confidence level of the third background prediction box based on the first classification feature and the second classification feature; performing non-maximum suppression on the second positioning prediction box and the third background prediction box based on the second confidence level and the third confidence level to obtain a third positioning prediction box; and determining the third positioning prediction box as the query detection box.

[0061] In an exemplary embodiment, the apparatus can obtain the second detection box by correcting the query detection box based on the third feature corresponding to the second supporting image set, the third feature corresponding to the third supporting image set, the detection box feature, and the initial feature rematching network included in the initial detection model, in the following manner: extracting an eighth feature from the third feature corresponding to the second supporting image set based on the first bounding box coordinates of each image in the second supporting image set; extracting a ninth feature from the third feature corresponding to the third supporting image set based on the second bounding box coordinates of each image in the third supporting image set; and for the eighth feature, Each of the ninth feature and any of the first target features in the detection box performs the following operations to obtain a tenth feature corresponding to each feature: inputting the first target feature into the Mth structural block of the initial backbone network to obtain the tenth feature; performing a mean fusion operation on the tenth feature corresponding to the eighth feature to obtain the eleventh feature, and performing a mean fusion operation on the tenth feature corresponding to the ninth feature to obtain the twelfth feature; inputting the tenth feature, the eleventh feature, and the twelfth feature corresponding to the eighth feature into the initial feature rematching network to correct the query detection box and obtain the second detection box.

[0062] In an exemplary embodiment, the apparatus can input the tenth, eleventh, and twelfth features corresponding to the eighth feature into the initial feature rematching network to correct the query detection box and obtain the second detection box in the following manner: Perform global mean pooling on the tenth feature corresponding to the eighth feature to obtain the thirteenth feature; perform the following operations on any one of the eleventh and twelfth features as a second target feature to obtain the second detection box: Perform global mean pooling on the second target feature to obtain the fourteenth feature; Perform channel merging on the thirteenth and fourteenth features to obtain the fifteenth feature; Input the fifteenth feature sequentially into a first number of fully connected layers to obtain the third classification. Features; convolution and global mean pooling are performed sequentially on the thirteenth and fourteenth features to obtain the sixteenth feature; the sixteenth feature is input into a fully connected layer to obtain the fourth classification feature; A×A convolution, B×B pooling downsampling, C×C convolution, D×D convolution, and E×E pooling downsampling are performed sequentially on the fifteenth feature to obtain the seventeenth feature, wherein A, B, C, D, and E are all positive integers; the seventeenth feature is input into a second number of fully connected layers to obtain the fifth classification feature and the third localization feature; the query detection box is corrected based on the third classification feature, the fourth classification feature, the fifth classification feature, and the third localization feature to obtain the second detection box.

[0063] In an exemplary embodiment, the apparatus can obtain a second detection box by correcting the query detection box based on the third classification feature, the fourth classification feature, the fifth classification feature, and the third location feature in the following manner: determining the offset of the query detection box based on the third location feature; correcting the query detection box based on the offset to obtain a third detection box; merging the third classification feature, the fourth classification feature, and the fifth classification feature to obtain a target classification feature; determining the fourth confidence level of the third detection box based on the target classification feature; and performing non-maximum suppression on the third detection box based on the fourth confidence level to obtain the second detection box.

[0064] In an exemplary embodiment, the apparatus can determine the target loss value of the initial detection model based on the second detection box and the first confidence level corresponding to the second detection box in the following manner: determining the foreground prediction box and the background prediction box determined by the initial detection model; determining a first loss value between the foreground prediction box and the third annotation box of the query image; determining a fifth confidence level of the foreground prediction box and a sixth confidence level of the background prediction box based on the first confidence level; determining a second loss value between the fifth confidence level and a first constant, and determining a third loss value between the sixth confidence level and a second constant; determining a fourth loss value between the first detection box and the third annotation box; and determining the target loss value based on the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0065] In an exemplary embodiment, the apparatus may be configured to, before training an initial detection model using the query image and the second support set images to obtain the target detection model, train a first initial network model using a target image dataset to obtain a first network model; remove fully connected layers from the first network model to obtain a second network model; determine the second network model as a teacher model; train the second initial network model using the target image dataset to obtain a third network model; determine the third network model as a student model, wherein the number of parameters in the second network model is greater than the number of parameters in the third network model; and determine the model including the teacher model and the student model as the backbone network of the initial detection model.

[0066] In an exemplary embodiment, the second determining module 36 can determine the target detection box of the target object by filtering the first detection box using a target re-identification network in the following manner: determining the sub-image to be detected corresponding to each of the first detection boxes from the image to be detected; inputting each sub-image to be detected into the target re-identification network to obtain the eighteenth feature; for each eighteenth feature, performing the following operations to determine the seventh confidence level of the detection box corresponding to each eighteenth feature: determining the target similarity between the eighteenth feature and the nineteenth feature of each image in the first supporting image set, and determining the maximum similarity included in the target similarity as the seventh confidence level; determining the detection box corresponding to the confidence level greater than a predetermined threshold included in the seventh confidence level as the target detection box.

[0067] In an exemplary embodiment, the apparatus may be configured to, before filtering the first detection box using a target re-identification network to determine the target detection box of the target object, acquire a first dataset, a second dataset, and a third dataset, wherein each image in the first dataset includes the object to be identified, each image in the second dataset includes the object to be identified, the images in the second dataset are captured under different conditions than the images included in the first dataset, and each image in the third dataset includes objects of the same category as the object to be identified; extract a twentieth feature from the first dataset using an initial re-identification network, extract a twentieth-first feature from the second dataset using the initial re-identification network, and extract a twentieth-second feature from the third dataset using the initial re-identification network; determine a triplet loss value among the twentieth feature, the twentieth-first feature, and the twentieth-second feature; and iteratively update the network parameters of the initial re-identification network based on the triplet loss value to obtain the target re-identification network.

[0068] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0069] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0070] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0071] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0072] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0073] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0074] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining a detection frame, characterized in that, include: The image to be detected and the first support image set corresponding to the image to be detected are input into the target detection model. The feature rematching network included in the target detection model is used to perform a matching operation on the first feature and the second feature. The first feature is the feature of the image to be detected extracted by the backbone network included in the target detection model, and the second feature is the feature of the first support image set extracted by the backbone network. Based on the matching operation, a first detection box for the target object included in the image to be detected is determined; The first detection box is filtered using a target re-identification network to determine the target detection box of the target object.

2. The method according to claim 1, characterized in that, Before inputting the image to be detected and the first set of supporting images corresponding to the image to be detected into the target detection model, the method further includes: Obtain a query image and a second set of supporting images corresponding to the query image, wherein each image in the second set of supporting images includes objects of the same category as the objects included in the query image; The initial detection model is trained using the query image and the second support image set to obtain the target detection model.

3. The method according to claim 2, characterized in that, The initial detection model is trained using the query image and the second support image set to obtain the target detection model, which includes: Obtain a third supporting image set corresponding to the query image, wherein the type of objects included in the third supporting image set is the same as the type of objects included in the query image; The initial detection model is trained using the query image, the second support image set, and the third support image set to obtain the target detection model.

4. The method according to claim 3, characterized in that, The initial detection model is trained using the query image, the second support image set, and the third support image set to obtain the target detection model, which includes: For any target image in the query image, the second support image set, and the third support image set, the following operations are performed to obtain the third feature corresponding to each target image: input the target image into the initial backbone network included in the initial detection model to obtain the third feature output by the Nth structural block of the initial backbone network; Based on the third feature and the initial region candidate network included in the initial detection model, the query detection box of the object included in the query image is determined; Extract detection box features corresponding to the position of the query detection box from the query image; Based on the third feature corresponding to the second support image set, the third feature corresponding to the third support image set, the detection box feature, and the initial feature rematching network included in the initial detection model, the query detection box is corrected to obtain the second detection box, and the first confidence of the second detection box is determined. The target loss value of the initial detection model is determined based on the second detection box and the first confidence level corresponding to the second detection box. The network parameters of the initial detection model are updated based on the target loss value to obtain the target detection model.

5. The method according to claim 4, characterized in that, Based on the third feature and the initial region candidate network included in the initial detection model, the query detection box for the objects included in the query image is determined as follows: The third feature corresponding to the second supporting image set is subjected to global max pooling and mean fusion operations in sequence to obtain the fourth feature; The third feature corresponding to the query image is broadcast multiplied with the fourth feature to obtain the fifth feature; The third feature corresponding to the third support image set is subjected to global max pooling and mean fusion operations in sequence to obtain the sixth feature; The third feature corresponding to the query image is broadcast multiplied with the sixth feature to obtain the seventh feature; The fifth feature is input into the initial region candidate network to obtain the first classification feature and the first localization feature, and the seventh feature is input into the initial region subsequent network to obtain the second classification feature and the second localization feature; The query detection box is determined based on the first classification feature, the first location feature, the second classification feature, and the second location feature.

6. The method according to claim 5, characterized in that, The query detection box is determined based on the first classification feature, the first location feature, the second classification feature, and the second location feature, including: Based on the first positioning feature, a first positioning prediction box for the object included in the query image is determined; A first foreground prediction box and a first background prediction box are determined from the first positioning prediction box; Based on the second localization feature, a second background prediction box including the object in the query image is determined; Select a second positioning prediction box and a third background prediction box from the first foreground prediction box, the first background prediction box, and the second background prediction box; The second confidence level of the second localization prediction box and the third confidence level of the third background prediction box are determined based on the first classification feature and the second classification feature. Based on the second confidence level and the third confidence level, non-maximum suppression is applied to the second localization prediction box and the third background prediction box to obtain the third localization prediction box; The third positioning prediction box is determined as the query detection box.

7. The method according to claim 4, characterized in that, Based on the third feature corresponding to the second supporting image set, the third feature corresponding to the third supporting image set, the detection box feature, and the initial feature rematching network included in the initial detection model, the query detection box is corrected to obtain the second detection box, which includes: Based on the coordinates of the first bounding box of each image in the second support image set, the eighth feature is extracted from the third feature corresponding to the second support image set; The ninth feature is extracted from the third feature corresponding to the third support image set based on the coordinates of the second bounding box of each image in the third support image set; For any one of the eighth feature, the ninth feature, and the detection box feature, the following operation is performed to obtain the tenth feature corresponding to each feature: the first target feature is input into the Mth structural block of the initial backbone network to obtain the tenth feature; A mean fusion operation is performed on the tenth feature corresponding to the eighth feature to obtain the eleventh feature, and a mean fusion operation is performed on the tenth feature corresponding to the ninth feature to obtain the twelfth feature; The tenth, eleventh, and twelfth features corresponding to the eighth feature are input into the initial feature rematching network to correct the query detection box and obtain the second detection box.

8. The method according to claim 7, characterized in that, The tenth, eleventh, and twelfth features corresponding to the eighth feature are input into the initial feature rematching network to correct the query detection box, resulting in the second detection box comprising: Perform global mean pooling on the tenth feature corresponding to the eighth feature to obtain the thirteenth feature; For any one of the eleventh and twelfth second target features, perform the following operations to obtain the second detection box: The second target feature is subjected to global mean pooling to obtain the fourteenth feature; the thirteenth and fourteenth features are combined to obtain the fifteenth feature; the fifteenth feature is sequentially input into a first number of fully connected layers to obtain the third classification feature; the thirteenth and fourteenth features are sequentially subjected to convolution and global mean pooling to obtain the sixteenth feature; the sixteenth feature is input into a fully connected layer to obtain the fourth classification feature; the fifteenth feature is sequentially subjected to A×A convolution, B×B pooling downsampling, C×C convolution, D×D convolution, and E×E pooling downsampling to obtain the seventeenth feature, wherein A, B, C, D, and E are all positive integers; the seventeenth feature is sequentially input into a second number of fully connected layers to obtain the fifth classification feature and the third localization feature; the query detection box is corrected based on the third classification feature, the fourth classification feature, the fifth classification feature, and the third localization feature to obtain the second detection box.

9. The method according to claim 8, characterized in that, Based on the third classification feature, the fourth classification feature, the fifth classification feature, and the third positioning feature, the query detection box is corrected to obtain the second detection box, which includes: The offset of the query detection box is determined based on the third positioning feature; The query detection box is corrected based on the offset to obtain a third detection box; The target classification feature is obtained by merging the third classification feature, the fourth classification feature, and the fifth classification feature; The fourth confidence level of the third detection box is determined based on the target classification features; The third detection box is subjected to non-maximum suppression based on the fourth confidence level to obtain the second detection box.

10. The method according to claim 4, characterized in that, Determining the target loss value of the initial detection model based on the second detection box and the first confidence level corresponding to the second detection box includes: Determine the foreground prediction box and background prediction box determined by the initial detection model; Determine a first loss value between the foreground prediction box and the third labeled box of the query image; The fifth confidence level of the foreground prediction box and the sixth confidence level of the background prediction box are determined based on the first confidence level. Determine the second loss value between the fifth confidence level and the first constant, and determine the third loss value between the sixth confidence level and the second constant; Determine a fourth loss value between the second detection box and the third annotation box; The target loss value is determined based on the first loss value, the second loss value, the third loss value, and the fourth loss value.

11. The method according to claim 2, characterized in that, Before training an initial detection model using the query image and the second support image set to obtain the target detection model, the method further includes: The first initial network model is obtained by training the target image dataset; The second network model is obtained by removing the fully connected layers from the first network model. The second network model is determined to be the teacher model; A second initial network model is trained using the target image dataset to obtain a third network model; The third network model is determined as the student model, wherein the number of parameters of the second network model is greater than the number of parameters of the third network model; The model including the teacher model and the student model is determined as the backbone network of the initial detection model.

12. The method according to claim 1, characterized in that, The target detection boxes for determining the target object include those obtained by filtering the first detection boxes using a target re-identification network: Determine the sub-image to be detected corresponding to each of the first detection boxes from the image to be detected; Each of the sub-images to be detected is input into the target re-identification network to obtain the eighteenth feature; For each of the eighteenth features, the following operations are performed to determine the seventh confidence level of the detection box corresponding to each of the eighteenth features: determine the target similarity between the eighteenth feature and the nineteenth feature of each image in the first support image set, and determine the maximum similarity included in the target similarity as the seventh confidence level; The detection boxes corresponding to the confidence levels greater than a predetermined threshold included in the seventh confidence level are determined as the target detection boxes.

13. The method according to claim 1, characterized in that, Before using a target re-identification network to filter the first detection box and determine the target detection box of the target object, the method further includes: Obtain a first dataset, a second dataset, and a third dataset, wherein each image in the first dataset contains an object to be identified, each image in the second dataset contains the object to be identified, the images in the second dataset are taken under different conditions than the images in the first dataset, and each image in the third dataset contains an object of the same category as the object to be identified. The twentieth feature of the first dataset is extracted using the initial re-identification network, the twenty-first feature of the second dataset is extracted using the initial re-identification network, and the twenty-second feature of the third dataset is extracted using the initial re-identification network. Determine the triplet loss value among the twentieth feature, the twentieth feature, and the twentieth feature; The network parameters of the initial re-identification network are iteratively updated based on the triplet loss value to obtain the target re-identification network.

14. A device for determining a detection frame, characterized in that, include: The matching module is used to input the image to be detected and the first support image set corresponding to the image to be detected into the target detection model, and to use the feature rematching network included in the target detection model to perform matching operations on the first feature and the second feature, wherein the first feature is the feature of the image to be detected extracted by the backbone network included in the target detection model, and the second feature is the feature of the first support image set extracted by the backbone network. The first determining module is used to determine a first detection box of the target object included in the image to be detected based on the matching operation; The second determining module is used to filter the first detection box using a target re-identification network to determine the target detection box of the target object.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to perform the method described in any one of claims 1 to 13 when executed.

16. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method as described in any one of claims 1 to 13.

Images