Training method and device of target detection model, electronic equipment and storage medium

Abstract

The present disclosure provides a target detection model training method and device, electronic equipment and storage medium, relating to the technical field of artificial intelligence, specifically to the technical fields of deep learning, image processing and computer vision, and particularly to the field of small sample target detection in the field of target detection. The specific implementation is: inputting a sample image into a feature extraction network to obtain a feature map of the sample image; inputting the feature map into a region proposal network to obtain an intermediate candidate box set output by the region proposal network and a to-be-identified candidate box set; determining a contrast learning loss based on positive sample pairs and negative sample pairs constructed based on the intermediate candidate box set; determining a screening loss and a detection loss; adjusting learnable parameters of the target detection model based on the contrast learning loss, the screening loss and the detection loss; and ending the training of the target detection model when a training convergence condition is met. Using the training method provided by the present disclosure, the accuracy of small sample target detection can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically deep learning, image processing, and computer vision technology, and particularly to the field of few-sample object detection in the field of object detection. Background Technology

[0002] The development of neural network technology has greatly improved object detection technology, and it is widely used in various fields of computer vision applications. However, overfitting occurs when detecting small sample objects, causing them to be misidentified as base class samples. Therefore, improving the accuracy of small sample object detection is an important research direction in current object detection technology. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, and storage medium for training an object detection model.

[0004] According to one aspect of this disclosure, a training method for a target detection model applied to small sample sizes is provided. The target detection model includes a feature extraction network, a region proposal network, and a region of interest identification network. The method includes:

[0005] The sample image is input into the feature extraction network to obtain the feature map of the sample image;

[0006] The feature map is input into the region proposal network to obtain the set of intermediate candidate boxes and the set of candidate boxes to be identified, where the set of candidate boxes to be identified is selected by the region proposal network from the set of intermediate candidate boxes.

[0007] Based on positive and negative sample pairs constructed from the set of intermediate candidate boxes, the contrastive learning loss is determined; and...

[0008] The loss of the region proposal network for filtering the set of candidate bounding boxes to be identified is determined; and,

[0009] Based on the processing results of the region of interest identification network on the candidate bounding box set to be identified, the detection loss of the target object in the sample image is determined.

[0010] The learnable parameters of the object detection model are adjusted based on the contrastive learning loss, the selection loss, and the detection loss. The training of the object detection model is terminated when the training convergence condition is met.

[0011] According to another aspect of this disclosure, a training apparatus for a target detection model applied to a small sample size is provided. The target detection model includes a feature extraction network, a region proposal network, and a region of interest identification network. The apparatus includes:

[0012] The feature extraction module is used to input sample images into the feature extraction network to obtain feature maps of the sample images;

[0013] The filtering module is used to input the feature map into the region proposal network and obtain the set of intermediate candidate boxes and the set of candidate boxes to be identified output by the region proposal network. The set of candidate boxes to be identified is selected by the region proposal network from the set of intermediate candidate boxes.

[0014] The loss determination module is used to determine the contrastive learning loss based on positive and negative sample pairs constructed from the intermediate candidate box set; and,

[0015] The loss of the region proposal network for filtering the set of candidate bounding boxes to be identified is determined; and,

[0016] Based on the processing results of the region of interest identification network on the candidate bounding box set to be identified, the detection loss of the target object in the sample image is determined.

[0017] The adjustment module is used to adjust the learnable parameters of the object detection model based on the contrastive learning loss, the selection loss, and the detection loss. When the training convergence condition is met, the training of the object detection model is terminated.

[0018] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0019] At least one processor; and,

[0020] The memory is communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a training method for any of the object detection models in this disclosure.

[0022] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform a training method according to any object detection model of this disclosure.

[0023] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a training method according to any object detection model of this disclosure.

[0024] This disclosure improves the accuracy of target detection for small samples by adding a contrastive learning loss to positive and negative sample pairs and adjusting the target detection model accordingly.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0026] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0027] Figure 1 This is a schematic diagram of the framework of a target detection model provided according to an embodiment of the present disclosure;

[0028] Figure 2(a) is a flowchart illustrating a training method for an object detection model according to an embodiment of the present disclosure;

[0029] Figure 2(b) is a schematic diagram of generating multiple candidate boxes according to an embodiment of the present disclosure;

[0030] Figure 3 This is a schematic diagram of a process for constructing positive and negative sample pairs according to an embodiment of the present disclosure;

[0031] Figure 4 Here is a flowchart illustrating the process of calculating the contrastive learning loss based on the constructed positive and negative sample pairs provided in another embodiment of this disclosure:

[0032] Figure 5 This is a schematic diagram of the structure of a training device for a target detection model according to an embodiment of the present disclosure;

[0033] Figure 6 This is a block diagram of an electronic device used to implement the training method of the target detection model in the embodiments of this disclosure. Detailed Implementation

[0034] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0035] Target detection technology has seen rapid development in recent years and is a popular area of ​​computer vision and digital image processing, with wide applications in fields such as robot navigation, intelligent video surveillance, industrial inspection, and aerospace.

[0036] A trained object detection model can identify the types of objects in an image and pinpoint their locations within the image. Training this model often requires a large amount of labeled training data to achieve good performance.

[0037] The sample set of this type of object detection model includes two categories: one is the base class sample, which is a relatively common and numerous sample in the training set, and the other is the small sample, which is a less common or smaller sample in the training set.

[0038] Because small sample sizes are limited in the training dataset, general object detection models perform poorly on such samples, easily misclassifying them as base class samples. For example, if an object detection model is trained using 10,000 samples labeled "ballpoint pen" and 10 samples labeled "stylus pen," the model will likely misclassify the stylus pen as a ballpoint pen due to their similar appearance.

[0039] Therefore, in order to improve the recognition accuracy of small samples, embodiments of this disclosure provide a training method for a target detection model applied to small samples. This method improves existing target detection models to enhance their accuracy in recognizing small samples. Figure 1 As shown, the target detection model provided in this embodiment includes:

[0040] Feature extraction network, region proposal network (RPN), and region of interest (ROI) identification network. Among them:

[0041] Feature extraction networks are used to extract features from input images and obtain feature maps;

[0042] Region proposal networks are used to obtain candidate bounding boxes to be identified based on feature maps;

[0043] The Region of Interest (ROI) recognition network consists of two parts: the ROI pool and the ROI head. The ROI pool is used to pool the candidate bounding boxes to be identified to the same scale; the ROI head identifies the category of each target object in the input image and the location of each target object in the input image based on the pooled candidate bounding boxes and feature maps.

[0044] Figure 2(a) shows a flowchart illustrating the training method for a target detection model applied to a small sample size provided in this embodiment of the present disclosure, including:

[0045] S201. Input the sample image into the feature extraction network to obtain the feature map of the sample image.

[0046] The feature extraction network is used to obtain feature maps, therefore this disclosure does not specify which feature extraction network to use. For example, a basic convolutional neural network can be used; alternatively, a network modified from a convolutional neural network can be used. Modified networks include, for example, a pre-trained backbone network for extracting image features, and networks that incorporate attention mechanisms such as Transformers.

[0047] S202. Input the feature map into the region proposal network to obtain the set of intermediate candidate boxes and the set of candidate boxes to be identified output by the region proposal network. The set of candidate boxes to be identified is selected by the region proposal network from the set of intermediate candidate boxes.

[0048] S203. Based on the positive and negative sample pairs constructed from the set of intermediate candidate boxes, determine the contrastive learning loss.

[0049] S204. Determine the filtering loss of the region proposal network for the set of candidate bounding boxes to be identified;

[0050] In some embodiments, the foreground and background of the sample image are pre-divided, and the set of candidate boxes to be identified selected by the region proposal network consists of intermediate candidate boxes with a higher probability of being in the foreground. The selection loss can be determined based on the comparison between the selection result and the division result.

[0051] S205. Based on the processing results of the region of interest recognition network on the set of candidate bounding boxes to be identified, determine the detection loss of the target object in the sample image.

[0052] S206. Adjust the learnable parameters of the target detection model based on the contrastive learning loss, the selection loss, and the detection loss. If the training convergence condition is met, terminate the training of the target detection model.

[0053] Adding a contrastive learning loss to positive and negative sample pairs, and adjusting the object detection model using the contrastive learning loss, can improve the object detection model's recognition accuracy for different types of objects, as well as improve the accuracy of object detection for small sample sizes.

[0054] Regarding the intermediate candidate boxes in step S202, in this embodiment of the disclosure, each point in the feature map has n corresponding initial candidate boxes. The position, size, and shape of these initial candidate boxes are predetermined, and the size and aspect ratio of these initial candidate boxes are all different. For example, typically, each point corresponds to 9 initial candidate boxes, and an example of these 9 initial candidate boxes is shown in Figure 2(b): For each point in the feature map, a total of 9 different initial candidate boxes can be generated with size ratios of 1:2:4 and aspect ratios of 1:1, 1:2, and 2:1. All initial candidate boxes in the feature map constitute the initial candidate box set.

[0055] After obtaining the initial candidate box set, the region proposal network needs to fine-tune the initial candidate box set based on the feature map to obtain the intermediate candidate box set. Specifically, this can be implemented by fine-tuning the shape and position of the initial candidate boxes so that the obtained intermediate candidate boxes match the target object in the sample image as closely as possible.

[0056] In this embodiment, each point in the feature map provides multiple initial candidate boxes, which can include the location of the target object in the initial candidate box set as much as possible, thereby improving the accuracy of the target object detection results. By fine-tuning the multiple initial candidate boxes, the intermediate candidate box set can have more expressive features, making it easier to accurately filter out the candidate box set to be identified, and improving the accuracy of the detected target object location.

[0057] Regarding the candidate bounding boxes to be identified in step S202, in some embodiments, the set of candidate bounding boxes to be identified is selected from the intermediate candidate box set, which can be implemented as follows:

[0058] Step A1: Set the target object in the sample image as the foreground and the rest as the background.

[0059] Step A2: Based on the feature map, calculate the probability that each intermediate candidate box belongs to the foreground.

[0060] Step A3: Select intermediate candidate boxes with a higher probability of belonging to the foreground, and further filter them using non-maximum suppression to obtain candidate boxes to be identified.

[0061] Using non-maximum suppression can reduce the overlap rate of candidate bounding boxes to be identified. Specifically, it can be implemented as follows:

[0062] Step B1: Divide the feature map into multiple regions.

[0063] Step B2: Repeat the following steps until all intermediate candidate boxes are marked or deleted:

[0064] Step B2-1: Find the middle candidate box A in each region that has the highest probability of belonging to the foreground and is not labeled.

[0065] Step B2-2: Delete the intermediate candidate boxes whose overlap rate with the intermediate candidate box A is higher than the filtering overlap rate threshold, and mark the intermediate candidate box A.

[0066] The number of candidate boxes to be identified can be changed by adjusting the threshold for the overlap rate.

[0067] In practice, the overlap rate between intermediate candidate box A and another intermediate candidate box B can be calculated by dividing the intersection of intermediate candidate box A and intermediate candidate box B by the union of intermediate candidate box A and intermediate candidate box B.

[0068] Step B3: Treat all marked intermediate candidate boxes as candidate boxes to be identified.

[0069] Regarding the construction of positive and negative sample pairs based on intermediate candidate boxes mentioned in step S203, in some embodiments, to facilitate understanding and differentiation of different candidate boxes, each intermediate candidate box in the intermediate candidate box set can be used as a first candidate box, such as... Figure 3 As shown, positive and negative sample pairs can be constructed for each first candidate box in the following manner:

[0070] S301. Select second candidate boxes from the set of intermediate candidate boxes whose overlap rate with the first candidate box is higher than the first overlap rate threshold, to obtain a set of second candidate boxes. Then, select third candidate boxes from the set of intermediate candidate boxes whose overlap rate with the first candidate box is lower than the second overlap rate threshold, to obtain a set of third candidate boxes. The second overlap rate threshold is lower than the first overlap rate threshold.

[0071] The method for calculating the overlap rate can refer to the method introduced in step B2-2 above, and will not be repeated here.

[0072] The first overlap rate threshold can be selected, for example, as 0.9, and the second overlap rate threshold can be selected, for example, as 0.1 or even smaller. In specific implementation, the first overlap rate threshold and the second overlap rate threshold can be determined according to actual needs.

[0073] S302. Based on the feature similarity between each second candidate box in the second candidate box set and the first candidate box, determine the positive sample pairs of the first candidate box; and based on the feature similarity between each third candidate box in the third candidate box set and the first candidate box, determine the negative sample pairs of the first candidate box.

[0074] In this embodiment, positive samples of the first candidate boxes can be initially screened based on a first overlap rate threshold. Further, positive sample pairs are constructed using feature similarity to ensure that the constructed positive samples are suitable for training the object detection model, thereby guaranteeing the rationality of constructing the positive sample pairs. Similarly, negative samples of the first candidate boxes can be initially screened based on a second overlap rate threshold. Further, negative sample pairs are constructed using feature similarity to ensure that the constructed negative samples are suitable for training the object detection model, thereby guaranteeing the rationality of constructing the negative sample pairs. Furthermore, using appropriate positive and negative sample pairs to train the object detection model allows the model to learn more expressive features from smaller samples, thereby enabling accurate detection of smaller sample objects.

[0075] In some embodiments, the features of either the positive sample pair or the negative sample pair can be obtained based on the following method:

[0076] Step C1: Use sample features to construct a network to map the feature map into n sub-feature maps, and obtain the sub-feature map corresponding to each class of intermediate candidate boxes.

[0077] In this embodiment, the sample feature construction network can be implemented as a fully connected layer network or a fully convolutional layer network. Taking a fully connected layer network as an example, the method for obtaining sub-feature maps is explained below:

[0078] For an m-dimensional feature map of size a*b obtained from the feature extraction network, an m*(m*n) mapping can be performed to obtain n feature maps of size m*a*b, which are n sub-feature maps. The weights of the fully connected layer network can be learned and adjusted based on a defined loss so that the fully connected layer network can accurately map the features of different types of candidate boxes.

[0079] Step C2: For any sample in the positive sample pair and the negative sample pair, take the feature value of the sample at the position point of the corresponding sub-feature map as the feature of the sample.

[0080] Taking the first, second, and third candidate boxes as examples, these three candidate boxes are all obtained by fine-tuning the shape and position of the initial candidate boxes, and therefore their dimensions are the same as the dimensions of the feature map. When calculating features, the features of the position of the candidate box in the feature map can be obtained as the features of that candidate box.

[0081] In this embodiment of the disclosure, multiple sub-feature maps are generated, and the features of the samples are determined by using the features of the sub-feature maps. This allows for more accurate acquisition of sample features, thereby enabling the calculation of contrastive learning loss based on sample-pair features to improve the accuracy of the target detection model in detecting small samples.

[0082] To improve training effectiveness and facilitate faster convergence of the object detection network training, this embodiment determines positive sample pairs of the first candidate boxes based on the feature similarity between each second candidate box in the second candidate box set and the first candidate box. This can be implemented as follows:

[0083] Step D1: Select the second candidate box with the lowest feature similarity to the first candidate box as a positive sample.

[0084] Step D2: Construct positive sample pairs based on the first candidate box and positive samples.

[0085] Similarly, in order to improve training effectiveness and facilitate the convergence of the object detection network training as quickly as possible, in this embodiment of the disclosure, negative sample pairs of the first candidate boxes are determined based on the feature similarity between each third candidate box in the third candidate box set and the first candidate box. This can be implemented as follows:

[0086] Step E1: Select the third candidate box with the highest feature similarity to the first candidate box as a negative sample.

[0087] Step E2: Construct negative sample pairs based on the first candidate box and the negative sample.

[0088] By selecting second candidate boxes with dissimilar features from the set of second candidate boxes with high overlap with the first candidate boxes as positive samples, and selecting third candidate boxes with similar features from the set of third candidate boxes with low overlap with the first candidate boxes as negative samples, the system aims to maximize the feature distance of positive sample pairs and minimize the feature distance of negative sample pairs, while satisfying the requirements for either positive or negative samples. This facilitates subsequent training of the target network model using a loss function, aiming to minimize the feature distance of positive sample pairs and maximize the feature distance of negative sample pairs. Therefore, by using contrastive learning loss, the system can better identify similar objects and distinguish between different classes, thereby improving the accuracy of the object detection model in detecting small samples.

[0089] After obtaining the positive and negative sample pairs, the contrastive learning loss in step S203 can be calculated using feature distance. Based on the preceding description of the initial candidate boxes, each point in the feature map corresponds to n intermediate candidate boxes. In some embodiments, such as... Figure 4 As shown, the contrastive learning loss can be calculated based on the positive and negative sample pairs constructed from the intermediate candidate box set using the following method:

[0090] S401. Perform the following operations for each intermediate candidate box: determine the first feature distance between two samples in the positive sample pair of the intermediate candidate box; and determine the second feature distance between two samples in the negative sample pair of the intermediate candidate box.

[0091] The feature distance can be calculated using the Euclidean distance method. For example, when the feature map dimension is 512, the features of the first candidate box are (a1, a2, ..., a...). 512 The second candidate box is (b1, b2...b 512 The features of the third candidate box are (c1, c2...c... 512 At this point, the first feature distance, i.e., the distance between the first candidate box and the second candidate box, is expressed as Euclidean distance as shown in equation (1):

[0092]

[0093] The first feature distance, which is the distance between the first candidate box and the third candidate box, is expressed as Euclidean distance as shown in equation (2):

[0094]

[0095] The Manhattan distance method can also be used to characterize the feature distance between two samples. That is, the distance is calculated for each dimension of the two features, and then these distances are summed. Taking the above features as an example, the first feature distance between the first candidate box and the second candidate box is expressed by the Manhattan distance as shown in equation (3):

[0096] |a1-b1|+|a2-b2|+...+|a 512 -b 512 | (3).

[0097] Of course, in practice, the cosine distance between vectors can also be used to represent the feature distance between samples. It should be noted that the feature distance obtained in this embodiment is only used to calculate the contrastive loss, and this disclosure does not specifically limit the method used to obtain the feature distance.

[0098] S402. Determine the contrastive learning loss based on the first feature distance and the second feature distance of each intermediate candidate box.

[0099] In some embodiments, either supervised training or unsupervised training can be used to determine the contrastive learning loss.

[0100] Method 1: Unsupervised training

[0101] For the first feature distance L1 and the second feature distance L2, a contrastive learning loss between positive and negative sample pairs is obtained by subtracting the first feature distance L1 and the second feature distance L2.

[0102] L 单 =L1-L2 (4);

[0103] Calculate the contrastive learning loss for all positive and negative sample pairs, and average them to obtain the contrastive learning loss of the model.

[0104]

[0105] Where x is the total number of positive sample pairs and the total number of negative sample pairs.

[0106] Method 2: Supervised Training

[0107] Supervised training introduces a new preset value: the margin. A larger margin increases the model's object detection accuracy, but also exponentially increases the training difficulty. Therefore, the margin can be initially set to a small value and then gradually increased to reduce the training difficulty.

[0108] The contrastive learning loss L between each positive and negative sample pair is calculated. 单 In addition to the difference calculation in unsupervised training, the marginal value will also be added, that is, the contrastive learning loss between a positive sample pair and a negative sample pair, as shown in expression (6):

[0109] L 单 =L1-L2+margin (6).

[0110] The subsequent calculation of the contrastive loss of the model is the same as that of the unsupervised method, and will not be elaborated here.

[0111] Unsupervised training can reduce the workload caused by setting preset edge values, while supervised training can more easily achieve the preset training goal and stop training; the specific method to use needs to be determined based on the specific situation.

[0112] Using the feature distance between positive and negative sample pairs to determine the contrast loss, we can obtain the feature differences between the first bounding box and the positive sample, and the feature differences between the first bounding box and the negative sample. This makes it easier to adjust the parameters of the target detection model so that the trained target detection model can better identify small samples.

[0113] Regarding the detection loss mentioned in step S205, in some embodiments, the candidate bounding box set to be identified can be processed first to obtain the position of the target object in the sample image and the classification result, and then the detection loss can be determined based on the processing result. In practice, the interest recognition network can process the candidate bounding box set to be identified using the following method:

[0114] Step F1: Pool the set of bounding boxes to be identified to make them the same size, and then fine-tune them again to make the target objects in the boundary feature maps of the bounding boxes the same.

[0115] Step F2: Use non-maximum suppression to filter the fine-tuned set of sample boxes to be identified, and obtain the target object sample boxes.

[0116] The specific implementation steps for nonmaximum suppression can be found in steps B1-B3, and will not be elaborated here.

[0117] Step F3: Perform target detection on the target object sample box to obtain the processing result, which includes the target object sample box, i.e. the classification result of the target object, and the position of each target object in the sample image.

[0118] After obtaining the processing result, it is compared with the position and label of the target object in the sample image to obtain the detection loss. The detection loss can be divided into two parts: one part is the offset loss caused by the difference between the position of the target object's sample box and the labeled position of the target object, i.e., the regression loss; the other part is the loss caused by the difference between the classification result and the labeled category of the target object, i.e., the classification loss.

[0119] Regarding the adjustment of learnable parameters in step S206, in some embodiments, the parameter adjustment of the target detection model provided in this disclosure can be divided into two stages:

[0120] Phase 1:

[0121] The object detection model is trained using base class samples. The parameter tuning in the first stage enables the object detection model to locate the target object in the sample image and classify the target object into its basic category.

[0122] In the first stage, the base class sample is input, and after obtaining the contrastive learning loss, the selection loss, and the detection loss, the model parameters of the feature extraction network, the region proposal network, and the region of interest identification network in the target detection model are adjusted based on the above losses.

[0123] Among them, the screening loss and detection loss are used to adjust the three networks mentioned above, while the contrastive learning loss is used to adjust the feature extraction network and the region proposal network.

[0124] The training convergence condition can be a specified number of iterations, or the total loss determined by the contrastive learning loss, the selection loss, and the detection loss is less than a specified loss value.

[0125] The purpose of using contrastive learning loss to adjust the feature extraction network and the region proposal network is to improve the object detection model's ability to distinguish between different types of objects, enabling it to detect target objects in sample images and identify small sample objects.

[0126] Phase Two:

[0127] The target detection model is trained using a small sample size. The parameter tuning in the second stage enables the target detection model to classify target objects in sample images more accurately, and also enables the classification of targets in small sample sizes.

[0128] In the second stage, the set of sample images input to the object detection model is the mini-sample set. The mini-sample set can be constructed using various mini-sample methods, or it can be constructed by randomly selecting a small number of samples from a label category with a large number of samples.

[0129] In the second stage of parameter tuning, the learnable parameters of the feature extraction network, region proposal network, and region of interest pool in the region of interest network are fixed and adjusted. The classification and regression layers of the region of interest head are adjusted using the contrastive learning loss, screening loss, and detection loss in the second stage so that they can detect and classify small sample targets.

[0130] Based on the same technical concept, this disclosure also provides a training device for a target detection model applied to a small sample size. The target detection model includes a feature extraction network, a region proposal network, and a region of interest recognition network, such as... Figure 5 As shown, the device includes:

[0131] Feature extraction module 501 is used to input sample images into the feature extraction network to obtain feature maps of the sample images;

[0132] The filtering module 502 is used to input the feature map into the region proposal network to obtain the set of intermediate candidate boxes and the set of candidate boxes to be identified output by the region proposal network, wherein the set of candidate boxes to be identified is filtered by the region proposal network from the set of intermediate candidate boxes;

[0133] The loss determination module 503 is used to determine the contrastive learning loss based on the positive and negative sample pairs constructed from the intermediate candidate box set; and to determine the filtering loss of the region proposal network for the candidate box set to be identified; and to determine the detection loss of the target object in the sample image based on the processing result of the region of interest identification network for the candidate box set to be identified.

[0134] The adjustment module 504 is used to adjust the learnable parameters of the object detection model based on the contrastive learning loss, the selection loss, and the detection loss, and to terminate the training of the object detection model when the training convergence condition is met.

[0135] In some embodiments, each intermediate candidate box in the set of intermediate candidate boxes is used as a first candidate box, and the device further includes:

[0136] The building module is used to construct positive and negative sample pairs for each first candidate box based on the following methods:

[0137] From the set of intermediate candidate boxes, select the second candidate box whose overlap rate with the first candidate box is higher than the first overlap rate threshold to obtain the second candidate box set. Then, from the set of intermediate candidate boxes, select the third candidate box whose overlap rate with the first candidate box is lower than the second overlap rate threshold to obtain the third candidate box set. The second overlap rate threshold is less than the first overlap rate threshold.

[0138] Based on the feature similarity between each second candidate box in the second candidate box set and the first candidate box, positive sample pairs of the first candidate box are determined; and based on the feature similarity between each third candidate box in the third candidate box set and the first candidate box, negative sample pairs of the first candidate box are determined.

[0139] In some embodiments, the construction module is configured to:

[0140] The second candidate box with the lowest feature similarity to the first candidate box is selected as the positive sample;

[0141] The first candidate box and the positive sample are identified as a positive sample pair.

[0142] In some embodiments, the building module is further configured to:

[0143] The third candidate box with the lowest feature similarity to the first candidate box is selected as the negative sample;

[0144] The first candidate box and the negative sample are identified as a negative sample pair.

[0145] In some embodiments, each point in the feature map corresponds to n intermediate candidate boxes, where n is a positive integer greater than 1. The device further includes:

[0146] The mapping module is used to construct a network using sample features to map the feature map into n sub-feature maps, thereby obtaining the sub-feature map corresponding to each class of intermediate candidate boxes.

[0147] The feature value module is used to take the feature value of the sample at the position point of the corresponding sub-feature map as the feature of the sample for any sample in the positive sample pair and negative sample pair.

[0148] In some embodiments, the loss determination module is configured to:

[0149] For each intermediate candidate box, perform the following operations: determine the first feature distance between two samples in the positive sample pair of the intermediate candidate box; and determine the second feature distance between two samples in the negative sample pair of the intermediate candidate box.

[0150] The contrastive learning loss is determined based on the first feature distance and the second feature distance of each intermediate candidate box.

[0151] In some embodiments, each point in the feature map corresponds to n initial candidate boxes, where n is a positive integer greater than 1. The initial candidate boxes form an initial candidate box set. The filtering module is then used for:

[0152] The feature map is input into the region proposal network, which then fine-tunes the initial candidate box set based on the feature map to obtain the intermediate candidate box set.

[0153] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0154] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0155] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0156] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the training method for an object detection model. For example, in some embodiments, the training method for an object detection model may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the training method for an object detection model described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the training method for an object detection model by any other suitable means (e.g., by means of firmware).

[0157] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0159] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0161] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0162] A computer system may include clients and servers. Clients and servers are generally located far apart and typically interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server incorporating blockchain technology. Embodiments of this disclosure may employ a server to execute a protein generation method.

[0163] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0164] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A training method for an object detection model, said object detection model comprising a feature extraction network, a region proposal network, and a region of interest identification network, the method comprising: The sample image is input into the feature extraction network to obtain the feature map of the sample image; The feature map is input into the region proposal network to obtain the set of intermediate candidate boxes and the set of candidate boxes to be identified output by the region proposal network, wherein the set of candidate boxes to be identified is selected by the region proposal network from the set of intermediate candidate boxes; Based on the positive and negative sample pairs constructed from the set of intermediate candidate boxes, the contrastive learning loss is determined. and, Determine the filtering loss of the region proposal network for the set of candidate boxes to be identified; as well as, Based on the processing results of the region of interest identification network on the set of candidate boxes to be identified, the detection loss of the target object in the sample image is determined; The learnable parameters of the target detection model are adjusted based on the contrastive learning loss, the screening loss, and the detection loss. Training of the target detection model ends when the training convergence condition is met. In this context, each point in the feature map corresponds to n intermediate candidate boxes, where n is a positive integer greater than 1. Each intermediate candidate box in the set of intermediate candidate boxes is taken as a first candidate box. For each first candidate box, positive and negative sample pairs are constructed based on the following method: From the set of intermediate candidate boxes, a second candidate box with an overlap rate higher than a first overlap rate threshold is selected to obtain a second candidate box set. From the set of intermediate candidate boxes, a third candidate box with an overlap rate lower than a second overlap rate threshold is selected to obtain a third candidate box set. The second overlap rate threshold is less than the first overlap rate threshold. Based on the feature similarity between each second candidate box in the second candidate box set and the first candidate box, positive sample pairs of the first candidate boxes are determined; and, Based on the feature similarity between each third candidate box in the third candidate box set and the first candidate box, negative sample pairs of the first candidate box are determined. Also includes: The feature construction network is used to map the feature map into n sub-feature maps, obtaining the sub-feature map corresponding to each class of intermediate candidate boxes; the feature construction network is used to map a The m-dimensional feature map of b is used to perform m... (m) The mapping of n) yields n elements of size m. a Sub-feature map of b; the contrastive learning loss is used to optimize the network parameters of the feature-constructed network; For any sample in the positive sample pair and the negative sample pair, the feature value of the corresponding position point in the corresponding sub-feature map of the sample is taken as the feature of the sample.

2. The method according to claim 1, wherein, The step of determining positive sample pairs of the first candidate boxes based on the feature similarity between each second candidate box in the second candidate box set and the first candidate box includes: The second candidate box with the lowest feature similarity to the first candidate box is selected as the positive sample; The first candidate box and the positive sample are identified as a positive sample pair.

3. The method according to claim 1, wherein, The step of determining negative sample pairs of the first candidate boxes based on the feature similarity between each third candidate box in the third candidate box set and the first candidate box includes: The third candidate box with the highest feature similarity to the first candidate box is selected as the negative sample; The first candidate box and the negative sample are identified as a negative sample pair.

4. The method according to any one of claims 1-3, wherein, The determination of the contrastive learning loss based on the positive and negative sample pairs constructed from the intermediate candidate box set includes: For each intermediate candidate box, perform the following operations: determine the first feature distance between two samples in the positive sample pair of the intermediate candidate box; and determine the second feature distance between two samples in the negative sample pair of the intermediate candidate box. The contrastive learning loss is determined based on the first feature distance and the second feature distance of each intermediate candidate box.

5. The method according to any one of claims 1-4, wherein, Each point in the feature map corresponds to n initial candidate boxes, where n is a positive integer greater than 1. Each initial candidate box is used to construct an initial candidate box set. The feature map is then input into the region proposal network to obtain an intermediate candidate box set output by the region proposal network, including: The feature map is input into the region proposal network, which then fine-tunes the initial candidate box set based on the feature map to obtain the intermediate candidate box set.

6. A training apparatus for an object detection model, the object detection model comprising a feature extraction network, a region proposal network, and a region of interest identification network, the apparatus comprising: The feature extraction module is used to input the sample image into the feature extraction network to obtain the feature map of the sample image; The filtering module is used to input the feature map into the region proposal network to obtain the set of intermediate candidate boxes and the set of candidate boxes to be identified output by the region proposal network, wherein the set of candidate boxes to be identified is filtered by the region proposal network from the set of intermediate candidate boxes; The loss determination module is used to determine the contrastive learning loss based on the positive and negative sample pairs constructed from the intermediate candidate box set. Furthermore, the filtering loss of the region proposal network for the set of candidate boxes to be identified is determined; Furthermore, based on the processing results of the region of interest identification network on the set of candidate boxes to be identified, the detection loss of the target object in the sample image is determined; The adjustment module is used to adjust the learnable parameters of the target detection model based on the contrastive learning loss, the screening loss, and the detection loss, and to end the training of the target detection model when the training convergence condition is met. In this device, each point in the feature map corresponds to n intermediate candidate boxes, where n is a positive integer greater than 1. Each intermediate candidate box in the set of intermediate candidate boxes is used as a first candidate box. The device further includes: The construction module is used to construct positive and negative sample pairs for each first candidate box based on the following methods: In this context, each point in the feature map corresponds to n intermediate candidate boxes, where n is a positive integer greater than 1. Each intermediate candidate box in the set of intermediate candidate boxes is taken as a first candidate box. For each first candidate box, positive and negative sample pairs are constructed based on the following method: From the set of intermediate candidate boxes, a second candidate box with an overlap rate higher than a first overlap rate threshold is selected to obtain a second candidate box set. From the set of intermediate candidate boxes, a third candidate box with an overlap rate lower than a second overlap rate threshold is selected to obtain a third candidate box set. The second overlap rate threshold is less than the first overlap rate threshold. Based on the feature similarity between each second candidate box in the second candidate box set and the first candidate box, positive sample pairs of the first candidate boxes are determined; and, Based on the feature similarity between each third candidate box in the third candidate box set and the first candidate box, negative sample pairs of the first candidate box are determined. Also includes: The mapping module is used to construct a network using sample features to map the feature map into n sub-feature maps, obtaining the sub-feature maps corresponding to each class of intermediate candidate boxes; the feature construction network is used to map a The m-dimensional feature map of b is used to perform m... (m) The mapping of n) yields n elements of size m. a Sub-feature map of b; the contrastive learning loss is used to optimize the network parameters of the feature-constructed network; The sample feature determination module is used to determine the feature value of the corresponding position point in the corresponding sub-feature map of any sample in the positive sample pair and the negative sample pair as the feature of the sample.

7. The apparatus according to claim 6, wherein, The building module is used for: The second candidate box with the lowest feature similarity to the first candidate box is selected as the positive sample; The first candidate box and the positive sample are identified as a positive sample pair.

8. The apparatus according to claim 6, wherein, The building module is also used for: The third candidate box with the highest feature similarity to the first candidate box is selected as the negative sample; The first candidate box and the negative sample are identified as a negative sample pair.

9. The apparatus according to any one of claims 6-8, wherein, The loss determination module is used for: For each intermediate candidate box, perform the following operations: determine the first feature distance between two samples in the positive sample pair of the intermediate candidate box; and determine the second feature distance between two samples in the negative sample pair of the intermediate candidate box. The contrastive learning loss is determined based on the first feature distance and the second feature distance of each intermediate candidate box.

10. The apparatus according to any one of claims 6-9, wherein, Each point in the feature map corresponds to n initial candidate boxes, where n is a positive integer greater than 1. These initial candidate boxes are used to construct an initial candidate box set. The filtering module is then used to: The feature map is input into the region proposal network, which then fine-tunes the initial candidate box set based on the feature map to obtain the intermediate candidate box set.

11. An electronic device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images