Label distribution method, label distribution device, electronic device and storage medium

Abstract

The present application discloses a label assignment method, a label assignment device, an electronic device and a storage medium. The label assignment method includes: obtaining multiple target prediction frame information of a sample image, each target prediction frame information includes a target prediction frame and a target confidence; determining a similarity between each target prediction frame and a real frame of the sample image to obtain multiple similarities, each similarity is obtained based on a target intersection-and-union ratio corresponding to a target prediction frame and a target confidence, and a target intersection-and-union ratio is obtained based on a target prediction frame and a real frame; assigning labels to multiple target prediction frames according to the multiple similarities to obtain positive samples and negative samples. The method implements label assignment to the prediction frame based on the similarity calculated based on the intersection-and-union ratio and confidence of the prediction frame relative to the real frame, and the number of positive and negative samples obtained by label assignment is easy to achieve a balance, which is conducive to improving the detection accuracy of the target detection model.

Description

Technical Field

[0001] The present application belongs to the field of computer vision technology, and in particular, relates to a label assignment method, a label assignment device, an electronic device, and a storage medium. Background Art

[0002] Label assignment is the process of matching the predicted box output in the forward propagation of the target detection model with the real box annotated by the sample. Label assignment divides the predicted box into positive samples and negative samples. The balance between positive and negative samples determines the detection accuracy of the target detection model.

[0003] At present, labels are mainly assigned to prediction boxes based on the intersection over union (IOU) between the prediction box and the true box. This can easily misclassify small prediction boxes with higher confidence as negative samples, resulting in a serious imbalance in the number of positive and negative samples, leading to reduced detection accuracy of the target detection model. Summary of the invention

[0004] In view of this, embodiments of the present application provide a label allocation method, a label allocation device, an electronic device, and a storage medium to overcome the above problems of the prior art.

[0005] In a first aspect, an embodiment of the present application provides a label assignment method, comprising: obtaining multiple target prediction box information of a sample image, each target prediction box information including a target prediction box and a target confidence; determining a similarity between each target prediction box and a real box of the sample image to obtain multiple similarities, each similarity being obtained based on a target intersection-and-union ratio corresponding to the target prediction box and the target confidence, and the target intersection-and-union ratio being obtained based on the target prediction box and the real box; performing label assignment on the multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples.

[0006] Among them, in some optional embodiments, determining a similarity between each target prediction box and the real box of the sample image includes: determining a target intersection-and-union ratio between each target prediction box and the real box; and determining the similarity based on the target intersection-and-union ratio corresponding to each target prediction box and the target confidence.

[0007] Among them, in some optional embodiments, determining the one similarity according to the one target intersection-and-union ratio and the one target confidence corresponding to each target prediction box includes: mapping each target prediction box to a preset two-dimensional space coordinate system according to the one target intersection-and-union ratio and the one target confidence corresponding to each target prediction box to obtain a first coordinate, the horizontal coordinate of the preset two-dimensional space coordinate system is used to represent the intersection-and-union ratio, and the vertical coordinate of the preset two-dimensional space coordinate system is used to represent the confidence; mapping the real box to the preset two-dimensional space coordinate system to obtain a second coordinate; and determining the one similarity according to each first coordinate and the second coordinate.

[0008] Among them, in some optional embodiments, determining the one similarity according to each first coordinate and the second coordinate includes: calculating a Euclidean distance between each first coordinate and the second coordinate; and determining the one similarity according to the one Euclidean distance.

[0009] Among them, in some optional embodiments, obtaining multiple target prediction box information of the sample image includes: inputting the sample image into a target detection model, so that the target detection model outputs multiple initial prediction box information; filtering the multiple initial prediction box information to obtain the multiple target prediction box information.

[0010] Among them, in some optional embodiments, the screening of the multiple initial prediction frame information to obtain the multiple target prediction frame information includes: determining the initial frame center corresponding to each initial prediction frame information; selecting the initial prediction frame information whose initial frame center is included in the real frame area of ​​the real frame from the multiple initial prediction frame information as the multiple target prediction frame information.

[0011] Among them, in some optional embodiments, assigning labels to multiple target prediction frames according to the multiple similarities to obtain positive samples and negative samples includes: sorting the multiple target prediction frames in descending order according to the multiple similarities to obtain a prediction frame sequence; selecting a preset number of target prediction frames from the first position of the prediction frame sequence to determine them as the first target prediction frame, and determining the target prediction frames that are not selected in the prediction frame sequence as the second target prediction frame; assigning a positive sample label to the first target prediction frame to obtain the positive sample; and assigning a negative sample label to the second target prediction frame to obtain the negative sample.

[0012] In the second aspect, an embodiment of the present application provides a label assignment device, which includes an acquisition module, a determination module, and an assignment module. The acquisition module is used to acquire multiple target prediction box information of a sample image, each target prediction box information includes a target prediction box and a target confidence; the determination module is used to determine a similarity between each target prediction box and a real box of the sample image to obtain multiple similarities, each similarity is obtained based on a target intersection-and-union ratio corresponding to the target prediction box and the target confidence, and the target intersection-and-union ratio is obtained based on the target prediction box and the real box; the assignment module is used to assign labels to multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory; one or more processors coupled to the memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the label allocation method provided in the first aspect above.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the label allocation method provided in the first aspect above.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is executed on a computer device, the computer device executes the label allocation method provided in the first aspect above.

[0016] The solution provided by the present application obtains multiple target prediction box information of a sample image, each target prediction box information includes a target prediction box and a target confidence, and determines a similarity between each target prediction box and a real box of the sample image to obtain multiple similarities, each similarity is obtained based on a target intersection-and-union ratio corresponding to a target prediction box and a target confidence, and a target intersection-and-union ratio is obtained based on a target prediction box and a real box, and labels are assigned to multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples, thereby realizing the similarity calculated based on the intersection-and-union ratio and confidence of the prediction box relative to the real box, labeling the prediction box, and easily achieving a balance in the number of positive and negative samples, which is beneficial to improving the detection accuracy of the target detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a label allocation method provided in an embodiment of the present application is shown.

[0019] Figure 2 A schematic diagram of a scenario for screening initial prediction box information in the label assignment method provided in an embodiment of the present application is shown.

[0020] Figure 3 A schematic diagram of a scenario in which a prediction box is mapped to a coordinate point in the label assignment method provided in an embodiment of the present application is shown.

[0021] Figure 4 Another flowchart of the label allocation method provided in an embodiment of the present application is shown.

[0022] Figure 5 A structural block diagram of a label distribution device provided in an embodiment of the present application is shown.

[0023] Figure 6 A functional block diagram of an electronic device provided in an embodiment of the present application is shown.

[0024] Figure 7 A computer-readable storage medium provided in an embodiment of the present application and used to store or carry a program code for implementing the label allocation method provided in an embodiment of the present application is shown.

[0025] Figure 8 A computer program product provided in an embodiment of the present application and used to store or carry program codes for implementing the label allocation method provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0028] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0029] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0031] Label assignment is the process of matching the predicted box output in the forward propagation of the target detection model with the real box annotated by the sample. Label assignment divides the predicted box into positive samples and negative samples. The balance between positive and negative samples determines the detection accuracy of the target detection model.

[0032] At present, labels are mainly assigned to prediction boxes based on the intersection over union (IOU) between the prediction box and the true box. This can easily misclassify small prediction boxes with higher confidence as negative samples, resulting in a serious imbalance in the number of positive and negative samples, leading to reduced detection accuracy of the target detection model.

[0033] In response to the above problems, the label assignment method, label assignment device, electronic device and storage medium provided in the embodiments of the present application obtain multiple target prediction box information of the sample image, each target prediction box information includes a target prediction box and a target confidence, and determine a similarity between each target prediction box and the real box of the sample image to obtain multiple similarities, each similarity is obtained based on a target intersection-and-union ratio corresponding to a target prediction box and a target confidence, and a target intersection-and-union ratio is obtained based on a target prediction box and a real box, and labels are assigned to multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples, thereby realizing the similarity calculated based on the intersection-and-union ratio and confidence of the prediction box relative to the real box, labeling the prediction box, and easily achieving a balance in the number of positive and negative samples, which is beneficial to improving the detection accuracy of the target detection model.

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0035] The label allocation method provided in the embodiment of the present application can be applied to a processing device, and the processing device can include but is not limited to any one of a server or a terminal device.

[0036] The server may include but is not limited to an independent physical server, a server cluster or distributed system consisting of multiple physical servers, and a cloud server.

[0037] Terminal devices may include but are not limited to mobile terminal devices (for example, mobile phones, PDAs, tablet PCs, laptops, smart watches, smart bracelets, etc.) and fixed terminal devices (for example, desktop computers, smart panels, all-in-one computers, etc.).

[0038] See also Figure 1 , which shows a flowchart of a label allocation method provided by an embodiment of the present application. In a specific embodiment, the label allocation method can be applied to a processing device. The following will take the processing device as an example to Figure 1 The process shown is described in detail, and the label allocation method may include the following steps 110 to 130.

[0039] Step 110: Obtain information of multiple target prediction boxes of the sample image.

[0040] In an embodiment of the present application, when the user needs to train the target detection model, a training instruction can be sent to the processing device, the processing device can receive and respond to the training instruction, input a sample image to the target detection model, the target detection model receives and responds to the sample image, and outputs multiple initial prediction box information to the processing device. The processing device receives multiple initial prediction box information output by the target detection model, and filters the multiple initial prediction box information to obtain multiple target prediction box information.

[0041] The target detection model may be any one of a YOLO model, an SSD model, a RetinaNet model, or a Faster R-CNN model. The type of the target detection model is not limited here and may be set according to actual needs.

[0042] Each initial prediction box information may include an initial prediction box and an initial confidence, and each target prediction box information may include a target prediction box and a target confidence.

[0043] The sum of multiple initial confidences corresponding to the multiple initial prediction box information is 1. For example, the multiple initial prediction box information may be a COCO dataset. The COCO dataset includes 80 initial prediction box information. The sum of 80 initial confidences corresponding to the 80 initial prediction box information is 1.

[0044] The processing device can determine the initial frame center corresponding to each initial prediction frame information, and select the initial prediction frame information whose initial frame center is included in the real frame area of ​​the real frame from multiple initial prediction frame information as multiple target prediction frame information. The initial prediction frame corresponding to the initial prediction frame information whose initial frame center is not in the real frame area has a low similarity with the real frame. The initial prediction frames with low similarity are excluded, which can reduce the amount of calculation in the process of determining the similarity between the prediction frame and the real frame, which is beneficial to improving the training efficiency of the target detection model.

[0045] As an example, Figure 2 As shown, multiple initial prediction box information may include initial prediction box information 1, initial prediction box information 2 and initial prediction box information 3, initial prediction box information 1 may include initial prediction box 1 (initial box 1), initial prediction box information 2 may include initial prediction box 2 (initial box 2), initial prediction box information 3 may include initial prediction box 3 (initial box 3), initial box center 1 (center 1) of initial box 1, initial box center 2 (center 2) of initial box 2 and initial box center 3 (center 3) of initial box 3 are all included in the real box area of ​​the real box (box 4), and initial prediction box information 1, initial prediction box information 2 and initial prediction box information 3 can be determined as target prediction box information.

[0046] In some embodiments, the processing device may detect the user's operation, and when it is determined based on the detected user operation that the user has input a training instruction for instructing to train the target detection model, the training instruction for training the target detection model is received. For example, when the user needs to train the target detection model, the user may perform a touch operation on the operation panel of the processing device, and the processing device responds to the user's touch operation, generates a corresponding touch signal, and analyzes the touch signal. When it is determined that the touch signal is a preset training signal for representing the training of the target detection model, it is determined that the training instruction for training the target detection model is received.

[0047] In some embodiments, the processing device may be provided with a speech recognition module. When a user needs to train a target detection model, the user may send a voice message within the speech collection range of the speech recognition module. The speech recognition module collects the voice message sent by the user, and performs speech recognition on the collected voice message. Based on the recognition result of the speech recognition, it is determined that the recognition result contains keywords for indicating the training of the target detection model, such as "target detection model training", and for example, "target detection model" and "training", etc., and it is determined that a training instruction for training the target detection model has been received.

[0048] As an example, the voice information sent by the user is: training the target detection model, and the recognition result of the voice recognition contains the keywords "target detection model" and "training", then it is determined that the training instruction for training the target detection model has been received.

[0049] In some embodiments, when a user needs to train a target detection model, a training instruction can be sent to a client. The client receives and responds to the training instruction, forwards the training instruction to a processing device through a network, and the processing device receives the training instruction forwarded by the client.

[0050] The client can be connected to the processing device through a network and perform data exchange with the processing device through the network.

[0051] The client can be any one of a mobile client (for example, a mobile phone client, a PDA client, a Tablet PC client, a laptop client, a smart watch client, a smart bracelet client or a wearable client, etc.) or a fixed client (for example, a desktop computer client, a smart panel client, etc.). The type of client is not limited here and can be set according to actual needs.

[0052] The network may be any one of a ZigBee network, a Bluetooth (BT) network, a Wireless Fidelity (Wi-Fi) network, a Thread network, a Long Range Radio (LoRa) network, a Low-Power Wide-Area Network (LPWAN), an infrared network, a Narrow Band Internet of Things (NB-IoT), a Controller Area Network (CAN), a Digital Living Network Alliance (DLNA) network, a Wide Area Network (WAN), a Local Area Network (LAN), a Metropolitan Area Network (MAN), or a Wireless Personal Area Network (WPAN), etc., without limitation here.

[0053] Step 120: Determine a similarity between each target prediction box and the real box of the sample image to obtain multiple similarities.

[0054] In an embodiment of the present application, after the processing device obtains information of multiple target prediction boxes of the sample image, it can determine a similarity between each target prediction box and a real box of the sample image to obtain multiple similarities.

[0055] Among them, each similarity can be obtained based on a target intersection over union (IOU) corresponding to a target prediction box and a target confidence, and a target IOU is obtained based on a target prediction box and a real box.

[0056] Regarding the process in which the processing device determines a similarity between each target prediction box and the real box of the sample image to obtain multiple similarities, in some embodiments, the processing device may determine a target IOU between each target prediction box and the real box, and determine a similarity based on a target IOU and a target confidence corresponding to each target prediction box to obtain multiple similarities.

[0057] Among them, each target IOU can be used to represent the ratio of the intersection area of ​​a target prediction box and the real box to the combined area of ​​the target prediction box and the real box, and each target IOU can be used to measure the degree of overlap between a target prediction box and the real box.

[0058] Regarding the process in which the processing device determines a similarity based on a target IOU and a target confidence corresponding to each target prediction box, in some embodiments, the processing device can map each target prediction box to a preset two-dimensional space coordinate system based on a target IOU and a target confidence corresponding to each target prediction box to obtain a first coordinate, and map the real box to the preset two-dimensional space coordinate system to obtain a second coordinate, and determine a similarity based on each first coordinate and the second coordinate, thereby realizing the conversion of two unrelated physical quantities IOU and confidence to coordinate points in the two-dimensional space coordinate system, and calculating the similarity at the coordinate points in the two-dimensional space coordinate system corresponding to the IOU and the confidence, which is beneficial to improving the calculation efficiency of the similarity.

[0059] The preset two-dimensional space coordinate system may be a Cartesian coordinate system, the abscissa of the preset two-dimensional space coordinate system may be used to represent the IOU, and the ordinate of the preset two-dimensional space coordinate system may be used to represent the confidence. Since the IOU of the true frame may be regarded as 1, the confidence of the true frame may be regarded as 1, and the second coordinate may be (1, 1).

[0060] Regarding the process in which the processing device determines a similarity based on each first coordinate and the second coordinate, in some embodiments, the processing device can calculate a Euclidean distance between each first coordinate and the second coordinate, and determine a similarity based on a Euclidean distance, thereby realizing the calculation of similarity based on Euclidean distance and simplifying the similarity calculation process.

[0061] In an application scenario, multiple target prediction boxes may include target prediction box 1 (target box 1), target prediction box 2 (target box 2) and target prediction box 3 (target box 3), target box 1 corresponds to target IOU1 (IOU1) and target confidence 1 (Conf1), target box 2 corresponds to target IOU2 (IOU2) and target confidence 2 (Conf2), and target box 3 corresponds to target IOU3 (IOU3) and target confidence 3 (Conf3).

[0062] like Figure 3 As shown, the processing device can respectively map the combination of IOU1 and Conf1 into coordinates A (IOU1, Conf1) in the Cartesian coordinate system, map the combination of IOU2 and Conf2 into coordinates B (IOU2, Conf2) in the Cartesian coordinate system, and map the combination of IOU3 and Conf3 into coordinates C (IOU3, Conf3) in the Cartesian coordinate system. Coordinates A (IOU1, Conf1), coordinates B (IOU2, Conf2) and coordinates C (IOU3, Conf3) are the first coordinates.

[0063] Since the IOU of the real box can be regarded as 1, the confidence of the real box can be regarded as 1, and the processing device can map the real box to the coordinate (1, 1) in the Cartesian coordinate system, and the coordinate (1, 1) is the second coordinate.

[0064] The processing device may calculate the first Euclidean distance d1 according to the coordinates A(IOU1, Conf1) and the coordinates (1, 1) according to Formula 1.

[0065] Formula 1 is:

[0066] The processing device may calculate the first similarity sim1 according to the first Euclidean distance d1 according to Formula 2.

[0067] Formula 2 is:

[0068] The processing device may calculate the second Euclidean distance d2 according to the coordinates B(IOU2, Conf2) and the coordinates (1, 1) according to Formula 3.

[0069] Formula 3 is:

[0070] The processing device may calculate the second similarity sim2 according to the second Euclidean distance d2 according to Formula 4.

[0071] Formula 4 is:

[0072] The processing device can calculate the third Euclidean distance d3 according to the coordinates C(IOU3, Conf3) and the coordinates (1, 1) according to Formula 5.

[0073] Formula 5 is:

[0074] The processing device may calculate the third similarity sim3 according to the third Euclidean distance d3 and formula (6).

[0075] Formula 6 is:

[0076] Step 130: Assign labels to multiple target prediction boxes according to multiple similarities to obtain positive samples and negative samples.

[0077] In an embodiment of the present application, the processing device determines a similarity between each target prediction box and the real box of the sample image to obtain multiple similarities. Labels can be assigned to the multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples. The similarity calculated based on the intersection-over-union ratio and confidence of the prediction box relative to the real box is implemented, and labels are assigned to the prediction boxes. The number of positive and negative samples obtained can be easily balanced, which is beneficial to improving the detection accuracy of the target detection model.

[0078] Specifically, the processing device determines a similarity between each target prediction frame and the real frame of the sample image to obtain multiple similarities. After that, the multiple target prediction frames can be sorted in descending order according to the multiple similarities to obtain a prediction frame sequence, and a preset number of target prediction frames are selected from the first position of the prediction frame sequence to be determined as the first target prediction frame, and the target prediction frame that is not selected in the prediction frame sequence is determined as the second target prediction frame, and a positive sample label is assigned to the first target prediction frame to obtain a positive sample, and a negative sample label is assigned to the second target prediction frame to obtain a negative sample. By retaining a preset number of target prediction frames, the problem of insufficient positive samples or excessive negative samples caused by retaining the target prediction frame according to the IOU threshold is avoided, the number of positive samples is greatly increased, and the balance of the number of positive and negative samples is greatly promoted. The retained preset number of positive samples is also beneficial to the regression of prediction frames in complex situations, which is beneficial to improving the generalization of the target detection model.

[0079] Among them, the preset number can be a number pre-set by the user, or a number automatically generated by the processing device according to a training process of multiple training of the target detection model, etc. The setting method of the preset number is not limited here, and it can be set according to actual needs.

[0080] The solution provided by the present application obtains multiple target prediction box information of a sample image, each target prediction box information includes a target prediction box and a target confidence, and determines a similarity between each target prediction box and a real box of the sample image to obtain multiple similarities, each similarity is obtained based on a target intersection-and-union ratio corresponding to a target prediction box and a target confidence, and a target intersection-and-union ratio is obtained based on a target prediction box and a real box, and labels are assigned to multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples, thereby realizing the similarity calculated based on the intersection-and-union ratio and confidence of the prediction box relative to the real box, labeling the prediction box, and easily achieving a balance in the number of positive and negative samples, which is beneficial to improving the detection accuracy of the target detection model.

[0081] See also Figure 4 , which shows a flowchart of a label allocation method provided by another embodiment of the present application. In a specific embodiment, the label allocation method can be applied to a processing device. The following will take the processing device as an example to Figure 4 The process shown is described in detail, and the label allocation method may include the following steps 210 to 250.

[0082] Step 210: Input a sample image to the target detection model, so that the target detection model outputs a plurality of initial prediction box information.

[0083] Step 220: Filter multiple initial prediction box information to obtain multiple target prediction box information.

[0084] Step 230: Determine a similarity between each target prediction box and the real box of the sample image to obtain multiple similarities.

[0085] Step 240: assign labels to multiple target prediction boxes according to multiple similarities to obtain positive samples and negative samples.

[0086] In this embodiment, step 210, step 220, step 230 and step 240 may refer to the contents of the corresponding steps in the aforementioned embodiments, which will not be repeated here.

[0087] Step 250: Update the parameters of the target detection model based on the positive samples and the negative samples.

[0088] In this embodiment, the processing device assigns labels to multiple target prediction boxes according to multiple similarities. After obtaining positive samples and negative samples, the parameters of the target detection model can be updated based on the positive samples and negative samples, thereby realizing the updating of the target detection model based on balanced positive samples and negative samples, which is beneficial to improving the detection accuracy of the target detection model.

[0089] Specifically, the processing device assigns labels to multiple target prediction boxes according to multiple similarities. After obtaining positive samples and negative samples, the loss function of the target detection model can be calculated based on the positive samples and negative samples, and based on the loss function, the parameters of the target detection model are updated through the back propagation algorithm to obtain an updated target detection model.

[0090] The solution provided in this embodiment inputs a sample image into a target detection model so that the target detection model outputs multiple initial prediction box information, and screens the multiple initial prediction box information to obtain multiple target prediction box information, and determines a similarity between each target prediction box and a real box of the sample image to obtain multiple similarities, and assigns labels to the multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples, and updates the parameters of the target detection model based on the positive samples and negative samples, thereby realizing the updating of the target detection model based on balanced positive samples and negative samples, which is beneficial to improving the detection accuracy of the target detection model.

[0091] See also Figure 5 , which shows a label distribution device 300 provided by an embodiment of the present application. The label distribution device 300 can be applied to a processing device. The processing device will be taken as an example below. Figure 5 The label allocating device 300 shown is described in detail. The label allocating device 300 may include an acquiring module 310 , a determining module 320 and an allocating module 330 .

[0092] The acquisition module 310 can be used to obtain multiple target prediction box information of the sample image, each target prediction box information can include a target prediction box and a target confidence; the determination module 320 can be used to determine a similarity between each target prediction box and the real box of the sample image to obtain multiple similarities, each similarity can be obtained based on a target intersection-and-union ratio corresponding to a target prediction box and a target confidence, and a target intersection-and-union ratio can be obtained based on a target prediction box and a real box; the allocation module 330 can be used to assign labels to multiple target prediction boxes according to multiple similarities to obtain positive samples and negative samples.

[0093] In some implementations, the determination module 320 may be a first determination unit and a second determination unit.

[0094] The first determination unit can be used to determine a target intersection-over-union ratio between each target prediction box and the real box; the second determination unit can be used to determine a similarity according to a target intersection-over-union ratio and a target confidence corresponding to each target prediction box.

[0095] In some embodiments, the second determining unit may include a first mapping subunit, a second mapping subunit, and a first determining subunit.

[0096] The first mapping subunit can be used to map each target prediction box to a preset two-dimensional space coordinate system according to a target intersection-and-union ratio and a target confidence corresponding to each target prediction box, to obtain a first coordinate, the horizontal coordinate of the preset two-dimensional space coordinate system can be used to characterize the intersection-and-union ratio, and the vertical coordinate of the preset two-dimensional space coordinate system can be used to characterize the confidence; the second mapping subunit can be used to map the real box to the preset two-dimensional space coordinate system to obtain a second coordinate; the first determination subunit can be used to determine a similarity based on each first coordinate and the second coordinate.

[0097] In some implementations, the first determining sub-unit may include a calculating sub-sub-unit and a determining sub-sub-unit.

[0098] The calculation sub-subunit may be used to calculate a Euclidean distance between each first coordinate and the second coordinate; the determination sub-subunit may be used to determine a similarity based on a Euclidean distance.

[0099] In some implementations, the acquisition module 310 may include an input unit and a screening unit.

[0100] The input unit can be used to input a sample image into the target detection model, so that the target detection model outputs multiple initial prediction box information; the screening unit can be used to screen the multiple initial prediction box information to obtain multiple target prediction box information.

[0101] In some embodiments, the screening unit may include a second determining subunit and a selecting subunit.

[0102] The second determination subunit can be used to determine the initial frame center corresponding to each initial prediction frame information; the selection subunit can be used to select the initial prediction frame information whose initial frame center is included in the real frame area of ​​the real frame from multiple initial prediction frame information as multiple target prediction frame information.

[0103] In some implementations, the allocation module 330 may include a sorting unit, a selecting unit, a first allocation unit, and a second allocation unit.

[0104] The sorting unit can be used to sort multiple target prediction frames in descending order according to multiple similarities to obtain a prediction frame sequence; the selection unit can be used to select a preset number of target prediction frames starting from the first position of the prediction frame sequence, determine them as the first target prediction frame, and determine the unselected target prediction frame in the prediction frame sequence as the second target prediction frame; the first allocation unit can be used to assign a positive sample label to the first target prediction frame to obtain a positive sample; the second allocation unit can be used to assign a negative sample label to the second target prediction frame to obtain a negative sample.

[0105] The solution provided by the present embodiment obtains multiple target prediction box information of the sample image, each target prediction box information includes a target prediction box and a target confidence, and determines a similarity between each target prediction box and the real box of the sample image to obtain multiple similarities, each similarity is obtained based on a target intersection-and-union ratio corresponding to a target prediction box and a target confidence, and a target intersection-and-union ratio is obtained based on a target prediction box and a real box, and labels are assigned to the multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples, thereby realizing the similarity calculated based on the intersection-and-union ratio and confidence of the prediction box relative to the real box, labeling the prediction box, and easily achieving a balance in the number of positive and negative samples, which is beneficial to improving the detection accuracy of the target detection model.

[0106] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Any processing method described in the method embodiment can be implemented by the corresponding processing module in the device embodiment, and will not be repeated one by one in the device embodiment.

[0107] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0108] See also Figure 6 , which shows a functional block diagram of an electronic device 500 provided by an embodiment of the present application, and the electronic device 500 may include one or more of the following components: a memory 510, a processor 520, and one or more applications, wherein the one or more applications may be stored in the memory 510 and configured to be executed by the one or more processors 520, and the one or more applications are configured to execute the method described in the aforementioned method embodiment.

[0109] The memory 510 may include a random access memory (RAM) or a read-only memory (ROM). The memory 510 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 510 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as obtaining multiple target prediction box information, determining similarity, label assignment, determining target intersection and union ratio, mapping, calculating Euclidean distance, inputting a sample image, outputting multiple initial prediction box information, screening multiple initial prediction box information, determining the initial box center, selecting initial prediction box information, sorting, and selecting a target prediction box, etc.), instructions for implementing the following various method embodiments, etc. The data storage area can also store data created by the electronic device 500 during use (such as sample images, target prediction box information, target prediction box, target confidence, real box, similarity, target intersection-over-union ratio, positive samples, negative samples, preset two-dimensional space coordinate system, first coordinate, second coordinate, Euclidean distance, target detection model, initial prediction box information, initial box center, real box area, prediction box sequence, first target prediction box and second target prediction box), etc.

[0110] The processor 520 may include one or more processing cores. The processor 520 uses various interfaces and lines to connect various parts of the entire electronic device 500, and executes various functions and processes data of the electronic device 500 by running or executing instructions, programs, code sets or instruction sets stored in the memory 510, and calling data stored in the memory 510. Optionally, the processor 520 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), programmable logic array (PLA), etc. The processor 520 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 520, but may be implemented separately through a communication chip.

[0111] Please refer to Figure 7 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 600 stores a program code 610, which can be called by a processor to execute the method described in the above method embodiment.

[0112] The computer readable storage medium 600 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 600 includes a non-transitory computer-readable storage medium. The computer readable storage medium 600 has storage space for program code 610 that performs any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 610 can be compressed, for example, in an appropriate form.

[0113] Please refer to Figure 8, which shows a structural block diagram of a computer program product 700 provided in an embodiment of the present application. The computer program product 700 includes a computer program / instruction 710, and the computer program / instruction 710 is stored in a computer-readable storage medium of a computer device. When the computer program product 700 is run on a computer device, a processor of the computer device reads the computer program / instruction 710 from the computer-readable storage medium, and the processor executes the computer program / instruction 710, so that the computer device performs the method described in the above method embodiment.

[0114] The solution provided by the present embodiment obtains multiple target prediction box information of the sample image, each target prediction box information includes a target prediction box and a target confidence, and determines a similarity between each target prediction box and the real box of the sample image to obtain multiple similarities, each similarity is obtained based on a target intersection-and-union ratio corresponding to a target prediction box and a target confidence, and a target intersection-and-union ratio is obtained based on a target prediction box and a real box, and labels are assigned to the multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples, thereby realizing the similarity calculated based on the intersection-and-union ratio and confidence of the prediction box relative to the real box, labeling the prediction box, and easily achieving a balance in the number of positive and negative samples, which is beneficial to improving the detection accuracy of the target detection model.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A label allocation method, characterized in that: include: Obtaining multiple target prediction box information of the sample image, each target prediction box information includes a target prediction box and a target confidence; Determine a similarity between each target prediction frame and a real frame of the sample image to obtain multiple similarities, each similarity is obtained based on a target intersection-and-union ratio corresponding to the target prediction frame and the target confidence, and the target intersection-and-union ratio is obtained based on the target prediction frame and the real frame; Labels are assigned to the multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples.

2. The label allocation method according to claim 1, characterized in that: Determining a similarity between each target prediction frame and a real frame of the sample image includes: Determine the target intersection-over-union ratio between each target prediction box and the one target true box; The similarity is determined according to the target intersection-union ratio corresponding to each target prediction box and the target confidence.

3. The label allocation method according to claim 2, characterized in that: The determining the similarity according to the target intersection-and-union ratio corresponding to each target prediction frame and the target confidence comprises: According to the target intersection-and-union ratio and the target confidence corresponding to each target prediction box, each target prediction box is mapped to a preset two-dimensional space coordinate system to obtain a first coordinate, where the abscissa of the preset two-dimensional space coordinate system is used to represent the intersection-and-union ratio, and the ordinate of the preset two-dimensional space coordinate system is used to represent the confidence; Mapping the real frame to the preset two-dimensional space coordinate system to obtain a second coordinate; The one similarity is determined according to each first coordinate and the second coordinate.

4. The label allocation method according to claim 3, characterized in that: The determining the similarity according to each first coordinate and the second coordinate includes: Calculate a Euclidean distance between each of the first coordinates and the second coordinate; The one similarity is determined according to the one Euclidean distance.

5. The label allocation method according to claim 1, characterized in that: The obtaining of information of multiple target prediction boxes of the sample image includes: Inputting the sample image into a target detection model so that the target detection model outputs a plurality of initial prediction box information; The multiple initial prediction box information are screened to obtain the multiple target prediction box information.

6. The label allocation method according to claim 5, characterized in that: The filtering of the multiple initial prediction box information to obtain the multiple target prediction box information includes: Determine the initial box center corresponding to each initial prediction box information; The initial prediction frame information whose center is included in the real frame area of ​​the real frame is selected from the multiple initial prediction frame information as the multiple target prediction frame information.

7. The label allocation method according to any one of claims 1 to 6, characterized in that: The step of assigning labels to the multiple target prediction frames according to the multiple similarities to obtain positive samples and negative samples includes: Sorting the multiple target prediction frames in descending order according to the multiple similarities to obtain a prediction frame sequence; Selecting a preset number of target prediction frames from the first position of the prediction frame sequence as first target prediction frames, and determining unselected target prediction frames in the prediction frame sequence as second target prediction frames; Assigning a positive sample label to the first target prediction box to obtain the positive sample; A negative sample label is assigned to the second target prediction box to obtain the negative sample.

8. A label distribution device, characterized in that: include: An acquisition module is used to acquire multiple target prediction box information of a sample image, each target prediction box information includes a target prediction box and a target confidence; A determination module, used to determine a similarity between each target prediction frame and a real frame of the sample image to obtain multiple similarities, each similarity is obtained based on a target intersection-and-union ratio corresponding to the target prediction frame and the target confidence, and the target intersection-and-union ratio is obtained based on the target prediction frame and the real frame; The allocation module is used to assign labels to multiple target prediction boxes according to the multiple similarities to obtain positive samples and negative samples.

9. An electronic device, characterized in that: include: Memory; One or more processors coupled to the memory; One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by one or more processors, and the one or more applications are configured to execute the label allocation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the label assignment method according to any one of claims 1 to 7.

Images