Data labeling method and electronic device

Abstract

The present disclosure relates to the technical field of image processing, and in particular relates to a data labeling method and an electronic device, which are used for solving the problem of the labeling speed of a conventional labeling operation being low. The method comprises: using a plurality of detection models associated with a labeling task to perform object detection on at least one of a plurality of images associated with the labeling task, so as to obtain a plurality of detection results corresponding to the at least one image, wherein each detection result corresponds to a detection model, and each detection result comprises at least one piece of detection information corresponding to each object detected in the corresponding image; on the basis of the plurality of detection results corresponding to the at least one image and a first confidence threshold, performing screening to obtain candidate detection information corresponding to each object in the at least one image; and on the basis of the candidate detection information corresponding to each object in the at least one image, obtaining a labeling result corresponding to the at least one image. The method improves the efficiency, speed and accuracy of labeling.

Description

Data labeling method and electronic device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to Chinese application No. 202410142447.6, filed on January 31, 2024; and No. 202411167399.2, filed on August 23, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the field of image processing technology, and in particular to a data annotation method and electronic device. Background Art

[0004] In the field of machine vision, massive amounts of collected images are used to train various models. Typically, before training these models, these images must be annotated. Currently, image annotation is often done manually, which is inefficient, slow, time-consuming, and costly.

[0005] Summary of the Invention

[0006] In a first aspect, some embodiments of the present disclosure provide a data labeling method, including:

[0007] Acquiring a plurality of images associated with the labeling task based on a labeling instruction for the labeling task;

[0008] Using the multiple detection models associated with the labeling task, perform object detection on at least one of the multiple images to obtain multiple detection results corresponding to the at least one image; wherein each detection result corresponds to a detection model; the detection result includes at least one detection information corresponding to each object detected in the corresponding image; and each detection information corresponds to a confidence level;

[0009] Filtering candidate detection information corresponding to each object in the at least one image based on the multiple detection results corresponding to the at least one image and a first confidence threshold;

[0010] Based on the candidate detection information corresponding to each object in the at least one image, a labeling result corresponding to the at least one image is obtained; wherein the labeling result includes the labeling information corresponding to each object in the at least one image.

[0011] In a second aspect, some embodiments of the present disclosure provide a data labeling method, including:

[0012] Acquire an image to be processed, and determine a target object in the image to be processed;

[0013] Determining the contour of the target object in the image to be processed based on the large model;

[0014] The outline of the target object is marked and displayed according to the sparsity of the stored marking points.

[0015] In a third aspect, some embodiments of the present disclosure provide an electronic device, including:

[0016] Memory, used to store computer programs or instructions;

[0017] A processor is configured to execute the computer program or instructions in the memory so that the method according to any one of the first aspect or the second aspect is performed.

[0018] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable storage medium, which, when instructions in the storage medium are executed by a processor, enables the processor to execute any one of the methods described in the first or second aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG1 is a schematic diagram of an application scenario applicable to a data annotation method provided by some embodiments of the present disclosure;

[0020] FIG2 is a schematic diagram of a human-computer interaction interface in a marking platform provided in some embodiments of the present disclosure;

[0021] FIG3 is another schematic diagram of a human-computer interaction interface in a marking platform provided by some embodiments of the present disclosure;

[0022] FIG4 is a flow chart of a data annotation method provided by some embodiments of the present disclosure;

[0023] FIG5 is a flowchart of determining candidate detection information provided by some embodiments of the present disclosure;

[0024] FIG6 is a flowchart of determining matching detection information provided by some embodiments of the present disclosure;

[0025] FIG7 is a flowchart of determining a reference confidence level according to some embodiments of the present disclosure;

[0026] FIG8 is a flowchart of determining the annotation information of each object in an image according to some embodiments of the present disclosure;

[0027] FIG9 is a flowchart illustrating an execution flow of a filter frame strategy according to some embodiments of the present disclosure;

[0028] FIG10 is a schematic diagram of an image with multiple marking boxes on the same object provided by some embodiments of the present disclosure;

[0029] FIG11 is a schematic diagram showing a comparison of data annotation results provided by some embodiments of the present disclosure;

[0030] FIG12 is another flow chart of a data annotation method provided by some embodiments of the present disclosure;

[0031] FIG13 is a schematic diagram of an image to be processed provided by some embodiments of the present disclosure;

[0032] FIG14 is a schematic diagram of marking a target object provided by some embodiments of the present disclosure;

[0033] FIG15 is a schematic diagram of annotating a target object outline according to some embodiments of the present disclosure;

[0034] FIG16 is a schematic diagram of a label modification provided by some embodiments of the present disclosure;

[0035] FIG17 is a schematic diagram of densely labeled points provided by some embodiments of the present disclosure;

[0036] FIG18 is a schematic diagram of sparse annotation points provided by some embodiments of the present disclosure;

[0037] FIG19 is a schematic diagram of a labeling result provided by some embodiments of the present disclosure;

[0038] FIG20 is a schematic diagram of an application scenario of target object outline annotation provided by some embodiments of the present disclosure;

[0039] FIG21 is a schematic diagram of the physical architecture of an electronic device provided in some embodiments of the present disclosure. DETAILED DESCRIPTION

[0040] The following will be combined with the accompanying drawings provided by some embodiments of the present disclosure to clearly and completely describe the technical solutions provided by some embodiments of the present disclosure. Obviously, the embodiments described are only some of the embodiments of the present disclosure and not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0041] In order to help those skilled in the art better understand the technical solutions of the present disclosure, some terms provided in some embodiments of the present disclosure are explained below.

[0042] (1) In some embodiments, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0043] (2) In some embodiments, the term "intersection over union (IOU)" is used to calculate the ratio of the intersection and union of two detection boxes.

[0044] (3) In some embodiments, the term "Complete Intersection-over-Union (CIOU)" takes into account the overlapping area, center point distance, and aspect ratio between two detection boxes, and considers the similarity between two detection boxes from the perspectives of overlapping area, center point distance, and aspect ratio. In some embodiments, CIOU can be expressed as follows:

[0045] Distance_2 represents the Euclidean distance between the center points of the two detection frames, and Distance_C represents the distance between the diagonals of the minimum circumscribed rectangular frames of the two detection frames.

[0046] It should be noted that the terms "first," "second," "third," etc., in the specification and claims of the present disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0047] The following is a brief introduction to the design concepts of some embodiments of the present disclosure.

[0048] In the field of machine vision, massive amounts of collected images are used to train various models. Typically, before model training begins, these massive amounts of collected images must be annotated to obtain labels for training the various models. Currently, conventional image annotation is often done manually, which suffers from low efficiency, slow speed, long labeling times, and high costs.

[0049] In view of this, in order to solve the problems of low labeling efficiency, slow labeling speed, long labeling time, high labeling cost, etc. existing in conventional labeling work methods, some embodiments of the present disclosure provide a data labeling method. In some embodiments, based on the labeling instructions for the labeling task, multiple images associated with the labeling task are obtained; multiple detection models associated with the labeling task are used to perform target detection on each of the multiple images to obtain multiple detection results corresponding to each image; wherein, each detection result corresponds to a detection model; the detection result includes at least one detection information corresponding to each object detected in the corresponding image; each detection information corresponds to a confidence level; for each image, the following operations are performed: for an image, based on the multiple detection results corresponding to the image and a first confidence threshold, the candidate detection information corresponding to each object in the image is screened out; based on the candidate detection information corresponding to each object in the image, the labeling result corresponding to the image is obtained; wherein the labeling result includes the detection box labeling information corresponding to each object in the image.

[0050] In some embodiments, first, target detection is performed on multiple images associated with the labeling task based on each detection model to obtain detection results. Then, through a multi-model fusion decision-making method, the detection information in the multiple detection results corresponding to each image is screened, and the screened candidate detection information is fused to obtain the labeling results corresponding to each image, so as to improve the precision and recall rate of the image labeling results, and the labeling personnel only need to perform secondary labeling based on this labeling result, which is faster than the labeling speed predicted by the traditional single model. It can be seen that the present disclosure obtains the preliminary labeling results corresponding to each image by adopting multiple models, which can improve the labeling efficiency and labeling speed while shortening the labeling time and reducing the labeling cost.

[0051] Some embodiments of the present disclosure provide a data annotation method applicable to electronic devices, and are used for deploying offline tasks with an annotation platform.

[0052] In some embodiments, the electronic device can be a smart terminal, a smart mobile terminal, a tablet computer, a laptop computer, a smart handheld device, a personal computer (PC), a computer, a smart screen, a personal digital assistant (PDA), etc.

[0053] In other implementations, the electronic device may also be a server connected to a device with a display function (such as a display device), for example, it may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms; wherein the server can be connected to the device with a display function by wire or wireless means, and the present disclosure does not limit the connection method.

[0054] It is understood that the present disclosure does not limit the specific types of the above-mentioned electronic devices.

[0055] The following, in conjunction with the accompanying drawings, illustrates a schematic diagram of an application scenario of the data annotation method provided by some embodiments of the present disclosure, taking an electronic device as an example server. As shown in Figure 1, the application scenario includes an annotation terminal 10 deployed with an annotation platform, and a server 20 associated with the annotation platform.

[0056] The annotation platform provides users with a human-computer interaction interface, as shown in Figure 2. The annotation manager's terminal can use this interface to construct annotation tasks, such as selecting multiple images and detection models associated with the task to be initiated. After constructing the annotation task, the annotation manager can click a preset area in the interface (such as the "OK" button in Figure 2) to issue a labeling instruction for the task.

[0057] After receiving the labeling instruction sent by the labeling platform, the labeling terminal of the labeling manager sends the labeling instruction to the server 20. The server 20 calls the deployed multi-model fusion decision algorithm and executes a data labeling method provided by some embodiments of the present disclosure to obtain the labeling results corresponding to each image associated with the labeling task; wherein the labeling results include the labeling information corresponding to each object in the corresponding image.

[0058] In some embodiments, the annotation result may be a pre-annotation result. After the server 20 obtains the annotation result corresponding to each image associated with the annotation task, the annotator who logs in to the annotation platform can obtain the annotation result of each image associated with the annotation task through the annotation platform based on the identification information of the annotation task through the human-computer interaction interface of the associated annotation terminal and display them separately. As shown in Figure 3, the annotation result of an image displayed in the human-computer interaction interface of any annotator's annotation terminal 10 can be re-annotated by the annotator through the human-computer interaction interface to obtain the final annotation result of the image, and then obtain the final annotation result corresponding to each image associated with the annotation task.

[0059] It should be noted that each detection model can be trained based on a small amount of manually labeled data, or it can be a pre-trained large model, and this disclosure does not limit it specifically.

[0060] Of course, the methods provided in some embodiments of the present disclosure are not limited to the application scenario shown in Figure 1, and can also be used in other possible application scenarios, which are not limited in some embodiments of the present disclosure. It should also be noted that the human-computer interaction interfaces shown in Figures 2 and 3 are only examples. In actual applications, the content displayed in the human-computer interaction interface can be customized based on actual needs, which is not specifically limited in this disclosure.

[0061] After introducing an optional application scenario of some embodiments of the present disclosure, a data labeling method provided by some embodiments of the present disclosure is introduced below in combination with the application scenario shown in FIG. 1 in some embodiments. Specifically, as shown in FIG. 4 , the method includes the following steps:

[0062] Step 400: Based on a labeling instruction for a labeling task, a plurality of images associated with the labeling task are acquired.

[0063] In some embodiments, after the annotation platform initiates an annotation task, the annotation terminal receives the annotation instruction for the task and sends the instruction to a server associated with the annotation platform. The server then executes step 400 and, based on the instruction, obtains multiple images associated with the annotation task. Simultaneously, it obtains multiple detection models associated with the annotation task and the detection categories of the objects to be detected in the images, such as cars, pedestrians, zebra crossings, buses, etc.

[0064] Step 410: Utilize the multiple detection models associated with the labeling task to perform target detection on each of the multiple images to obtain multiple detection results corresponding to each image; wherein each detection result corresponds to a detection model; the detection result includes at least one detection information corresponding to each object detected in the corresponding image; and each detection information corresponds to a confidence level.

[0065] When executing step 410, the following operations are performed for each image in the multiple images to obtain multiple detection results corresponding to each image: for one image (i.e., any one image in the multiple images), target detection is performed on the objects in the image using multiple detection models associated with the labeling task, and detection information of each object in the image detected by each detection model and the confidence corresponding to the detection information are obtained; wherein the detection information includes the coordinates of the detection box that frames the corresponding object in the image, and the confidence is the probability that the object framed by the corresponding detection box is the corresponding detection category.

[0066] In some embodiments, the annotation result corresponding to each image can be determined by setting multiple frame selection strategies; the multiple frame selection strategies disclosed herein may include some or all of the retention frame strategy, the edit frame strategy, the confirmation frame strategy, and the filter frame strategy;

[0067] Among them, the retain box strategy is used to select which detection information needs to be retained and which detection information needs to be deleted from the model detection results; the edit box strategy is used to fuse multiple detection information associated with an object, so that the marking box in each annotation information can more appropriately and accurately frame the corresponding object; the confirmation box strategy is used to filter out the marking boxes detected by each detection model and with high confidence, and highlight such annotation information; the filter box strategy is used to delete redundant annotation information and retain only the necessary annotation information.

[0068] In some embodiments, the detection results corresponding to multiple detection models can be used to select which detection information needs to be retained and which detection information needs to be deleted through the retention frame strategy; then, the detection information selected for retention can be edited for the second time through the editing frame strategy. The editing frame strategy is used to fuse multiple detection information associated with an object, so that the marking frame in each annotation information can frame the corresponding object more appropriately and accurately; then, through the confirmation frame strategy, the marking frames that are detected by each detection model and have a high confidence level are filtered out from the determined annotation information, and such annotation information is highlighted, that is, in the annotation interface of the annotation platform (that is, the aforementioned human-computer interaction interface), a specific The color is displayed. When the annotator performs the second annotation, this type of marking box does not need to be checked and modified. It is skipped by default if it is correct, thereby greatly improving the annotation speed and the efficiency of the annotation work. Finally, the filter box strategy is used to delete redundant annotation information and only retain the necessary annotation information. This strategy is mainly used for an object that is detected as different detection categories by different detection models, resulting in different annotation information for the same object in the multiple annotation information obtained. At this time, only one annotation information is retained. When the annotator performs the second annotation, he can only modify or delete the retained annotation information, reducing the redundant operations on multiple annotation information of the same object.

[0069] Through the above strategies, the fused annotation result is finally output, and the precision and recall of the result are better than the prediction result of a single model. The annotator only needs to perform secondary annotation based on this result, which improves the annotation speed. The annotation speed is better than the annotation speed based on the prediction of a single model and shortens the annotation time. At the same time, using multiple detection models to detect the target for each image can also improve the accuracy of each annotation information in the obtained annotation result, thereby improving the detection accuracy of the final model obtained by subsequent model training based on the obtained annotation results. After an overall overview of the processing logic provided by some embodiments of the present disclosure, the following steps 4201 to 4202 are performed for each of the multiple images associated with the annotation task:

[0070] Step 4201: For an image, based on multiple detection results corresponding to the image and a first confidence threshold, filter out candidate detection information corresponding to each object in the image.

[0071] 5 , when executing step 4201 , for each of the multiple test results, the following steps are performed:

[0072] Step 500: For a test result, determine the baseline test information in the test result; wherein the baseline test information is the unprocessed test information in the test result.

[0073] In some embodiments, for a detection result corresponding to a detection model, when executing step 500, each unprocessed detection information in the detection result is used as a benchmark detection information. For each determined benchmark detection information, the following steps are performed:

[0074] Step 5101: For a benchmark detection information, based on the IOU value between the benchmark detection information and each target detection information in other detection results, determine the matching detection information corresponding to the benchmark detection information in each other detection result; wherein the target detection information is the detection information that has the same detection category as the benchmark detection information and is not matched.

[0075] When executing step 5101, referring to FIG6 , the following steps are specifically performed:

[0076] Step 600: Determine the IOU value between the reference detection information and each target detection information in other detection results.

[0077] In some embodiments, when executing step 600, based on the detection frames corresponding to the benchmark detection information and each target detection information in other detection results, the ratio of the intersection and union of the corresponding two detection frames is calculated to obtain the IOU value corresponding to the benchmark detection information and each target detection information in other detection results.

[0078] Step 610: Use the other detection information corresponding to the maximum IOU value in each other detection result as matching detection information.

[0079] In some embodiments, when executing step 610, the following operations are performed for each other detection result: for one other detection result, the maximum IOU value is filtered out from the IOU values ​​corresponding to the benchmark detection information and each detection result in the other detection result, and the other detection information corresponding to the maximum IOU value is determined as the matching detection information that matches the benchmark detection information.

[0080] Step 5102: Based on the confidence of the benchmark detection information and the candidate detection information with a match greater than the IOU threshold in all matching detection information, a reference confidence of the object corresponding to the benchmark detection information is determined.

[0081] When executing step 5102, please refer to FIG. 7 , and specifically perform the following steps:

[0082] Step 700: Based on the IOU values ​​of all matching detection information and the benchmark detection information, and the IOU values ​​between every two matching detection information in all matching detection information, select candidate detection information with a value greater than the IOU threshold from all matching detection information and the benchmark detection information.

[0083] When executing step 700, based on all matching detection information, the IOU value of the detection box corresponding to every two matching detection information is determined, and from the IOU values ​​of the benchmark detection information and all matching detection information, as well as the IOU values ​​of the detection box corresponding to every two matching detection information, the alternative IOU values ​​greater than the IOU threshold are screened out, and the detection information corresponding to each alternative IOU value is used as the alternative detection information.

[0084] Step 710: Determine the maximum confidence level among the confidence levels corresponding to the candidate detection information as the reference confidence level of the object corresponding to the reference detection information.

[0085] When executing step 5101, if there is no target detection information in other detection results, the confidence corresponding to the benchmark detection information is used as the reference confidence of the object corresponding to the benchmark detection information. Then, it is determined whether the reference confidence is greater than the first confidence threshold, and after determining that the reference confidence is greater than the first confidence threshold, the subsequent step 5103 is executed; if it is determined that the reference confidence is less than or equal to the first confidence threshold, the benchmark detection information is deleted.

[0086] Step 5103: When it is determined that the reference confidence is greater than the first confidence threshold, each candidate detection information greater than the IOU value is determined as candidate detection information of the object corresponding to the reference detection information in the image.

[0087] In some embodiments, after executing step 5102 to obtain the reference confidence, if it is determined that the reference confidence is less than or equal to the first confidence threshold, it means that the detection information detected by the model this time is likely to be a false detection, then the benchmark detection information and each matching detection information in other detection results that match the benchmark detection information are deleted.

[0088] In order to better filter the detection information detected by each detection model and retain more accurate detection information corresponding to each object in each image, in some embodiments, the first confidence threshold includes multiple sub-thresholds, and specific values ​​can be set for the multiple sub-thresholds included in the first confidence threshold according to the specific number of detection models used.

[0089] In some embodiments, the first confidence threshold may be determined based on the number of determined candidate detection information (ie, the reference detection information and detection information greater than the IOU threshold among all matching detection information).

[0090] In some embodiments, depending on the number of candidate detection information determined, the following three situations are included but not limited to:

[0091] Case 1: If the number of determined alternative detection information is equal to the total number of multiple detection models, that is, the IOU value between the baseline detection information and each matching detection information, and the IOU value of each two matching detection information are greater than the IOU threshold, it means that the objects framed by the baseline detection information and all matching detection information are the same real object, and the object is detected by all three models, then the first sub-threshold (denoted as p1) among the multiple sub-thresholds is used as the first confidence threshold; at this time, if the reference confidence p is greater than p1, it is retained, that is, the baseline detection information and all matching detection information are retained; if the reference confidence p is less than p1, it is not retained, that is, the baseline detection information and all matching detection information are deleted.

[0092] In the second case, if the number of determined alternative detection information is less than the total number of multiple detection models and greater than a preset value (such as 1), that is, the IOU value between the baseline detection information and each matching detection information, and at least one of the IOU values ​​of every two matching detection information is greater than the IOU threshold, then the second sub-threshold (denoted as p2) among the multiple sub-thresholds is used as the first confidence threshold; at this time, if the reference confidence p is greater than p2, the detection information associated with the value greater than the IOU threshold is retained; wherein, if there is detection information other than the detection information associated with the value greater than the IOU threshold in the baseline detection information and all matching detection information, the detection information is deleted; if it is less than the confidence threshold p2, it is not retained, that is, the baseline detection information and all matching detection information are deleted; wherein p2 is greater than or equal to p1; compared with the case where the object is detected by all three models in the first case, the second case indicates that the object framed by the baseline detection information and the matching detection information greater than the IOU threshold is the same real object, that is, the object in the second case is detected by at least two models. Since the object in case 2 is detected by at least two of the three models, its value of being retained is higher. p2 can be set to be the same as or slightly larger than p1 to improve the screening accuracy of the detection information.

[0093] Case three, if the number of determined alternative detection information is equal to the preset value (such as 1), that is, the IOU value between the benchmark detection information and each matching detection information, and the IOU value of each two matching detection information are all less than the IOU threshold, or, it is determined that there is no target detection information in other detection results, then the third sub-threshold (denoted as p3) among the multiple sub-thresholds is used as the first confidence threshold; at this time, if the reference confidence p is greater than p3, the benchmark detection information is retained; if the reference confidence p is less than p3, it is not retained, that is, the benchmark detection information is deleted.

[0094] Case 3 indicates that there is no matching detection information that matches the baseline detection information. In other words, the object is detected by only one model and has a low value for retention. Based on this, p3 can be set to a larger value than p2, that is, p3>p2. This means that when an object is detected by only one model, only those with a high confidence level (greater than p3) will be retained.

[0095] The specific processing flow of step 4201 is described in detail below with a specific example.

[0096] For example, take the case where the labeling task is associated with three detection models.

[0097] In some embodiments, after executing step 410, the detection results corresponding to each image associated with the labeling task are obtained, that is, the three detection results are obtained by using these three detection models to perform target detection on objects of preset detection categories in each image.

[0098] Assume that the three detection models are recorded as the first model, the second model and the third model respectively.

[0099] When executing step 4201, the detection result corresponding to the first model is used as the benchmark detection result, and each detection information in the benchmark detection result is used as the benchmark detection information. For each benchmark detection information, the following operations are performed respectively:

[0100] Operation 1 (corresponding to the aforementioned step 600) is to determine, for a benchmark detection information, the IOU value of the benchmark detection information and each other detection information in the second model and the third model that has the same detection category as the benchmark detection information and does not match, that is, to calculate the intersection-over-union ratio of the detection box included in the benchmark detection information and the detection box included in each other detection information in the second model and the third model, and obtain the corresponding IOU value of each.

[0101] Operation 2 (corresponding to the aforementioned step 610): based on the IOU values ​​obtained in operation 1, the other detection information corresponding to the maximum IOU values ​​in the second model and the third model are determined as the matching detection information in the second model and the third model that matches the benchmark detection information, which are respectively recorded as matching detection information 1 and matching detection information 2; wherein, the IOU values ​​between the benchmark detection information and each matching detection information are respectively recorded as max_iou12 and max_iou13.

[0102] At the same time, the IOU value between matching detection information 1 and matching detection information 2 is determined and recorded as max_iou23.

[0103] Then, based on max_iou12, max_iou13, and max_iou23, the alternative detection information that is greater than the IOU threshold is screened out from the benchmark detection information, matching detection information 1, and matching detection information 2 (i.e., corresponding to the aforementioned step 700); the maximum confidence level among the confidence levels corresponding to the alternative detection information is determined as the reference confidence level of the object corresponding to the benchmark detection information (i.e., corresponding to the aforementioned step 710).

[0104] Again, the specific value of the first confidence threshold is determined based on the number of determined alternative detection information. The specific value of the first confidence threshold can also be determined based on the number of max_iou12, max_iou13, and max_iou23 that are greater than the IOU threshold.

[0105] Specifically, if max_iou12, max_iou13, and max_iou23 are all greater than the IOU threshold (corresponding to the aforementioned case one), it means that the object framed by the detection boxes included in the three detection information is the same object, that is, the object is detected by all three models, then the first sub-threshold (denoted as p1) among the multiple sub-thresholds is used as the first confidence threshold; at this time, the maximum confidence among the three prediction information is used as the reference confidence of the object corresponding to the benchmark detection information, denoted as p; if the reference confidence p is greater than p1, the three detection information are retained, that is, the benchmark detection information, matching detection information 1 and matching detection information 2 are all retained and determined as candidate detection information; if the reference confidence p is less than p1, the three detection information are deleted, that is, the benchmark detection information, matching detection information 1 and matching detection information 2 are deleted; if retained, the three detection information are subjected to the "edit box" operation to output the final labeling information.

[0106] If one or two of max_iou12, max_iou13, and max_iou23 are greater than the IOU threshold (corresponding to the aforementioned case 2), it means that the object framed by the detection box included in the benchmark detection information and the matching detection information greater than the IOU threshold is the same object, that is, the object is detected by at least two models, then the second sub-threshold (denoted as p2) among the multiple sub-thresholds is used as the first confidence threshold; at this time, the maximum confidence of the benchmark detection information and the matching detection information greater than the IOU threshold is used as the reference confidence of the object corresponding to the benchmark detection information, denoted as p; if the reference confidence p is greater than p2, the detection information associated with the value greater than the IOU threshold is retained and determined as the candidate detection information; wherein, if there is still detection information other than the detection information associated with the value greater than the IOU threshold in the benchmark detection information, matching detection information 1 and matching detection information 2, the detection information is deleted; if it is less than the confidence threshold p2, it is not retained, that is, the benchmark detection information, matching detection information 1 and matching detection information 2 are deleted; wherein, p2 is greater than or equal to p1; if it is retained, the two detection information are subjected to the "edit box" operation to output the final labeling information.

[0107] If max_iou12, max_iou13, and max_iou23 are all smaller than the IOU threshold, or it is determined that there is no unmatched detection information (i.e., the aforementioned target detection information) in the detection results corresponding to the second model and the third model (corresponding to the aforementioned case three), indicating that there is no matching detection information that matches the benchmark detection information, the third sub-threshold among the multiple sub-thresholds (denoted as p3) is used as the first confidence threshold; at this time, the confidence corresponding to the benchmark detection information is used as the reference confidence of the object corresponding to the benchmark detection information, denoted as p. If the reference confidence p is greater than p3, the benchmark detection information is retained and determined as candidate detection information; if the reference confidence p is less than p3, it is not retained, that is, the benchmark detection information is deleted; wherein, p3 is greater than p2. In practical applications, p3 can be set to a larger value, indicating that if only one model is detected, it can only be retained if its confidence is greater than this larger p3).

[0108] After processing the detection results corresponding to the first model, the detection results corresponding to the second model are used as the benchmark, and the unprocessed detection information is used as the benchmark detection information. For each benchmark detection information, perform the following operations respectively:

[0109] For a benchmark detection information, the IOU value is calculated one by one between the benchmark detection information and the unprocessed detection information in the detection result corresponding to the third model, and the detection information with the largest IOU value is found as the matching detection information 3 of the benchmark detection information in the third model. The IOU value between the benchmark detection information and the matching detection information 3 is still recorded as max_iou23.

[0110] According to the relationship between max_iou23 and the IOU threshold, the specific value of the first confidence threshold is determined.

[0111] If max_iou23 is greater than the IOU threshold (corresponding to the aforementioned case 2), it means that the objects framed by the detection boxes included in the benchmark detection information and the matching detection information 3 are the same real object, that is, both models have detected the object, and the second sub-threshold (denoted as p2) among the multiple sub-thresholds is used as the first confidence threshold; at this time, the maximum confidence of the two detection information is used as the reference confidence of the object corresponding to the benchmark detection information, denoted as p; if the reference confidence p is greater than p2, the benchmark detection information and the matching detection information 3 are retained and determined as candidate detection information; if the reference confidence p is less than p2, they are not retained, that is, the benchmark detection information and the matching detection information 3 are deleted; if they are retained, the two detection information are subjected to the "edit box" operation, and the final annotation information is output;

[0112] If max_iou23 is less than the IOU threshold, or it is determined that there is no unmatched detection information (i.e., the aforementioned target detection information) in the detection result corresponding to the third model (corresponding to the aforementioned case three), it means that the benchmark detection information has no matching detection information, and the third sub-threshold (denoted as p3) among the multiple sub-thresholds is used as the first confidence threshold; at this time, the confidence corresponding to the benchmark detection information is used as the reference confidence p of the object corresponding to the benchmark detection information. If the reference confidence p is greater than p3, the benchmark detection information is retained and determined as candidate detection information; if the reference confidence p is less than the threshold p3, it is not retained, that is, the benchmark detection information is deleted.

[0113] After processing the detection results corresponding to the second model, the detection results corresponding to the third model are used as the benchmark, and the unprocessed detection information is used as the benchmark detection information. For each benchmark detection information, perform the following operations respectively:

[0114] For a benchmark detection information, the third sub-threshold (denoted as p3) among the multiple sub-thresholds is used as the first confidence threshold (corresponding to the aforementioned case three); at this time, the confidence corresponding to the benchmark detection information is used as the reference confidence of the object corresponding to the benchmark detection information, denoted as p; if the reference confidence p is greater than p3, the benchmark detection information is retained and determined as candidate detection information; if the reference confidence p is less than p3, it is not retained, that is, the benchmark detection information is deleted.

[0115] In this way, through the above processing flow, the candidate detection information corresponding to each object in the image can be obtained, preparing for the subsequent "edit box" operation.

[0116] Step 4202: Based on the candidate detection information corresponding to each object in the image, obtain the labeling result corresponding to the image; wherein the labeling result includes the labeling information corresponding to each object in the image.

[0117] When executing step 4202, referring to FIG8 , the following steps are specifically performed:

[0118] Step 800: For each object in the image, perform the following steps respectively:

[0119] Step 8001: If there are multiple candidate detection information for an object, the multiple candidate detection information are fused to obtain the labeling information of the object.

[0120] After executing step 4201 and before executing step 8001, if it is determined that the candidate detection information corresponding to the object is the same as the number of the multiple detection models, and the confidence level corresponding to each candidate detection information is greater than the second confidence threshold, the identifier of the object is set to a valid value. This identifier is used to set a specific color for the fused marking box of the object, so that the annotator does not need to modify this type of annotation information during the second annotation, thereby significantly improving the annotation speed and efficiency.

[0121] In some embodiments, when executing step 8001, any one of the following fusion methods may be used to obtain the annotation information of the object:

[0122] Method 1: The average value of the coordinates of the detection boxes corresponding to multiple candidate detection information is used as the coordinates of the final detection box, and the labeling information of the object is obtained based on the coordinates of the final detection box.

[0123] In some embodiments, the candidate detection information includes the coordinates of the detection box that frames the corresponding object in the corresponding image; wherein each detection box is represented by a center point, as well as the width and height of the detection box, and the center point is denoted as (x i ,y i ), width and height are w i ,h i .

[0124] Then, when using method 1, the average of the center points of all detection frames is taken as the center point of the final detection frame after fusion, which is recorded as Take the average of the width and height of all detection frames as the width of the final detection frame after fusion high In some embodiments, the coordinates of the final fused detection frame, that is, the coordinates of the marking frame included in the annotation information of object i, can be expressed by the following formula:

[0125] In the above formula, i is the object number, and n is the total number of objects contained in the corresponding image.

[0126] Method 2: Based on the coordinates of the detection boxes corresponding to the multiple candidate detection information, the coordinates of the minimum bounding rectangle containing the multiple detection boxes are determined, and the labeling information of the object is obtained based on the coordinates of the minimum bounding rectangle.

[0127] When using the second method, for any object, the coordinates of the upper left corner of each detection box are (x 1j ,y 1j ), the coordinates of the lower right corner are (x 2j ,y 2j); where j represents the jth prediction box of the corresponding object; then, the coordinates of the upper left corner of the fused detection box (i.e., the aforementioned minimum bounding rectangle) are expressed as Indicates that the coordinates of the lower right corner are expressed as In some embodiments, the coordinates of the minimum bounding rectangle after fusion can be expressed as follows:

[0128] Step 8002: If there is candidate detection information for an object, the candidate detection information is used as the labeling information of the object.

[0129] In some embodiments, after executing step 800 to obtain the annotation information of the object, before executing the subsequent step 810 to determine the annotation information corresponding to each object in the image as the annotation result corresponding to the image, it is also necessary to adjust the mark box color in the annotation information corresponding to the object from the first color to the second color when it is determined that the identifier of the object is a valid value; wherein the first color is the default mark box color, and the second color is the aforementioned specific color.

[0130] For example, let’s take the case where the labeling task is associated with three detection models.

[0131] In some embodiments, when it is determined that a certain benchmark detection information meets situation one, that is, the detection box contained in the benchmark detection information is detected by each model, it is determined whether the confidence of the three detection information of the object corresponding to the benchmark detection information in each detection model is greater than the second confidence threshold; wherein, the second confidence threshold is greater than the first confidence threshold. In some embodiments, the second confidence threshold is greater than the third sub-threshold. If they are all greater than the second confidence threshold, the identifier of the object corresponding to the benchmark detection information is set to a valid value, such as sure_flag=1.

[0132] In some embodiments, before executing step 810, when sorting out each fused annotation information, it is determined that the identifier of a certain object is a valid value, that is, it is determined that the sure_flag of the object is 1, then the color of the marking box in the annotation information corresponding to the object is adjusted from the first color to the second color; wherein the first color is the default color corresponding to the corresponding detection category, and the second color is the aforementioned specific color; in this way, the object corresponding to the benchmark detection information that meets the first situation can be distinguished by the color of the marking box. Through this operation, the annotation speed of the annotation personnel can be further improved, and the annotation work efficiency can be improved.

[0133] In some embodiments, the process shown in FIG. 9 may be executed before executing step 810. Specifically, as shown in FIG. 9 , the process includes the following steps:

[0134] Step 900: For each piece of annotation information, perform the following operations respectively: for one piece of annotation information, determine the CIOU value between the annotation information and each other piece of annotation information respectively.

[0135] After the above strategy, the following problems may arise: for some objects, different detection models detect different detection categories. After the above strategy, the same object may have two almost identical marking boxes, but different detection categories. As shown in Figure 10, for the bus on the right side of the image, after the above-mentioned retain box strategy, edit box strategy, and determine box strategy, the bus has two annotation information, namely "3" and "9" shown in Figure 10. One annotation information (i.e., "3" shown in Figure 10) includes a marking box with a yellow color and a detection category of "car"; the other annotation information (i.e., "9" shown in Figure 10) includes a marking box with an orange color and a detection category of "bus".

[0136] If these two annotations are not further processed, the annotator will need to perform another deletion operation when performing the second annotation, which reduces the annotation speed and increases the annotation time. Therefore, in some embodiments, only one annotation information is required for the same object. This makes it easier for the annotator to check and correct based on the single annotation information. If the detection category is incorrect, only the detection category needs to be modified, thereby reducing the annotation workload and improving the annotation speed.

[0137] Based on this, after executing step 800 to obtain the labeling information corresponding to each object, before executing step 810, the aforementioned filter box strategy is executed.

[0138] For the annotation information retained after the multiple detection models execute the edit box strategy, a filtering operation is performed, that is, step 900 is executed:

[0139] For each annotation information, the following formula is used to calculate the CIOU value of the marking box included in the annotation information and each other annotation information:

[0140] Where Distance_2 represents the Euclidean distance between the center points of the two marked boxes, and Distance_C represents the distance between the diagonals of the minimum circumscribed rectangles of the two marked boxes.

[0141] Step 910: For each CIOU value, perform the following operations respectively: for a CIOU value, when it is determined that the CIOU value is greater than the CIOU threshold, delete the annotation information corresponding to the smaller confidence level of the two annotation information associated with the CIOU value.

[0142] In some embodiments, when executing step 910, the following operations are performed for each CIOU value: for a CIOU value, the CIOU value is compared with the CIOU threshold. If the CIOU value is greater than the CIOU threshold, then for the two marking information, the marking information with a higher confidence level is retained, that is, the marking information with a lower confidence level is deleted.

[0143] By using the CIOU value, we can fully consider the overlap area, center point distance, and aspect ratio between each pair of markers. Ultimately, we consider the objects selected by two markers with a CIOU value greater than a certain threshold (CIOU threshold) to be the same object. We filter out the marker information with lower confidence and retain only the marker information with higher confidence. Considering the similarity between markers based on the overlap area, center point distance, and aspect ratio, and filtering out the annotation information corresponding to redundant markers, can effectively improve the accuracy of the model's pre-marked boxes.

[0144] Still referring to Figure 10, by executing the filter box strategy, one annotation information is filtered out and only one annotation information is retained, that is, only one marking box is retained; if the annotation information corresponding to "car" is retained (that is, "3" shown in Figure 10), the annotator only needs to correct the detection category when performing the secondary annotation. If the annotation information corresponding to "bus" is retained (that is, "9" shown in Figure 10), the annotator does not need to perform any further operations when performing the secondary annotation, which will further save annotation time and improve annotation speed.

[0145] Step 810: Determine the annotation information corresponding to each object in the image as the annotation result corresponding to the image.

[0146] Figure 11 is a schematic diagram comparing the data annotation results of single-model detection and multi-model fusion provided by some embodiments of the present disclosure. Referring to Figure 11 , taking the three detection models associated with this annotation task as an example, the corresponding detection results obtained for the same image are shown. The upper left, upper right, and lower left corners of Figure 11 are the detection results of the three single models, respectively, while the lower right corner shows the annotation result after multi-model fusion.

[0147] By comparing the results of single model and those based on multi-model fusion, it can be clearly seen that in the annotation results based on multi-model fusion, the detection results of each single model are complemented by each other. As shown in Figure 11, only three objects are detected in the single model detection result shown in the upper left corner, and the detection category is "car"; four objects are detected in the single model detection result shown in the upper right corner, and the detection categories are "rider", "rider", "bus" and "person" respectively; among them, only one object appears in the single model detection result shown in the upper left corner, but the detection categories of the two are different, one is "car" and the other is "bus"; 8 objects are detected in the single model detection result shown in the lower left corner, and the detection categories are "car", "car", "rider", "other", "rider", "car", "rider", and "person" respectively; among them, 3 objects appear in the left corner. In the single-model detection results shown in the upper corner, the detection categories are consistent. Four objects appear in the single-model detection results shown in the upper right corner, and three of the detection categories are consistent, while the other detection category is inconsistent. In the annotation information based on multi-model fusion, there are seven annotation information, and the detection categories are "car," "car," "rider," "other," "rider," "car," and "person," respectively. Among them, the four objects with detection categories of "rider," "other," "rider," and "person" that were missed in the single-model detection results shown in the upper left corner are annotated, and the three objects corresponding to "car," "car," and "other" that were missed in the single-model detection results shown in the upper right corner are annotated. The object with the detection category of "bus" that was mistakenly detected is deleted. The object with the detection category of "rider" that was mistakenly detected in the single-model detection results shown in the lower left corner is deleted. This further improves the accuracy of the annotation information.

[0148] The data labeling method provided in some embodiments of the present disclosure can obtain expected results that are better than those of a single model, which meets the characteristic of reliability in the trustworthiness characteristic; the method is applicable to all detection models (algorithms), and when the user receives a new labeling task, he only needs to pre-train multiple detection models before using the above-mentioned data labeling method, which meets the characteristic of generalizability in the trustworthiness characteristic; and the data labeling method in the present disclosure can obtain the same results in different operating environments, which meets the characteristic of reproducibility in the trustworthiness characteristic; in any case, it can maintain the characteristic of being better than the effect of a single model, which meets the characteristic of robustness in the trustworthiness characteristic; it can be intervened by other intelligent agents during the reasoning process, which meets the characteristic of controllability in the trustworthiness characteristic.

[0149] A data annotation method provided by some embodiments of the present disclosure can also obtain an image to be processed and determine the target object in the image to be processed; then determine the outline of the target object in the image to be processed based on the target model, and annotate and display the outline of the target object according to the sparsity of the saved annotation points. Therefore, there is no need to manually outline the outline of the target object in detail. It is only necessary to determine the outline of the target object in the image to be processed based on a large model, and then annotate the outline of the target object according to the sparsity of the pre-saved annotation points. Compared with the related art that often uses manual methods to annotate the outline of the target object, the efficiency of the target object outline annotation can be improved. When implemented, it can be shown in Figure 12 and include the following steps:

[0150] Step 1201: Acquire an image to be processed and determine a target object in the image to be processed.

[0151] In some embodiments, an image to be processed can be obtained when annotating the outline of a target object. The image to be processed can be any image. The image to be processed can be sent by other electronic devices connected to the electronic device, or sent by the user of the electronic device. For example, the user of the electronic device imports an image based on a visual operation interface displayed on the front end, and the imported image received by the electronic device can be used as the image to be processed. When importing an image, the user of the electronic device can import one image or multiple images. When importing multiple images, they can be imported one by one or a set number of images can be imported in batches. When receiving multiple images, the electronic device can determine each image as an image to be processed and annotate the outline of the target object in each image to be processed.

[0152] In order to clearly identify which target object in the image to be processed is to be labeled, in some embodiments, the target object in the image to be processed can also be determined. The target object can be any element selected by the user of the electronic device in the image to be processed. The target object can be a pedestrian, a vehicle, a tree, a flower, an animal, etc. In some embodiments, the user of the electronic device can input rough location information of the target object, and the target object in the image to be processed can be subsequently determined based on the rough location information. The rough location information is information used to describe the approximate location of the target object in the image to be processed.

[0153] For example, the rough location information can be the location information of the annotation box of the target object, and the annotation box can be determined based on the location information, and the elements included in the annotation box are determined as the target object. For another example, the rough location information can also be the information of a certain pixel point in the area where the target object is located, and image recognition can be performed on the image to be processed to determine which element in the image to be processed includes the pixel point, so as to determine the corresponding element as the target object. The pixel point can be the pixel point clicked by the user of the electronic device in the image when importing the image. In some embodiments, although the target object also requires manual participation in determination, compared with the traditional method of manually selecting and outlining the irregular outline of the target object, the operation of clicking or framing any element can greatly improve efficiency.

[0154] FIG13 is a schematic diagram of an image to be processed provided by some embodiments of the present disclosure. The rectangular frame marked in FIG13 can be regarded as a frame selected by a user of the electronic device, and the vehicle included in the rectangular frame can be determined as a target object.

[0155] Step 1202: Determine the contour of the target object in the image to be processed based on the target model.

[0156] After determining the target object in the image to be processed, in order to determine the outline of the target object, the outline of the target object in the image to be processed can be determined based on the target model. The target model provided in some embodiments of the present disclosure refers to any network model with image recognition processing capabilities. During implementation, any network model with image recognition processing capabilities can be used to analyze and process the image to be processed and identify the outline of the target object in the image to be processed.

[0157] During implementation, the prompt information of the image to be processed, the target object, and the prompt word can be input into the multimodal target model, so that the multimodal target model analyzes the image area where the target object in the image to be processed is located according to the prompt of the prompt word and determines the outline of the target object. In some embodiments, the prompt information of the target object is input into the target model to enable the target model to know which object to analyze. In some embodiments, the target object can be marked in the image to be processed, or the position of the target object in the image to be processed can be described in the form of text. Some embodiments of the present disclosure do not limit this.

[0158] Step 1203: Mark out the outline of the target object according to the sparsity of the stored annotation points and display it.

[0159] After the outline of the target object in the image to be processed is determined, the outline of the target object can be marked in the image to be processed using marking points and displayed.

[0160] In some embodiments, the outline of the target object can be marked in the image to be processed based on the sparsity of the saved annotation points. The sparsity can be that the outline of the target object is marked with one annotation point for every set number of pixel points. A person skilled in the art can configure the sparsity as needed. The annotation point can be understood as a point obtained by enlarging the corresponding pixel point, or as a point added to the image to be processed. The annotation point can be solid or hollow, and the annotation point can be of any shape. When marking the same object, the appearance of the annotation points used can be the same or different. For ease of understanding, the annotation points are explained below with reference to FIG14. FIG14 is a schematic diagram of a target object annotation provided in some embodiments of the present disclosure. In FIG14, a plurality of small rectangular boxes are used to outline the outline of the vehicle. These small rectangular boxes can be regarded as annotation points. It should be noted that a person skilled in the art can configure the method of marking the outline of the target object as needed, and some embodiments of the present disclosure do not limit this.

[0161] Some embodiments of the present disclosure obtain the target object contour annotation results interactively, without the need to manually outline and annotate irregular contours, with good effect and fast speed, and the quality of the segmentation result is better than the result of manually outlining and annotating irregular contours.

[0162] In some embodiments, an image to be processed is acquired, a target object in the image to be processed is determined, and then the contour of the target object in the image to be processed is determined based on a target model. The contour of the target object is annotated and displayed based on the sparsity of the stored annotation points. This eliminates the need to manually outline the contour of the target object in detail. Instead, the contour of the target object in the image to be processed only needs to be determined based on the target model, and then the contour of the target object is annotated based on the sparsity of the pre-stored annotation points, thereby improving the efficiency of target object contour annotation.

[0163] In order to improve the accuracy of target object contour annotation, based on the above embodiment, in some embodiments, determining the contour of the target object in the image to be processed based on the target model includes:

[0164] Inputting the image to be processed into the target model, so that the target model determines the pixel points of the target object in the image to be processed;

[0165] The contour of the target object is determined according to the pixel points of the target object.

[0166] In order to improve the accuracy of target object contour annotation, when determining the contour of the target object in the image to be processed based on the target model, the target model can be used to first identify all the pixel points included in the target object. In some embodiments, the target model can be any target model with image processing capabilities. Optionally, the target model can be a segmentation model (Segment Anything Model, SAM) or a fast segmentation model (Fast Segment Anything Model, FastSAM). It should be noted that those skilled in the art can configure the target model as needed, and some embodiments of the present disclosure do not limit this. The user of the electronic device can choose which target model to use for target object contour annotation. For example, when creating a target object contour annotation task, the user of the electronic device can select the target model to be used from the target model list in the front-end visual interface.

[0167] In some embodiments, the image to be processed can be input into the target model, so that the target model analyzes the image area where the target object is located in the image to be processed and identifies the pixel points of the target object. That is, the target model is used to identify the pixel points of the target object and the pixel points of the non-target object in the image to be processed. For example, the target model can output a mask image corresponding to the image to be processed, and the location area where the target object is located is marked in the mask image. For example, the values ​​of the pixels corresponding to the location where the target object is located in the mask image are all 1, and the values ​​of the pixels corresponding to other non-target objects are all 0. It should be understood that the specific size of the mask image can be set based on actual needs. For example, the size of the mask image can be set to be the same as that of the image to be processed, or the size of the mask image can be set to be different from that of the image to be processed. This is not limited in the present disclosure.

[0168] After determining the target object's pixels in the image to be processed, the target object's outline can be determined based on the target object's pixels, facilitating subsequent labeling of the target object's outline. For example, the target object's pixels can be traversed sequentially by pixel rows in the order in which they appear in the image to be processed. The first and last target object pixels in each row are determined as pixels on the target object's outline. For ease of description, these pixels can be referred to as outline points.

[0169] In order to clearly mark the target object, based on the above embodiment, in some embodiments, before displaying the outline of the marked target object, the method further includes:

[0170] receiving identification information for identifying the target object;

[0171] A label of the target object is determined according to the identification information, and the label is added to the target object in the image to be processed.

[0172] Because the same image to be processed may include multiple target objects, in some embodiments, a label can be added to each target object to facilitate the user of the electronic device to intuitively see the outline of each target object. The label can be used to describe the target object. For example, if the target object is a person running, the label of the target object can be "pedestrian". For another example, if the target object is a person walking their dog, the label of the target object can also be "pedestrian".

[0173] When determining the label of each target object, in some embodiments, identification information identifying the target object may be received; the identification information is text used to describe the target object. The identification information may be input by the user of the electronic device. For example, when the user of the electronic device imports an image to be processed, the user enters the identification information while selecting the target object. The identification information may be input by the user of the electronic device using an input device, or may be selected from a list of identification information on a front-end visual interface. For example, the identification information may be "pedestrian," "running person," or "man in red clothes."

[0174] After receiving the identification information of the target object, the tag of the target object can be determined based on the identification information. When determining the tag of the target object, the identification information can be determined as the tag, or the identification information can be semantically analyzed to obtain keywords, and the keywords can be determined as the tags.

[0175] After the label of the target object is determined, the label can be added to the image to be processed.

[0176] In order to further clearly mark the target object, based on the above embodiments, in some embodiments, determining the label of the target object according to the identification information includes:

[0177] Obtaining other identification information of other objects whose outlines are marked in the image to be processed;

[0178] If the identification information matches any of the other identification information, the label of the target question is determined according to the label of the object corresponding to the other identification information.

[0179] Since multiple target objects may be annotated at once in the image to be processed, in order to more clearly annotate each target object, when determining the label of the target object based on the identification information, in some embodiments, other identification information of other objects whose outlines have been annotated in the image to be processed can be obtained, and it can be determined whether the identification information of the target object matches any of the other identification information. Here, a match can be understood as similarity, or a similarity greater than a set threshold. In other words, it is determined whether there is an object similar to the target object among the other annotated objects. If so, the label of the target object can be determined based on the label of the other matching objects.

[0180] If it is determined that the identification information matches any other identification information, the label of the target object is determined based on the label of the object corresponding to the other identification information. In some embodiments, the label of the object corresponding to the other identification information can be determined as the label of the target object. It is also possible to add serial numbers incrementally based on the label of the object corresponding to the other identification information to distinguish different objects. For example, the image to be processed includes a person walking a dog and a person running. Among them, the person walking the dog is the target object, and the person walking the dog is another object with an outline marked. When the label of the person walking the dog is "Pedestrian 1", the label of the target object "person running" can be determined as "Pedestrian 2".

[0181] In some embodiments, to facilitate distinguishing different tags, a background color may be set for each tag when adding each tag to the image to be processed. Tags of the same type have the same background color, while tags of different types have different background colors.

[0182] Figure 15 is a schematic diagram of annotating the outline of a target object according to some embodiments of the present disclosure. The label list shown on the right side of Figure 15 (left and right in the diagram) indicates that two labels, pedestrian and vehicle, are pre-set. The background color of the pedestrian label is white, while the background color of the vehicle label is black. The labels set in this label list can be those created by the user of the electronic device when creating a target object outline annotation task. In other words, the user of the electronic device can customize the label content and background color based on the purpose of the image to be processed and the actual application scenario. Once set, the label of the corresponding target object is displayed with the specified color and content. When annotating the target object outline, you can first select a label. The selected label indicates that the target object to be annotated belongs to the selected label category. After annotating the target object outline, the corresponding label is added to the image to be processed. The image shown on the left side of Figure 15 is the processed image to be processed. The irregular shape annotated in the image to be processed is the outline of the target object. The "1" with a black background color corresponding to the target object's location indicates that the target object corresponds to the vehicle label and is the first vehicle identified.

[0183] In some embodiments, users of electronic devices can customize labels, which are widely used. Users can customize labels according to actual business needs and select corresponding labeling categories.

[0184] In order to improve the accuracy of tag identification, based on the above embodiments, in some embodiments, the method further includes:

[0185] If a first instruction for modifying the tag is received, the tag is updated using the modification value carried in the first instruction.

[0186] Since each tag is annotated by the user of the electronic device, it is very likely that mislabeling will occur during the tagging process. For example, the target object is a vehicle, but the tag of the target object is mistakenly identified as a pedestrian. In some embodiments, a first instruction to modify a tag can be received. This first instruction can be issued by the user of the electronic device when modifying a certain tag. The first instruction carries a modification value, which can be understood as the content to which the tag is desired to be modified.

[0187] If the first instruction is received, the corresponding tag may be updated using the modification value carried in the first instruction.

[0188] Specifically, assuming that a label in the image to be processed is "vehicle," if the user of the electronic device discovers an error, the user can modify "vehicle" to "pedestrian" and send "pedestrian" as the modified value in a first instruction to the electronic device. After receiving the first instruction, the electronic device can modify the corresponding label "vehicle" to "pedestrian."

[0189] Figure 16 is a schematic diagram of label modification provided by some embodiments of the present disclosure. The user of the electronic device can find the labels of all target objects marked in the image to be processed in the result list on the right side of Figure 16 (left and right in the figure). When the user of the electronic device believes that a certain label needs to be modified, the user can click the modification icon to modify the corresponding label. The user of the electronic device can select the label as the modification value to be carried in the first instruction in the pop-up "Result Correction" window. After the user of the electronic device clicks "OK" in the "Result Correction" window, the first instruction is sent to the electronic device.

[0190] In order to clearly display the outline of the target object, based on the above embodiments, in some embodiments, after marking the outline of the target object according to the sparsity of the stored annotation points and before displaying the outline of the target object, the method further includes:

[0191] Counting the target number of the marking points that mark the contour of the target object;

[0192] A first sparse value stored for a quantity interval in which the target quantity is located is obtained, and annotated points that mark the contour of the target object are adjusted according to the first sparse value.

[0193] Normally, the denser the annotation points used to outline the contour of the target object, the closer the polygon formed will fit the real contour. However, in some scenarios, the segmentation target is relatively large, and so many dense points are not needed. Sparse points can also achieve good segmentation, and the more dense points there are, the more operations are required when editing the points in case the segmentation result needs to be manually corrected. Therefore, in some embodiments, after marking the contour of the target object according to the sparsity of the saved annotation points, before displaying the contour of the target object, the target number of annotation points that mark the contour of the target object can also be counted. And based on the correspondence between the pre-saved number interval and different sparse values, the number interval where the target number is located is determined, and the sparse value corresponding to the number interval is determined as the first sparse value. Then, the annotation points that mark the contour of the target object are adjusted according to the first sparse value.

[0194] Specifically, assume that three quantity intervals are divided based on the quantities num1 and num2. The sparse value corresponding to the interval (0, num1] is thres1, the sparse value corresponding to the interval (num1, num2] is thres2, and the sparse value corresponding to the interval (num2, ∞) is thres3. This correspondence can be expressed using the following formula:

[0195] Among them, point_num represents the number of targets.

[0196] When the target number is less than num1, the first sparse value may be determined to be thres1. When the target number is greater than or equal to num1 and less than num2, the first sparse value may be determined to be thres2. When the target number is greater than or equal to num2, the first sparse value may be determined to be thres3.

[0197] When adjusting the annotation points of the outline of the target object according to the first sparse value, the interval points can be taken according to the determined first sparse value. Here, taking the interval points can be understood as retaining only one annotation point for every certain number of annotation points. That is to say, according to the determined first sparse value, part of the outline of the target object is selected from the already marked annotation points. For example, when the first sparse value is 1, all the annotation points can be retained. When the first sparse value is 2, one annotation point can be retained for every two annotation points. When the first sparse value is 3, one annotation point can be retained for every three annotation points. That is to say, when adjusting the annotation points of the outline of the target object according to the first sparse value, some of the original annotation points can be deleted, or no annotation points can be deleted.

[0198] In some embodiments, a default correspondence between quantity intervals and sparse values ​​may be preconfigured, and users of electronic devices may modify corresponding parameters according to their own labeling requirements.

[0199] In some embodiments, the default correspondence can be expressed using the following formula:

[0200] Among them, point_num represents the number of targets.

[0201] In the above formula, when the target number point_num is between (0,50], all contour points are determined as annotation points; when the target number point_num is between (50,300], one annotation point is retained for every 2 annotation points; when the target number point_num is greater than 300, one annotation point is retained for every 8 annotation points.

[0202] In order to clearly display the outline of the target object, based on the above embodiments, if the configured outline marking method is adaptive, after marking the outline of the target object according to the sparsity of the saved marking points, before displaying the outline of the target object, the target area of ​​the detection frame of the target object can also be determined; obtain a second sparse value saved for the area interval where the target area is located, and adjust the marking points that mark the outline of the target object according to the second sparse value and the target number of marking points that mark the outline of the target object.

[0203] In some embodiments, a target object outline annotation method may be preconfigured, and this annotation method may be configured by the user of the electronic device when creating a target object outline annotation task. If the preconfigured outline annotation method is adaptive, after the target object outline is annotated based on the sparsity of the stored annotation points, and before the target object outline is displayed, a second sparsity value may be adaptively determined based on the target object's target area in the image to be processed.

[0204] When determining the area of ​​the target object, in some embodiments, a detection frame of the target object may be determined, and a target area of ​​the detection frame may be determined.

[0205] After determining the target area, a second sparse value corresponding to the area interval in which the target area is located can be determined based on the pre-stored correspondence between different area intervals and sparse values. The annotation points that mark the outline of the target object are then adjusted based on the second sparse value and the target number of annotation points that mark the outline of the target object.

[0206] In some embodiments, when the target area is less than 32*32 pixels, the target object can be determined to be a small target and a high sparse value can be used; when the target area is between 32*32 pixels and 96*96 pixels, the target object can be determined to be a medium target and a medium sparse value can be used; when the target area is greater than 96*96 pixels, the target object can be determined to be a large target and a low sparse value can be used. Table 1 shows the parameters corresponding to the high, medium, and low sparse values ​​provided in some embodiments of the present disclosure:

[0207] Table 1

[0208] As shown in Table 1, when the target area is less than 32*32 pixels, a high sparsity value can be used. For high coefficient values, if the number of targets is less than or equal to the threshold of 50, all annotation points can be determined as annotation points without adjustment; if the number of targets is greater than the threshold of 50 and less than or equal to the threshold of 100, all annotation points can be determined as annotation points without adjustment; if the number of targets is greater than the threshold of 100, one annotation point can be retained for every two annotation points.

[0209] When the target area is between 32*32 pixels and 96*96 pixels, a medium sparsity value can be used. For a medium sparsity value, if the number of targets is less than or equal to the threshold of 50, all annotation points can be determined as annotation points without adjustment. If the number of targets is greater than the threshold of 50 and less than or equal to the threshold of 300, one annotation point can be retained for every 2 annotation points. If the number of targets is greater than the threshold of 300, one annotation point can be retained for every 8 annotation points.

[0210] When the target area is larger than 96*96 pixels, a low sparsity value can be used. For a low sparsity value, if the number of targets is less than or equal to the threshold of 100, one annotation point can be retained for every 2 annotation points; if the number of targets is greater than the threshold of 100 and less than or equal to the threshold of 200, one annotation point can be retained for every 4 annotation points; if the number of targets is greater than the threshold of 200, one annotation point can be retained for every 8 annotation points.

[0211] Figure 17 is a schematic diagram of densely labeled points provided by some embodiments of the present disclosure. As shown in Figure 17, the target object in the image is a vehicle, and the outline of the target object is outlined using densely packed labeled points. If the outline is modified, more labeled points need to be moved.

[0212] Figure 18 is a schematic diagram of sparse annotation points provided by some embodiments of the present disclosure. As shown in Figure 18, the target object in this image is the same as the target object in Figure 17, but in Figure 18, fewer annotation points are used to outline the outline. Intuitively, the outline outlined with fewer annotation points is clearer, and if the outline needs to be modified, it can be modified by moving fewer annotation points.

[0213] In some embodiments, users of electronic devices can independently configure sparse values ​​or use an adaptive method to adaptively select sparse values ​​based on the target object. In this way, the speed of manual modification can be increased, further improving the annotation efficiency.

[0214] In order to improve the accuracy of target segmentation, based on the above embodiments, if a second instruction is received to modify the marked contour, the pixel points corresponding to the target position information carried in the second instruction can be adjusted to marking points, and the marking points corresponding to the initial position information carried in the second instruction can be deleted.

[0215] Because in some special cases, the result of the target object outline annotation may be inaccurate, therefore, in some embodiments, the user of the electronic device can modify the outline of the marked target object. If the user of the electronic device determines that the annotation result is inaccurate, a second instruction to modify the marked outline can be issued. For example, the user of the electronic device can edit the annotation point by clicking the "Edit" button. Among them, editing can include adding points, moving points, and deleting points, so that the annotation result perfectly fits the outline of the target object. In some embodiments, the initial position information and target position information of the annotation point that needs to be modified can be carried in the second instruction and sent to the electronic device.

[0216] If the electronic device receives a second instruction to modify the marked outline, it can adjust the pixel points corresponding to the target position information to the marked points and delete the marked points corresponding to the initial position information. In other words, the pixel points processed by the initial position information are restored to their original state without any marking.

[0217] Specifically, if the user of the electronic device wishes to move a marked point, the user of the electronic device can select the marked point to be moved and drag the marked point. When generating the second instruction, the information of the original position of the marked point can be determined as the initial position information, and the information of the position to which the marked point is dragged can be determined as the target position information.

[0218] If the user of the electronic device wishes to add a marking point, the user of the electronic device can click at the location where the marking point is desired to be added, thereby performing the operation of adding the marking point. When generating the second instruction, the information corresponding to the location clicked by the user of the electronic device can be determined as the target location information, and the initial location information identifier carried in the second instruction can be left blank.

[0219] If the user of the electronic device wishes to delete a marked point, the user of the electronic device can click Delete at the location of the marked point to be deleted, thereby deleting the point. When generating the second instruction, the location information corresponding to the marked point selected by the user of the electronic device can be determined as the initial location information, and the target location information identifier carried in the second instruction can be left blank.

[0220] In some embodiments, manual modification can be performed to achieve better annotation effects.

[0221] In order to further improve the effect of contour marking, based on the above embodiments, in some embodiments, the method further includes:

[0222] If a deletion instruction to delete the outline of the target object is received, the marking points marking the outline of the target object are deleted.

[0223] In actual applications, it is common to adjust the target object. For example, target object A may be marked previously, but later, due to changes in business needs, it is necessary to change the marked target object A to target object B. This requires deleting the outline of the marked target object A from the image to be processed. Therefore, in some embodiments, a deletion instruction can also be received. This deletion instruction can be sent by the user of the electronic device. The user of the electronic device can select the target object to be deleted and click the "Delete" button to send the deletion instruction.

[0224] If the electronic device receives a deletion instruction to delete the outline of the marked target object, the outline points of the marked target object can be deleted. The user of the electronic device can create a target segmentation task. When creating the target segmentation task, the user can select and upload the image to be processed. The image to be processed can be an image stored in the image set. And the label of the target object, the sparse value, and the target object marking instruction are initiated. When marking the outline of the target object, the user of the electronic device can select a rectangular box in the image to be processed. The object included in the rectangular box is the target object, and the image to be processed is input into the segmentation model to identify the pixel points of the target object in the image area corresponding to the target object in the image to be processed, so as to determine the outline of the target object based on the pixel points included in the target object, thereby marking the outline of the target object and displaying it on the front end. The user of the electronic device can make corrections and supplements based on the displayed results, and finally complete the marking of the outline of the target object in the image to be processed and export the marking results.

[0225] In some embodiments, the process of labeling the target object contour can be applied to the initial stage of segmentation model training. Since the trained segmentation model does not have segmentation capabilities at this time, it is necessary to train the segmentation model with labeled data. At this time, if it is time-consuming and laborious to determine the labeled data one by one, the determination of the labeled data based on the data labeling method provided by the present disclosure can greatly improve the labeling speed and the quality of the labeled data. In addition, the process of labeling the target object contour in some embodiments of the present disclosure does not require training any model. In any scenario, it can be labeled with the help of the segmentation results of the model. After testing, it was found that its effect is more than 3 times that of manual labeling, and it can be deployed on any platform. It can obtain results that are better than those obtained by simple manual labeling, which meets the reliability feature of the trustworthy characteristic; it is applicable to the labeling of target object contours in all images, which meets the generalizability feature of the trustworthy characteristic; in any case, it can maintain the characteristic of being better than the effect of simple manual labeling, which meets the robustness feature of the trustworthy characteristic; it can be intervened by other intelligent agents during the labeling process, which meets the controllability feature of the trustworthy characteristic.

[0226] It should be noted that the data annotation method provided by the present disclosure can annotate multiple target objects at once, and the types of different target objects can be the same or different. Figure 19 is a schematic diagram of an annotation result provided by some embodiments of the present disclosure. As shown in Figure 19, the outlines of two vehicles and the outline of a pedestrian are annotated in the image to be processed, and each annotated outline has a corresponding label.

[0227] Figure 20 is a schematic diagram of an application scenario in which a data annotation method provided by some embodiments of the present disclosure is used to annotate the contours of a target object. The data annotation method provided by some embodiments of the present disclosure can be deployed as an application in any electronic device. As shown in Figure 20, the electronic device can be a computer. The user of the electronic device can operate the computer through external devices such as a mouse and a keyboard to issue various image processing instructions. The content displayed on the computer in Figure 20 is a schematic diagram of the front-end page for annotating the contours of the target object.

[0228] Based on the same inventive concept, referring to FIG. 21 , some embodiments of the present disclosure further provide an electronic device, including:

[0229] Memory 111, for storing computer programs or instructions;

[0230] The processor 112 is configured to execute the computer program or instructions in the memory 111 so that any one of the methods in the above embodiments is executed.

[0231] The processor 112 may include one or more central processing units (CPUs) or digital processing units, etc., configured to execute computer programs or instructions in the memory 111 so that any of the methods in the above embodiments is executed.

[0232] It should be noted that in some embodiments, the specific connection medium between the memory 111 and the processor 112 is not limited. In Figure 21 of some embodiments of the present disclosure, the memory 111 and the processor 112 are connected via a bus 113. The connection methods between other components are merely illustrative and not limiting. The bus 113 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 21 only uses a single thick line, but this does not mean that there is only one bus or only one type of bus.

[0233] Based on the same inventive concept, some embodiments of the present disclosure provide a computer-readable storage medium. When instructions in the storage medium are executed by a processor, the processor is enabled to perform any of the methods described in the above embodiments. Because the principles for solving the problem solved by the computer-readable storage medium are similar to those of a data labeling method, the implementation of the computer-readable storage medium can be referred to as the implementation of the method, and any repetitions will not be repeated.

[0234] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0235] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0236] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0237] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more blocks in the block diagram. Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include such modifications and variations.

Claims

1. A data annotation method, comprising: Acquiring a plurality of images associated with the labeling task based on a labeling instruction for the labeling task; Using the multiple detection models associated with the labeling task, perform object detection on at least one of the multiple images to obtain multiple detection results corresponding to the at least one image; wherein each detection result corresponds to a detection model; the detection result includes at least one detection information corresponding to each object detected in the corresponding image; and each detection information corresponds to a confidence level; Filtering candidate detection information corresponding to each object in the at least one image based on the multiple detection results corresponding to the at least one image and a first confidence threshold; Based on the candidate detection information corresponding to each object in the at least one image, a labeling result corresponding to the at least one image is obtained; wherein the labeling result includes the labeling information corresponding to each object in the at least one image.

2. The method according to claim 1, wherein the screening of candidate detection information corresponding to each object in the at least one image based on the multiple detection results corresponding to the at least one image and the first confidence threshold comprises: For each of the multiple test results, perform the following operations: for a test result, determining baseline test information in the test result; wherein the baseline test information is unprocessed test information in the test result; For each benchmark detection information determined, perform the following operations: For a reference detection information, determining matching detection information corresponding to the reference detection information in each other detection result based on the intersection-over-union (IOU) value between the reference detection information and each target detection information in the other detection results; wherein the target detection information is detection information that has the same detection category as the reference detection information and is not matched; Determining a reference confidence of the object corresponding to the one benchmark detection information based on the confidence of the candidate detection information greater than the IOU threshold in the one benchmark detection information and all matching detection information; When it is determined that the reference confidence is greater than the first confidence threshold, each candidate detection information greater than the IOU threshold is respectively determined as candidate detection information of the object corresponding to the one benchmark detection information in the one image.

3. The method according to claim 2, wherein determining, based on an intersection-over-union (IOU) value between the one benchmark detection information and each target detection information in the other detection results, matching detection information corresponding to the one benchmark detection information in each other detection result comprises: Determine an IOU value between the one benchmark detection information and each target detection information in other detection results; The other detection information corresponding to the maximum IOU value in each other detection result is used as the matching detection information; The determining, based on the confidence of the one benchmark detection information and the candidate detection information greater than the IOU threshold in all matching detection information, a reference confidence of the object corresponding to the one benchmark detection information includes: Based on the IOU values of all matching detection information and the one benchmark detection information, and the IOU values between every two matching detection information in all matching detection information, screening out candidate detection information having an IOU greater than a threshold from all matching detection information and the one benchmark detection information; The maximum confidence level among the confidence levels corresponding to the candidate detection information is determined as the reference confidence level of the object corresponding to the one reference detection information.

4. The method according to claim 2, wherein the first confidence threshold comprises a plurality of sub-thresholds; If the number of candidate detection information determined is equal to the total number of the multiple detection models, the multiple sub-thresholds are The first sub-threshold in is used as the first confidence threshold; If the number is less than the total number and greater than a preset value, using a second sub-threshold among the multiple sub-thresholds as the first confidence threshold; If the number is equal to the preset value, using the third sub-threshold among the multiple sub-thresholds as the first confidence threshold; The first sub-threshold is less than or equal to the second sub-threshold, and the second sub-threshold is less than the third sub-threshold.

5. The method according to claim 4, further comprising: If the target detection information does not exist in the other detection results, the confidence level corresponding to the one benchmark detection information is used as the reference confidence level; When it is determined that the reference confidence is greater than the third sub-threshold, the one piece of reference detection information is determined as candidate detection information of an object corresponding to the one piece of reference detection information in the one image.

6. The method according to any one of claims 1 to 5, wherein obtaining a labeling result corresponding to the at least one image based on the candidate detection information corresponding to each object in the at least one image comprises: For each object in the at least one image, perform the following operations: If there are multiple candidate detection information for an object, the multiple candidate detection information are fused to obtain the labeling information of the object; If there is one candidate detection information for an object, the candidate detection information is used as the labeling information of the object; The annotation information corresponding to each object in the image is determined as the annotation result corresponding to the image.

7. The method according to claim 6, wherein the candidate detection information comprises coordinates of a detection box defining the corresponding object in the corresponding image; and fusing the plurality of candidate detection information to obtain the annotation information of the object comprises: Taking the average value of the coordinates of the detection boxes corresponding to the plurality of candidate detection information as the coordinates of the final detection box, and obtaining the labeling information of the object based on the coordinates of the final detection box; or Based on the coordinates of the detection boxes corresponding to the multiple candidate detection information, the coordinates of the minimum bounding rectangle containing the multiple detection boxes are determined, and based on the coordinates of the minimum bounding rectangle, the labeling information of the object is obtained.

8. The method according to claim 6, before fusing the plurality of candidate detection information, further comprising: When it is determined that the number of candidate detection information corresponding to the one object is the same as the number of the multiple detection models, and the confidence corresponding to each piece of candidate detection information is greater than a second confidence threshold, setting the identifier of the one object to a valid value; After obtaining the annotation information of the object and before determining the annotation information corresponding to each object in the image as the annotation result corresponding to the image, the method further includes: When it is determined that the identifier of the object is the valid value, the color of the mark box in the annotation information corresponding to the object is adjusted from the first color to the second color.

9. The method according to claim 6, before determining the annotation information corresponding to each object in the image as the annotation result corresponding to the image, further comprising: For each annotation information, perform the following operations: For one piece of annotation information, respectively determine a CIOU value of the one piece of annotation information and each piece of other annotation information; For each CIOU value, perform the following operations: For a CIOU value, when it is determined that the CIOU value is greater than the CIOU threshold, the annotation information corresponding to the smaller confidence level of two annotation information associated with the CIOU value is deleted.

10. A data annotation method, comprising: Acquire an image to be processed, and determine a target object in the image to be processed; Determining the contour of the target object in the image to be processed based on the large model; The outline of the target object is marked and displayed according to the sparsity of the stored marking points.

11. The method according to claim 10, wherein determining the contour of the target object in the image to be processed based on the target model comprises: Inputting the image to be processed into the target model, so that the target model determines the pixel points of the target object in the image to be processed; The contour of the target object is determined according to the pixel points of the target object.

12. The method according to claim 10, before displaying the marked outline of the target object, further comprising: receiving identification information for identifying the target object; A label of the target object is determined according to the identification information, and the label is added to the target object in the image to be processed.

13. The method according to claim 12, wherein determining the label of the target object according to the identification information comprises: Obtaining other identification information of other objects whose outlines are marked in the image to be processed; If the identification information matches any of the other identification information, the label of the target object is determined according to the label of the object corresponding to the other identification information.

14. The method according to claim 12, further comprising: If a first instruction for modifying the tag is received, the tag is updated using the modification value carried in the first instruction.

15. The method according to claim 10, after marking the outline of the target object according to the sparsity of the stored annotation points, further comprising: Before displaying the outline of the target object, counting the target number of the marking points marking the outline of the target object; A first sparse value stored for a quantity interval in which the target quantity is located is obtained, and annotated points that mark the contour of the target object are adjusted according to the first sparse value.

16. The method according to claim 10, after marking the outline of the target object according to the sparsity of the stored annotation points, further comprising: If the configured contour marking mode is adaptive, then before displaying the contour of the target object, a target area of the detection frame of the target object is determined; A second sparse value saved for the area interval where the target area is located is obtained, and the annotation points that mark the outline of the target object are adjusted according to the second sparse value and a target number of annotation points that mark the outline of the target object.

17. The method according to claim 10, further comprising: If a second instruction for modifying the marked outline is received, the pixel point corresponding to the target position information carried in the second instruction is adjusted to the marked point, and the marked point corresponding to the initial position information carried in the second instruction is deleted.

18. The method according to claim 10, further comprising: If a deletion instruction is received to delete the outline of the target object, the outline of the target object will be marked. Click Delete.

19. An electronic device comprising: Memory, used to store computer programs or instructions; A processor is configured to execute the computer program or instructions in the memory so that the method according to any one of claims 1 to 18 is performed.

Images