Labeling method and device, computer device and storage medium
By setting the locked and unlocked states of the annotation boxes in the image annotation and cropping them according to the overlap relationship, the problem of overlapping annotation boxes affecting the model training accuracy is solved, and the annotation of each region with a unique label is achieved, thus improving the model training quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU WERIDE TECH LTD CO
- Filing Date
- 2022-12-06
- Publication Date
- 2026-07-10
AI Technical Summary
Existing image annotation methods generate training images of low quality, which can easily affect the training accuracy of the model.
During image annotation, the annotation boxes are set to locked and unlocked states. The selection area is cropped based on the overlap between the selected area and the existing annotation boxes to ensure that the annotation boxes do not overlap and to avoid the same area being annotated multiple times.
By avoiding overlapping bounding boxes and ensuring that each region is labeled with at most one type of label, the accuracy of model training is improved.
Smart Images
Figure CN115761053B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data annotation technology, and in particular to an annotation method, apparatus, computer equipment, and storage medium. Background Technology
[0002] In the field of autonomous driving, many artificial intelligence models need to be trained to achieve autonomous driving of vehicles. And in order to train the models, images often need to be labeled.
[0003] Existing methods typically involve identifying the labels to be annotated in an image and then drawing the corresponding annotation shapes in the image to generate bounding boxes for those labels. However, traditional annotation methods often produce training images of low quality, which can negatively impact the accuracy of model training. Summary of the Invention
[0004] The purpose of this application is to address at least one of the aforementioned technical defects, particularly those in the prior art that can adversely affect the accuracy of model training.
[0005] Firstly, this application provides an annotation method, including:
[0006] In response to a selection operation targeting a label, a selection area is defined on the image to be labeled;
[0007] If the selected area does not overlap with an existing label box, set a label box for the target label based on the selected area;
[0008] If the selected area overlaps with an existing annotation box, the existing annotation box that overlaps with the selected area will be identified as the target annotation box.
[0009] For any target annotation box, if the target annotation box is unlocked, the target annotation box is cropped according to the overlap relationship between the target annotation box and the selected area, and the annotation box is set for the target label according to the selected area. If the target annotation box is locked, the selected area is cropped according to the overlap relationship between the target annotation box and the selected area, and the annotation box is set for the target label according to the cropped selected area.
[0010] In one embodiment, the target annotation box is cropped based on the overlap between the target annotation box and the selected area, including:
[0011] If the target label is contained within or equal to the selected area, then delete the target label.
[0012] In one embodiment, the target annotation box is cropped based on the overlap between the target annotation box and the selected area, including:
[0013] If the target annotation box contains a selected area, and the selected area does not overlap with the cutout portion of the target annotation box, then the portion of the filled portion of the target annotation box that overlaps with the selected area is trimmed, forming a new cutout portion of the target annotation box;
[0014] If the target annotation box contains a selected area, and the selected area overlaps with the cutout portion of the target annotation box, then the portion of the filled portion of the target annotation box that overlaps with the selected area is trimmed, so that the cutout portion that overlaps with the selected area is enlarged.
[0015] In one embodiment, the target annotation box is cropped based on the overlap between the target annotation box and the selected area, including:
[0016] If the target annotation box intersects with the selected area, and the selected area and the cutout portion of the target annotation box do not overlap, then the portion of the filled portion of the target annotation box that overlaps with the selected area is trimmed. If the filled portion of the target annotation box is not connected after trimming, a corresponding annotation box is set for each unconnected filled portion, and the target annotation box is deleted.
[0017] If the target annotation box intersects with the selected area, and the selected area overlaps with the cutout portion of the target annotation box, then the overlapping portion of the filled portion of the target annotation box with the selected area is trimmed, and the overlapping portion of the cutout portion of the target annotation box with the selected area is deleted. If the filled portions of the target annotation box are not connected after trimming, corresponding annotation boxes are set for each unconnected filled portion, and the target annotation box is deleted.
[0018] In one embodiment, the selected area is cropped based on the overlap between the target bounding box and the selected area, including:
[0019] If the target annotation box contains or is equal to the selected area, and the selected area does not overlap with the cutout portion of the target annotation box, then the selection operation is ignored;
[0020] If the target annotation box contains or is equivalent to the selected area, and the selected area overlaps with the cutout portion of the target annotation box, then the area where the selected area overlaps with the cutout portion is retained.
[0021] In one embodiment, the selected area is cropped based on the overlap between the target bounding box and the selected area, including:
[0022] If the target label box is included in the selected area, the area in the selected area that overlaps with the filled part of the target label box is cropped, and the non-connected areas in the cropped selected area are divided into multiple sub-candidate areas; wherein, each sub-candidate area is used to set a label box of the same type but with different serial numbers for the target label.
[0023] In one embodiment, the selected area is cropped based on the overlap between the target bounding box and the selected area, including:
[0024] If the target label box intersects with the selected area, the area in the selected area that overlaps with the filled part of the target label box will be cropped, while the area in the selected area that overlaps with the cutout part of the target label box will be retained. If there are non-connected areas in the cropped selected area, the non-connected areas in the cropped selected area will be divided into multiple sub-candidate areas. Each sub-candidate area is used to set a label box of the same type but with different serial numbers for the target label.
[0025] Secondly, embodiments of this application provide a labeling device, comprising:
[0026] The bounding box module is used to define a bounding box area on the image to be labeled in response to a bounding box operation for a target label.
[0027] The first annotation module is used to set annotation boxes for target labels based on the selected area, provided that the selected area does not overlap with existing annotation boxes.
[0028] The target annotation box determination module is used to determine the overlapping annotation box in the existing annotation box as the target annotation box when the selected area overlaps with the selected area.
[0029] The second annotation module, for any target annotation box, if the target annotation box is in an unlocked state, then according to the overlap relationship between the target annotation box and the selected area, it clips the target annotation box and sets an annotation box for the target label according to the selected area; if the target annotation box is in a locked state, then according to the overlap relationship between the target annotation box and the selected area, it clips the selected area and sets an annotation box for the target label according to the clipped selected area.
[0030] Thirdly, embodiments of this application provide a computer device, including one or more processors and a memory, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, they perform the steps of the annotation method in any of the above embodiments.
[0031] Fourthly, embodiments of this application provide a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the annotation method in any of the above embodiments.
[0032] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:
[0033] Based on any of the above embodiments, existing bounding boxes are set to two states: locked and unlocked. When an annotator selects a region on the image to be annotated, based on the principle that the generated bounding box cannot overlap with existing bounding boxes, if the selected region overlaps with a locked bounding box, the locked bounding box remains unchanged, and the candidate region is cropped. If the selected region overlaps with an unlocked bounding box, the candidate region remains unchanged, and the unlocked bounding box is cropped. This method ensures that there is no overlap between all bounding boxes, and each region on the image to be annotated can be labeled with at most one type of label, avoiding the problem of overlapping bounding boxes affecting the model training accuracy. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0035] Figure 1 A flowchart illustrating an annotation method provided in one embodiment of this application;
[0036] Figure 2 This is a schematic diagram of the annotation effect when the candidate region contains or is equivalent to an unlocked annotation box in one embodiment of this application;
[0037] Figure 3 This is a schematic diagram of a cutout portion and a filled portion in one embodiment of this application;
[0038] Figure 4 This is a schematic diagram illustrating the annotation effect when a candidate region is contained within an unlocked annotation box in one embodiment of this application.
[0039] Figure 5 This is a schematic diagram illustrating the annotation effect when the candidate region is contained within an unlocked annotation box in another embodiment of this application.
[0040] Figure 6This is a schematic diagram illustrating the annotation effect when a candidate region intersects with an unlocked annotation box in one embodiment of this application;
[0041] Figure 7 This is a schematic diagram illustrating the annotation effect when a candidate region intersects with an unlocked annotation box in another embodiment of this application;
[0042] Figure 8 This is a schematic diagram illustrating the annotation effect when the candidate region intersects with the unlocked annotation box in another embodiment of this application;
[0043] Figure 9 This is a schematic diagram illustrating the annotation effect when a candidate region intersects with an unlocked annotation box in another embodiment of this application;
[0044] Figure 10 This is a schematic diagram of the annotation effect when the locked annotation box contains or is equivalent to the candidate region in one embodiment of this application;
[0045] Figure 11 This is a schematic diagram of the annotation effect when the locked annotation box contains or is equivalent to the candidate region in another embodiment of this application;
[0046] Figure 12 This is a schematic diagram of the annotation effect when the locked annotation box is contained within the candidate region in one embodiment of this application;
[0047] Figure 13 This is a schematic diagram of the annotation effect when the locked annotation box is contained within the candidate area in another embodiment of this application;
[0048] Figure 14 This is a schematic diagram illustrating the annotation effect when the locked annotation box intersects with the candidate region in one embodiment of this application;
[0049] Figure 15 This is a schematic diagram of the annotation effect when the locked annotation box intersects with the candidate area in another embodiment of this application. Detailed Implementation
[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0051] Firstly, this application provides an annotation method, please refer to [link to relevant documentation]. Figure 1 This includes steps S102 to S108.
[0052] S102, in response to a selection operation for a target label, defines a selection area on the image to be labeled.
[0053] It's understandable that the target label is one of several candidate labels. In the field of autonomous driving, candidate labels might include ground, traffic lights, pedestrians, animals, static objects, and dynamic objects. Each annotation box under the same candidate label corresponds to a different number for easy differentiation. For example, the number can increase sequentially based on the order of annotation. For an image of pedestrian A and pedestrian B, if the candidate label "pedestrian" is selected, and pedestrian A is annotated, the label number for pedestrian A is "pedestrian 10". The number for pedestrian B is "pedestrian 11". When an annotator needs to annotate an object in the image, they select one from the label list displayed on the annotation interface. The annotator can then draw the area to be annotated in the image using a box selection operation, and the computer can determine the selected area.
[0054] Specifically, the drawing operation can include polygon drawing, allowing annotators to arbitrarily adjust the shape of the selected area. It can also draw specific shapes, such as rectangular or triangular selections. Alternatively, a contour recognition algorithm can be configured in the annotation platform. When annotators click on any object in the image, an initial selection area for that object is automatically generated. Annotators can then adjust this initial selection area to obtain the final selection.
[0055] S104, determine whether the selected area overlaps with an existing annotation box.
[0056] It is understandable that the selected area is the basis for the generated annotation box. In order to avoid the situation where the same object is labeled with different labels due to the overlap between annotation boxes, it is necessary to first determine whether the annotation box to be generated will overlap with the existing annotation box. If yes, proceed to step S106; if no, proceed to step S108.
[0057] S106, if the selected area does not overlap with an existing label box, set a label box for the target label based on the selected area.
[0058] It is understandable that if the selected area does not overlap with any existing label box, a label box under the target label can be generated directly based on the selected area, and this label box will not interfere with other label boxes.
[0059] S108, if the selected area overlaps with an existing annotation box, the annotation box in the existing annotation box that overlaps with the selected area is determined as the target annotation box.
[0060] It's understandable that if the selected area overlaps with existing annotation boxes, further analysis is needed. Therefore, all annotation boxes that overlap with the selected area are identified as target annotation boxes.
[0061] S110, for any target annotation box, if the target annotation box is unlocked, the target annotation box is cropped according to the overlap relationship between the target annotation box and the selected area, and an annotation box is set for the target label according to the selected area. If the target annotation box is locked, the selected area is cropped according to the overlap relationship between the target annotation box and the selected area, and an annotation box is set for the target label according to the cropped selected area.
[0062] Understandably, to facilitate annotation, annotators want some annotation boxes to remain unchanged once confirmed, while others can be freely modified. To meet this need, the annotation platform supports changing the state of annotation boxes, allowing existing boxes to be set to a locked or unlocked state. Locked annotation boxes cannot be cropped, while unlocked boxes can be cropped. Specifically, a locking control can be set for each annotation box under each selectable label in the label list displayed on the annotation interface. Annotators can switch the corresponding annotation box between locked and unlocked states by manipulating the locking control.
[0063] Therefore, when the target annotation box is locked, it cannot be modified. Thus, the selected area is cropped based on the overlap between the target and candidate areas to prevent overlap. When the target annotation box is unlocked, the later-drawn annotation box has higher priority than existing ones, and annotation boxes can be generated based on the candidate areas drawn by the annotator. Simultaneously, to prevent the target annotation box from interfering with the generated annotation box, it is cropped based on the overlap between the target and candidate areas to avoid overlap.
[0064] Based on the annotation method in this embodiment, existing bounding boxes are set to two states: locked and unlocked. When an annotator selects a region on the image to be annotated, based on the principle that the generated bounding box cannot overlap with existing bounding boxes, if the selected region overlaps with a locked bounding box, the locked bounding box remains unchanged, and the candidate region is cropped. If the selected region overlaps with an unlocked bounding box, the candidate region remains unchanged, and the unlocked bounding box is cropped. This method ensures that there is no overlap between all bounding boxes, and each region on the image to be annotated can be labeled with at most one type of label, avoiding the problem of overlapping bounding boxes affecting the model training accuracy.
[0065] In one embodiment, the overlap relationship includes the target annotation box being contained within or equivalent to the selected area, that is, the target annotation box is completely inside the selected area or is exactly the same as the selected area. Under this overlap relationship, the target annotation box is cropped according to the overlap relationship between the target annotation box and the selected area, including: if the target annotation box is contained within or equivalent to the selected area, then the target annotation box is deleted.
[0066] It's understandable that when a target bounding box is inside or the same size as the selected area, since it's unlocked, later-generated bounding boxes have higher priority. The image originally labeled with the target bounding box should be replaced by the label of the later-generated bounding box. Therefore, while setting bounding boxes for target labels based on candidate areas, the original target bounding box is directly deleted. For example... Figure 2 As shown, dashed lines represent candidate areas, solid lines represent target annotation boxes, and pattern-filled areas indicate the range where corresponding annotation boxes are marked. The fill pattern for the "Ground" label is a diagonal line, and the fill pattern for the "Vehicle" label is a horizontal line. The target label type is "Vehicle," and its sequence number is "Vehicle 7." The sequence number of the target annotation box corresponding to "Vehicle 7" is "Ground 11." If the target annotation box "Ground 11" is contained within the candidate area, then the annotation box "Vehicle 7" is set according to the candidate area, and the target annotation box "Ground 11" is deleted.
[0067] In one embodiment, the overlap relationship includes the target bounding box containing the candidate region, meaning the candidate region is completely inside the target bounding box. Under this overlap relationship, the target bounding box is cropped based on the overlap between the target bounding box and the selected region, including:
[0068] (1) If the target annotation box contains a selected area and the selected area does not overlap with the cutout part of the target annotation box, then the part of the filled part of the target annotation box that overlaps with the selected area is cut off, and a new cutout part of the target annotation box is formed.
[0069] It can be understood that the target annotation box consists of a filled portion and a cut-out portion. The filled portion represents the image portion to be annotated that is inside the annotation box and has the corresponding label, while the cut-out portion represents the image portion to be annotated that is inside the annotation box but has not been labeled. For example... Figure 3As shown, the outline of the "Ground 12" label includes both an outer and inner outline. The pattern-filled area between the inner and outer outlines is the filled area, and the white area inside the inner outline is the cutout area. The image data of the filled areas are all labeled "Ground 12," while the cutout areas are not. The cutout area is formed when the label box is in an unlocked state. Specifically, when the selected area is inside the target label box, since the target label box is unlocked, the later-generated label box has higher priority, and the image originally labeled with the target label box should be replaced by the label of the later-generated label box. The part labeled with the target label box is the filled area. Therefore, the portion of the filled area of the target label box that overlaps with the selected area is cropped. Since the selected area does not overlap with other cutout areas, the cropped portion will form a new cutout area. Figure 4 As shown, the fill pattern corresponding to the "Pedestrian" label is a grid. The target label type is pedestrian, and its serial number is "Pedestrian 13". The label serial number of the target annotation box corresponding to "Pedestrian 13" is "Ground 12". The candidate area is inside the target annotation box "Ground 12", and the target annotation box "Ground 12" has no cutout portion. Therefore, the candidate area does not overlap with the cutout portion of the target annotation box "Ground 12". Thus, the annotation box is set according to the candidate area. The portion of "Ground 12" that overlaps with "Pedestrian 13" is cropped. The cropped "Ground 12" annotation box is shown below. Figure 3 As shown. It's worth noting that in practical applications, different types of labels are generally distinguished by different colors, rather than using pattern fills. Furthermore, a hidden control can be set for each annotation box under each selectable label in the label list displayed on the annotation interface. By operating the hidden control, the outline and fill of the corresponding annotation box can be hidden, making it easier for the annotator to observe. Figure 4 The "Ground 12" on the right is hidden, and you can obtain it. Figure 3 The display effect in the video.
[0070] (2) If the target annotation box contains a selected area and the selected area overlaps with the cutout portion of the target annotation box, then the part of the filled portion of the target annotation box that overlaps with the selected area is trimmed so that the cutout portion that overlaps with the selected area is enlarged.
[0071] Similarly, when the selected area is inside the target annotation box, since the target annotation box is unlocked, the later-generated annotation box has higher priority. The image originally labeled with the target annotation box should be replaced by the label of the later-generated annotation box. The portion labeled with the target annotation box is the filled area; therefore, the part of the filled area of the target annotation box that overlaps with the selected area is cropped. Because the selected area overlaps with other cutout areas, the cropped area will enlarge the cutout areas that overlap with the selected area. Figure 4 Based on this, perform a selection operation, such as... Figure 5 As shown, the target label is "pedestrian," and its sequence number is "pedestrian 14." The label sequence number of the target annotation box corresponding to "pedestrian 14" is "ground 12." The candidate area is located inside the target annotation box "ground 14" and overlaps with the cutout portion of the target annotation box "ground 12" caused by "pedestrian 13." Therefore, by setting the annotation box according to the candidate area, the filled portion of "ground 12" that overlaps with "pedestrian 14" is clipped. By operating the hidden control and only displaying "ground 12," it can be seen that the cutout portion caused by "pedestrian 13" is enlarged. In addition, this only describes the relationship between "pedestrian 14" and "ground 12." However, "pedestrian 14" also overlaps with "pedestrian 13" at this time. Therefore, the methods in the various embodiments of this application can be used to clip "pedestrian 14" or "pedestrian 13."
[0072] In one embodiment, the overlap relationship includes the intersection of the target bounding box and the candidate region, meaning that the target bounding box and the candidate region partially overlap. Under this overlap relationship, the target bounding box is cropped based on the overlap relationship between the target bounding box and the selected region, including:
[0073] (1) If the target annotation box intersects with the selected area and the selected area does not overlap with the cutout part of the target annotation box, then the part of the filled part of the target annotation box that overlaps with the selected area is cut off. If the filled part of the target annotation box is not connected after the cut-off, a corresponding annotation box is set for each unconnected filled part, and the target annotation box is deleted.
[0074] It's understandable that when the target annotation box intersects with the selected area, since the target annotation box is unlocked, the later-generated annotation box has higher priority. The image originally labeled with the target annotation box should be replaced by the label of the later-generated annotation box. The portion labeled with the target annotation box is the filled area; therefore, the part of the filled area of the target annotation box that overlaps with the selected area is cropped.
[0075] However, after cropping, there are two scenarios: one is that the filled portions of the target annotation boxes remain connected after cropping, meaning the selected area does not divide the target annotation boxes. Figure 5 For example, if "Pedestrian 13" is not locked, please refer to [link / reference]. Figure 6 (The outlines of "Pedestrian 13" and "Pedestrian 14" on the right should partially overlap, but since these two label boxes belong to the same category, for the sake of showing the effect after cropping, Figure 6A certain interval is set between the two. The candidate region intersects with the target annotation box "Pedestrian 13", and there is no cutout portion in the target annotation box "Pedestrian 13". Therefore, the portion of the filled part of the target annotation box "Pedestrian 13" that overlaps with the candidate region is cropped. This type of cropping does not create new cutout portions.
[0076] Another scenario is that the filled portion of the target annotation box after cropping exhibits discontinuous sections, meaning the selected area divides the target annotation box into multiple independent parts. Figure 2 For example, if "Ground 12" is not locked, please refer to [link / reference]. Figure 7 (The "Ground 13" and "Ground 14" on the right should coincide with the outline of pole 4, but since these two annotation boxes belong to the same type, for the sake of showing the effect after cropping, Figure 7 (A certain interval is set in between), the type of target label is "rod", the label number of the target label is "rod 4", the candidate area intersects with the target annotation box "ground 12", and there is no hollow part in the target annotation box "ground 12". Therefore, the part of the filled part of the target annotation box "ground 12" that overlaps with the candidate area is trimmed. However, after trimming, the filled part of "ground 12" is divided into two unconnected parts. Then, a corresponding annotation box will be set for each unconnected filled part, that is, a new label number will be reassigned to each unconnected filled part, such as... Figure 7 The target annotation boxes corresponding to "Ground 13" and "Ground 14" in the original text should be deleted, and the target annotation boxes corresponding to "Ground 12" should be replaced by the two new annotation boxes "Ground 13" and "Ground 14".
[0077] (2) If the target annotation box intersects with the selected area and the selected area overlaps with the cutout portion of the target annotation box, then the part of the filled portion of the target annotation box that overlaps with the selected area is trimmed, and the part of the cutout portion of the target annotation box that overlaps with the selected area is deleted. In the case that the filled portion of the target annotation box is not connected after trimming, a corresponding annotation box is set for each unconnected filled portion, and the target annotation box is deleted.
[0078] It's understandable that when the target annotation box intersects with the selected area, since the target annotation box is unlocked, the later-generated annotation box has higher priority. The image originally labeled with the target annotation box should be replaced by the label of the later-generated annotation box. The portion labeled with the target annotation box is the filled portion; therefore, the part of the filled portion of the target annotation box that overlaps with the selected area is cropped. Furthermore, the cutout portion was originally defined by the inner contour of the target annotation box, but after cropping, the inner contour is opened up and becomes the inner contour itself. Therefore, the cutout portion overlapping with the selected area needs to be removed from the components of the target annotation box.
[0079] After cropping, there are also two scenarios: one is that the filled portions of the target annotation boxes remain connected after cropping. For example... Figure 8 As shown, the target label's label number is "Pole 5". The candidate area intersects with the target annotation box "Ground 12". Therefore, the portion of the filled area of the target annotation box "Ground 12" that overlaps with "Pole 5" is cropped, and the hollowed-out portion of "Pole 5" overlaps with the hollowed-out portion of "Ground 12". After cropping, the inner contour corresponding to the hollowed-out portion of "Ground 12" becomes the outer contour, and thus the hollowed-out portion no longer exists; that is, it is deleted from the components of "Ground 12".
[0080] Another scenario is that the filled portion of the target annotation box becomes discontinuous after cropping. For example... Figure 9 As shown, the target label's label number is "Pole 6". The candidate area intersects with the target annotation box "Ground 12". Therefore, the portion of the filled area of the target annotation box "Ground 12" that overlaps with "Pole 6" is trimmed. After trimming, the filled area of "Ground 12" includes two unconnected filled areas. Corresponding annotation boxes are set for these two unconnected filled areas, and the original annotation box corresponding to "Ground 12" is deleted. The hollowed-out portions of "Pole 6" and "Ground 12" overlap. After trimming, the inner contours corresponding to the overlapping hollowed-out portions become part of the outer contours of "Ground 15" and "Ground 16", respectively, thus the hollowed-out portion no longer exists.
[0081] In one embodiment, the selected area is cropped based on the overlap between the target bounding box and the selected area, including:
[0082] (1) If the target annotation box contains or is equivalent to the selected area, and the selected area does not overlap with the cutout part of the target annotation box, then the selection operation is ignored.
[0083] It's understandable that when a target annotation box is locked, its fill portion cannot be changed. If the selected area is completely within or equal to the target annotation box, and does not overlap with the cutout portion of the target annotation box, then it's equivalent to the selected area only overlapping the fill portion, and therefore no effect will occur. For example... Figure 10 As shown, after obtaining Figure 3 When the target area is completely within the target area, the selection operation will be ignored if the "Ground 12" is locked by using the locking control.
[0084] (2) If the target annotation box contains or is equivalent to the selected area, and the selected area overlaps with the cutout portion of the target annotation box, then the area where the selected area overlaps with the cutout portion is retained.
[0085] It is understandable that the filled portion of a locked target annotation box cannot be changed, but the cut-out portion can be labeled with other label numbers. If the selected area is completely within or equal to the target annotation box, and overlaps with the cut-out portion of the target annotation box, the overlapping area between the selected area and the filled portion should be cropped, but the area overlapping with the cut-out portion can be retained. Figure 11 As shown, after obtaining Figure 3 When the label box shown is set, the "Ground 12" is set to the locked state by operating the locking control. The candidate area is completely inside the target label box. The area that overlaps with the filling part of "Ground 12" should be clipped, but the area that overlaps with the cut-out part is retained. Finally, the label box of "Pedestrian 16" is set according to the retained candidate area.
[0086] In one embodiment, the selected area is cropped based on the overlap between the target label box and the selected area. This includes: if the target label box is included in the selected area, cropping the area in the selected area that overlaps with the filled portion of the target label box, retaining the area in the selected area that overlaps with the cutout portion of the target label box, and dividing the non-connected areas in the cropped selected area into multiple sub-candidate areas. Each sub-candidate area is used to set a label box of the same type but with different serial numbers for the target label.
[0087] It is understandable that if the selected area completely contains the target annotation box, the fill portion of the target annotation box, which is in a locked state, cannot be changed. Therefore, the area in the candidate area that overlaps with the fill portion of the target annotation box will be cropped.
[0088] However, the hollowed-out portion of the target annotation box can be labeled with other label numbers. If the target annotation box does not have a hollowed-out portion, the cropped portion of the candidate region forms the hollowed-out portion of the newly generated annotation box, and the retained portion forms the filled portion of the newly generated annotation box. For example... Figure 12 As shown, when generating the selection area "Vehicle 7", if "Ground 12" is locked, the area in the selection area "Vehicle 7" that overlaps with the filling part of "Ground 12" will be cropped, and finally the cropped candidate area will be used to set the label box "Vehicle 7".
[0089] If the target label box has a cutout portion, the areas of the candidate region that do not overlap with the target label box, and the areas of the candidate region that overlap with the cutout portion of the target label box, will not be connected. Therefore, they will be divided into independent sub-candidate regions, and each sub-candidate region will be set with a label box of the same type as the target label but with a different sequence number. For example... Figure 13 As shown, Figure 2 Replace "Ground 12" with Figure 3In the target annotation box "Ground 12", there is a hollowed-out portion. The areas of the candidate region that do not overlap with the target annotation box "Ground 12" are not connected to the areas of the candidate region that overlap with the hollowed-out portion of the target annotation box "Ground 12". Therefore, the candidate region should be divided into two sub-candidate regions. Finally, two annotation boxes with different label numbers, "Vehicle 7" and "Vehicle 8", are set for each sub-candidate region.
[0090] In one embodiment, the selected area is cropped based on the overlap between the target annotation box and the selected area. This includes: if the target annotation box intersects with the selected area, cropping the area in the selected area that overlaps with the filled portion of the target annotation box, and retaining the area in the selected area that overlaps with the cutout portion of the target annotation box; and if there are non-connected areas in the cropped selected area, dividing the non-connected areas in the cropped selected area into multiple sub-candidate areas. Each sub-candidate area is used to set annotation boxes of the same type but with different serial numbers for the target label.
[0091] It's understandable that if the selected area partially overlaps with the target annotation box, the fill portion of the locked target annotation box cannot be changed. Therefore, the area in the candidate area that overlaps with the fill portion of the target annotation box will be cropped. However, the cutout portion of the target annotation box can be labeled with other label numbers; therefore, the area in the selected area that overlaps with the cutout portion of the target annotation box will be retained.
[0092] After cropping, the candidate region may not contain disconnected areas, such as... Figure 14 As shown, in performing similar Figure 6 During the drawing process, if the target annotation box "Pedestrian 13" is locked, the candidate area should be cropped.
[0093] After cropping, the candidate regions may contain disconnected areas, such as... Figure 15 As shown, in performing similar Figure 9 During the drawing process, if the target annotation box "Ground 12" is locked, the candidate area should be clipped. The area where the candidate area overlaps with the filled part of the target annotation box "Ground 12" will be clipped, while the area that overlaps with the cut-out part will be retained. The candidate area is then divided into three non-connected sub-candidate areas, and each sub-candidate area is assigned a different but the same type of label number.
[0094] This application provides an annotation device, including:
[0095] The bounding box module is used to define a bounding box area on the image to be labeled in response to a bounding box operation for a target label.
[0096] The first annotation module is used to set annotation boxes for target labels based on the selected area, provided that the selected area does not overlap with any existing annotation boxes.
[0097] The target annotation box determination module is used to determine the overlapping annotation box in the existing annotation box as the target annotation box when the selected area overlaps with the selected area.
[0098] The second annotation module, for any target annotation box, if the target annotation box is in an unlocked state, then according to the overlap relationship between the target annotation box and the selected area, it clips the target annotation box and sets an annotation box for the target label according to the selected area; if the target annotation box is in a locked state, then according to the overlap relationship between the target annotation box and the selected area, it clips the selected area and sets an annotation box for the target label according to the clipped selected area.
[0099] In one embodiment, the second annotation module includes a first unlocked annotation unit. The first unlocked annotation unit is used to delete the target annotation box if the target annotation box is contained within or equivalent to the selected area.
[0100] In one embodiment, the second annotation module includes a second unlocked annotation unit. The second unlocked annotation unit is used to trim the portion of the filled portion of the target annotation frame that overlaps with the selected area if the target annotation frame contains a selected area and the selected area does not overlap with the cutout portion of the target annotation frame, thereby forming a new cutout portion of the target annotation frame;
[0101] If the target annotation box contains a selected area, and the selected area overlaps with the cutout portion of the target annotation box, then the portion of the filled portion of the target annotation box that overlaps with the selected area is trimmed, so that the cutout portion that overlaps with the selected area is enlarged.
[0102] In one embodiment, the second annotation module includes a third non-locking annotation unit. The third non-locking annotation unit is configured to: if the target annotation box intersects with the selected area and the selected area and the cutout portion of the target annotation box do not overlap, then trim the portion of the filled portion of the target annotation box that overlaps with the selected area; if the filled portion of the target annotation box is not connected after trimming, set corresponding annotation boxes for each disconnected filled portion, and delete the target annotation box; if the target annotation box intersects with the selected area and the selected area and the cutout portion of the target annotation box overlap, then trim the portion of the filled portion of the target annotation box that overlaps with the selected area; delete the portion of the cutout portion of the target annotation box that overlaps with the selected area; and if the filled portion of the target annotation box is not connected after trimming, set corresponding annotation boxes for each disconnected filled portion, and delete the target annotation box.
[0103] In one embodiment, the second annotation module includes a first locking annotation unit. The first locking annotation unit is used to ignore the selection operation if the target annotation box contains or is equivalent to the selected area and the selected area does not overlap with the cutout portion of the target annotation box; and to retain the area where the selected area overlaps with the cutout portion if the target annotation box contains or is equivalent to the selected area and the selected area overlaps with the cutout portion of the target annotation box.
[0104] In one embodiment, the second annotation module includes a second locking annotation unit. The second locking annotation unit is used to, if the target annotation box is included in the selected area, crop the area in the selected area that overlaps with the filled portion of the target annotation box, and divide the non-connected areas in the cropped selected area into multiple sub-candidate areas; wherein each sub-candidate area is used to set annotation boxes of the same type but with different serial numbers for the target label.
[0105] In one embodiment, the second annotation module includes a third locking annotation unit. The third locking annotation unit is used to, if the target annotation box intersects with the selected area, crop the area in the selected area that overlaps with the filled portion of the target annotation box, and retain the area in the selected area that overlaps with the cutout portion of the target annotation box; and if there are non-connected areas in the cropped selected area, divide the non-connected areas in the cropped selected area into multiple sub-candidate areas; each sub-candidate area is used to set annotation boxes of the same type but with different serial numbers for the target label.
[0106] Specific limitations regarding the annotation device can be found in the limitations of the annotation method above, and will not be repeated here. Each module in the aforementioned annotation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module. It should be noted that the module division in this embodiment is illustrative and only represents a logical functional division; other division methods may be used in actual implementation.
[0107] This application provides a computer device including one or more processors and a memory storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, they perform the steps of the annotation method in any of the above embodiments.
[0108] This application provides a storage medium storing computer-readable instructions. When executed by one or more processors, these computer-readable instructions cause the one or more processors to perform the steps of the annotation method in any of the above embodiments.
[0109] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0110] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.
[0111] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A labeling method, characterized in that, include: In response to a selection operation targeting a label, a selection area is defined on the image to be labeled; If the selected area does not overlap with an existing label box, set the label box for the target label based on the selected area; If the selected area overlaps with an existing annotation box, the existing annotation box that overlaps with the selected area is identified as the target annotation box. The target annotation box is composed of a filled part and a cut-out part; the filled part represents the part of the image to be annotated that is inside the target annotation box and has the label corresponding to the target annotation box, and the cut-out part represents the part of the image to be annotated that is inside the target annotation box but has not been labeled with the label corresponding to the target annotation box; For any of the target annotation boxes, if the target annotation box is unlocked, then according to the overlap relationship between the target annotation box and the selected area, the portion of the filled portion that overlaps with the selected area is cropped, and the annotation box is set for the target label according to the selected area. If the target annotation box is locked, then according to the overlap relationship between the target annotation box and the selected area, the area of the selected area that overlaps with the filled portion is cropped, while the area of the selected area that overlaps with the cutout portion is retained, and the annotation box is set for the target label according to the cropped selected area.
2. The annotation method according to claim 1, characterized in that, The step of cropping the target annotation box based on the overlap between the target annotation box and the selected area includes: If the target annotation box is contained within or equivalent to the selected area, then the target annotation box is deleted.
3. The annotation method according to claim 1, characterized in that, The step of cropping the target annotation box based on the overlap between the target annotation box and the selected area includes: If the target annotation box includes the selected area, and the selected area does not overlap with the cutout portion of the target annotation box, then the portion of the filled portion of the target annotation box that overlaps with the selected area is trimmed to form a new cutout portion of the target annotation box; If the target annotation box includes the selected area, and the selected area overlaps with the cutout portion of the target annotation box, then the portion of the filled portion of the target annotation box that overlaps with the selected area is trimmed, so that the cutout portion that overlaps with the selected area is enlarged.
4. The annotation method according to claim 1, characterized in that, The step of cropping the target annotation box based on the overlap between the target annotation box and the selected area includes: If the target annotation box intersects with the selected area, and the selected area and the cutout portion of the target annotation box do not overlap, then the portion of the filled portion of the target annotation box that overlaps with the selected area is trimmed. If the filled portion of the target annotation box is not connected after trimming, a corresponding annotation box is set for each unconnected filled portion, and the target annotation box is deleted. If the target annotation box intersects with the selected area, and the selected area overlaps with the cutout portion of the target annotation box, then the portion of the filled portion of the target annotation box that overlaps with the selected area is trimmed, and the portion of the cutout portion of the target annotation box that overlaps with the selected area is deleted. Furthermore, if the filled portions of the target annotation box are not connected after trimming, corresponding annotation boxes are set for each unconnected filled portion, and the target annotation box is deleted.
5. The annotation method according to claim 1, characterized in that, The step of cropping the selected region based on the overlap between the target bounding box and the selected region includes: If the target annotation box contains or is equivalent to the selected area, and the selected area does not overlap with the cutout portion of the target annotation box, then the selection operation is ignored. If the target annotation box contains or is equivalent to the selected area, and the selected area overlaps with the cutout portion of the target annotation box, then the area where the selected area overlaps with the cutout portion is retained.
6. The annotation method according to claim 1, characterized in that, The step of cropping the selected region based on the overlap between the target bounding box and the selected region includes: If the target label box is included in the selected area, the area in the selected area that overlaps with the filled portion of the target label box is cropped, and the non-connected areas in the cropped selected area are divided into multiple sub-candidate areas; wherein, each sub-candidate area is used to set a label box of the same type but with different serial numbers for the target label.
7. The annotation method according to claim 1, characterized in that, The step of cropping the selected region based on the overlap between the target bounding box and the selected region includes: If the target label box intersects with the selected area, the area in the selected area that overlaps with the filled portion of the target label box is cropped, while the area in the selected area that overlaps with the cutout portion of the target label box is retained. If there are non-connected areas in the selected area after cropping, the non-connected areas in the selected area after cropping are divided into multiple sub-candidate areas. Each sub-candidate area is used to set a label box of the same type but with different serial numbers for the target label.
8. A labeling device, characterized in that, include: The bounding box module is used to define a bounding box area on the image to be labeled in response to a bounding box operation for a target label. The first annotation module is used to set the annotation box for the target label according to the selected area when the selected area does not overlap with an existing annotation box. The target annotation box determination module is used to determine the existing annotation box that overlaps with the selected area as the target annotation box when the selected area overlaps with the existing annotation box. The target annotation box is composed of a filled part and a cut-out part; the filled part represents the part of the image to be annotated that is inside the target annotation box and has the label corresponding to the target annotation box, and the cut-out part represents the part of the image to be annotated that is inside the target annotation box but has not been labeled with the label corresponding to the target annotation box; The second annotation module, for any one of the target annotation boxes, is configured to: if the target annotation box is unlocked, trim the portion of the filled portion that overlaps with the selected area based on the overlap relationship between the target annotation box and the selected area, and set the annotation box for the target label based on the selected area; if the target annotation box is locked, trim the area of the selected area that overlaps with the filled portion based on the overlap relationship between the target annotation box and the selected area, retain the area of the selected area that overlaps with the cut-out portion, and set the annotation box for the target label based on the trimmed selected area.
9. A computer device, characterized in that, It includes one or more processors and a memory storing computer-readable instructions that, when executed by the one or more processors, perform the steps of the annotation method as described in any one of claims 1-7.
10. A storage medium, characterized in that, The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the annotation method as described in any one of claims 1-7.