Automatic labeling method, device and equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence and the technical field of data annotation, and discloses an automatic annotation method, device and equipment and a storage medium, and the method comprises the steps: obtaining a current object image, and obtaining a candidate region outputted by a candidate box generation network based on the current object image; performing feature extraction on the region image corresponding to the candidate region to obtain a current object feature; generating category similarity corresponding to each preset object category; selecting a preset object category with the maximum category similarity as a current object category corresponding to the current object feature; the method comprises the steps of obtaining a polygonal envelope corresponding to an object in a current object image, combining information of the polygonal envelope with a current object category, generating annotation information of the current object image, adding the annotation information to the current object image, and generating an annotated current object image. According to the invention, the marking efficiency of the current object image is improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence technology and data annotation technology, and in particular to an automatic annotation method, device, equipment and storage medium. Background Art

[0002] Computer vision is an important branch of artificial intelligence technology, which enables computers to interpret and understand visual information from images or multidimensional data, similar to human visual perception. Data annotation is a key link in computer vision. By adding annotation information to object images, computer systems can significantly improve their accuracy in identifying objects. This improvement not only directly enhances the accuracy of image recognition, but also plays a huge role in commercial applications.

[0003] However, the current object image annotation process is cumbersome, which is not conducive to improving the annotation efficiency of the current object image. The reason is that the existing technology mainly adopts the manual annotation method to manually identify and distinguish the objects in the current object image, and then manually mark the box on the current object image to obtain the annotation in the form of a box. However, there are many object images at present, and the manual annotation method will consume a lot of human resources and time resources, which increases the annotation time of the current object image. Therefore, it is not conducive to improving the annotation efficiency of the current object image. Summary of the invention

[0004] The present invention provides an automatic labeling method, device, computer equipment and storage medium to solve the technical problem that the current object image labeling process is cumbersome and is not conducive to improving the labeling efficiency of the current object image.

[0005] In a first aspect, an automatic annotation method is provided, comprising: Using a preset object feature extraction network, feature extraction is performed on preset object images corresponding to each preset object category to obtain preset object features corresponding to each preset object category; Acquire a current object image, and acquire a candidate region output by the candidate frame generation network based on the current object image; Extracting features from the region image corresponding to the candidate region to obtain features of the current object; Adding the feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories to obtain the sum of the similarities corresponding to each of the preset object categories, and dividing the sum of the similarities corresponding to each of the preset object categories by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories; Selecting the preset object category with the greatest category similarity as the current object category corresponding to the current object feature; Obtain a polygonal envelope corresponding to an object in the current object image, combine information of the polygonal envelope with the current object category, generate annotation information of the current object image, add the annotation information to the current object image, and generate an annotated current object image.

[0006] Furthermore, the obtaining of the current object image and the obtaining of the candidate region output by the candidate frame generation network based on the current object image include: Acquire a current object image, and input a feature vector of the current object image into a preset candidate box generation network; Obtain a candidate area output by the candidate box generation network based on the feature vector of the current object image.

[0007] Furthermore, the extracting features of the region image corresponding to the candidate region to obtain the current object features includes: By using the object feature extraction network, feature extraction is performed on the regional image corresponding to the candidate region to obtain image features of the regional image; The image features of the region image are selected as the current object features.

[0008] Further, the feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories are added to obtain the sum of the similarities corresponding to each of the preset object categories, and the sum of the similarities corresponding to each of the preset object categories is divided by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories, including: Selecting the number of preset object features corresponding to the preset object category as the number of features corresponding to the preset object category; The feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories are added to obtain the sum of the similarities corresponding to each of the preset object categories, and the sum of the similarities corresponding to each of the preset object categories is divided by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories.

[0009] Furthermore, the selecting the preset object category with the greatest category similarity as the current object category corresponding to the current object feature includes: Obtaining a preset similarity, and determining whether the category similarity is greater than the preset similarity; When the category similarity is greater than the preset similarity, the preset object category with the greatest category similarity is selected as the current object category corresponding to the current object feature.

[0010] Further, the obtaining a polygon envelope corresponding to the object in the current object image, combining information of the polygon envelope with the current object category to generate annotation information of the current object image, adding the annotation information to the current object image to generate the annotated current object image includes: Inputting the current object image, the candidate region and the current object category into a preset image segmentation model, and performing image segmentation on the current object image according to the candidate region and the current object category through the image segmentation model to generate a polygonal envelope corresponding to the object in the current object image; The information of the polygon envelope and the current object category are combined to obtain the annotation information of the current object image, and the annotation information is added to the current object image to generate the annotated current object image.

[0011] Further, after obtaining the polygon envelope corresponding to the object in the current object image, combining the information of the polygon envelope with the current object category to generate the annotation information of the current object image, adding the annotation information to the current object image to generate the annotated current object image, the automatic annotation method includes: A display window of the display page is created, and the annotation information is displayed through the display window.

[0012] In a second aspect, an automatic labeling device is provided, comprising: A first extraction module is used to use a preset object feature extraction network to perform feature extraction on preset object images corresponding to each preset object category to obtain preset object features corresponding to each preset object category; A first acquisition module is used to acquire a current object image and acquire a candidate region output by the candidate frame generation network based on the current object image; A second extraction module is used to extract features from the region image corresponding to the candidate region to obtain current object features; A generating module, configured to add the feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories to obtain a sum of similarities corresponding to each of the preset object categories, and to divide the sum of similarities corresponding to each of the preset object categories by the number of features corresponding to each of the preset object categories to generate a category similarity corresponding to each of the preset object categories; A selection module, configured to select the preset object category with the greatest category similarity as the current object category corresponding to the current object feature; The labeling module is used to obtain the polygon envelope corresponding to the object in the current object image, combine the information of the polygon envelope with the current object category, generate the labeling information of the current object image, add the labeling information to the current object image, and generate the labeled current object image.

[0013] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned automatic labeling method when executing the computer program.

[0014] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned automatic labeling method are implemented.

[0015] The present application provides an automatic labeling method, apparatus, computer equipment and storage medium, which use a preset object feature extraction network to extract features from preset object images corresponding to each preset object category to obtain preset object features corresponding to each preset object category; obtain a current object image, obtain a candidate area output by the candidate frame generation network based on the current object image; extract features from the area image corresponding to the candidate area to obtain current object features; add feature similarities between the current object features and the preset object features corresponding to each of the preset object categories to obtain the sum of similarities corresponding to each of the preset object categories, divide the sum of similarities corresponding to each of the preset object categories by the number of features corresponding to each of the preset object categories to generate category similarities corresponding to each of the preset object categories; select the preset object category with the largest category similarity as the current object The method comprises the following steps: obtaining a polygonal envelope corresponding to the object in the current object image, combining the information of the polygonal envelope and the current object category, generating annotation information of the current object image, adding the annotation information to the current object image, and generating the annotated current object image. The method has two beneficial effects. On the one hand, obtaining a polygonal envelope corresponding to the object in the current object image, combining the information of the polygonal envelope and the current object category, generating annotation information of the current object image, adding the annotation information to the current object image, and generating the annotated current object image. Since no manual annotation is required, the annotation time of the current object image is reduced, and therefore, it is beneficial to improve the annotation efficiency of the current object image. On the other hand, since it will not be affected by manual intervention, it is beneficial to the reliability of the annotated current object image. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0017] Figure 1 is a schematic diagram of an application environment of an automatic marking method in one embodiment of the present invention; Figure 2 A schematic diagram of a flow chart of an automatic marking method provided by an embodiment of the present invention; Figure 3 yes Figure 1 A schematic flow chart of a specific implementation of step S23; Figure 4 yes Figure 1 A schematic flow chart of a specific implementation of step S25; Figure 5 yes Figure 1 A schematic flow chart of a specific implementation of step S26; Figure 6 is a schematic structural diagram of an automatic marking device in one embodiment of the present invention; Figure 7 It is a structural diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an application environment of an automatic annotation method according to an embodiment of the present invention. The automatic annotation method provided by the embodiment of the present invention can be applied in the following embodiments: Figure 1 In an application environment, a client communicates with a server through a network.

[0020] The server uses a preset object feature extraction network through the client to extract features of preset object images corresponding to each preset object category, and obtains preset object features corresponding to each preset object category; Acquire a current object image, and acquire a candidate region output by the candidate frame generation network based on the current object image; Extracting features from the region image corresponding to the candidate region to obtain features of the current object; Adding the feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories to obtain the sum of the similarities corresponding to each of the preset object categories, and dividing the sum of the similarities corresponding to each of the preset object categories by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories; Selecting the preset object category with the greatest category similarity as the current object category corresponding to the current object feature; Obtain a polygonal envelope corresponding to an object in the current object image, combine information of the polygonal envelope with the current object category, generate annotation information of the current object image, add the annotation information to the current object image, and generate an annotated current object image.

[0021] In the scheme implemented by the above-mentioned automatic labeling method, device, equipment and medium, the beneficial effects are in two aspects. On the one hand, the polygon envelope corresponding to the object in the current object image is obtained, the information of the polygon envelope and the current object category are combined to generate the labeling information of the current object image, and the labeling information is added to the current object image to generate the labeled current object image. Since no manual labeling is required, the labeling time of the current object image is reduced, which is beneficial to improving the labeling efficiency of the current object image. On the other hand, since it will not be affected by manual intervention, it is beneficial to the reliability of the labeled current object image.

[0022] The device running the client is referred to as a client device.

[0023] The device running the server is referred to as the server device.

[0024] Among them, client devices include but are not limited to smartphones, personal computers, Internet of Vehicles terminals, tablets and portable wearable devices.

[0025] The server device can be implemented by an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0026] See also Figure 2 , Figure 2 A flowchart of an automatic labeling method provided by an embodiment of the present invention includes the following steps: S21, using a preset object feature extraction network to extract features from preset object images corresponding to each preset object category, to obtain preset object features corresponding to each preset object category; The object feature extraction network includes a Dinov2 network, a ResNet network, or a combination thereof.

[0027] Preferably, the object feature extraction network adopts the Dinov2 network.

[0028] The full name of Dinov2 in Chinese is: Dual-Stage Implicit Object-Oriented Network Version 2. The Dinov2 network effectively combines image classification and object detection tasks through a dual-stage training method, thus performing well in a variety of downstream tasks, including classification, segmentation, image retrieval, etc.

[0029] Among them, the full Chinese name of the ResNet network is: residual network, and the full English name of the ResNet network is: Residual Neural Network. The ResNet network solves the problems of information loss and attrition in traditional convolutional layers or fully connected layers when transmitting information.

[0030] There may be one or more preset object images corresponding to each preset object category. For ease of explanation, an example is given below: For example, there are three preset object categories, and the three preset object categories are object category 1, object category 1, and object category 3; The preset object images corresponding to the object category 1 are preset image T1, preset image T2, and preset image T3, feature extraction is performed on the preset image T1, preset image T2, and preset image T3, and image features of the preset image T1, preset image T2, and preset image T3 are selected as preset object features corresponding to the object category 1; The preset object images corresponding to the object category 2 are preset image R1, preset image R2, and preset image R3, feature extraction is performed on the preset image R1, preset image R2, and preset image R3, and image features of the preset image R1, preset image R2, and preset image R3 are selected as preset object features corresponding to the object category 2; The preset object images corresponding to the object category 3 are preset image E1, preset image E2, and preset image E3, feature extraction is performed on the preset image E1, preset image E2, and preset image E3, and image features of the preset image E1, preset image E2, and preset image E3 are selected as preset object features corresponding to the object category 3; S22, obtaining a current object image, and obtaining a candidate region output by the candidate frame generation network based on the current object image; The candidate box generation network includes a Grounding DINO model, a Faster R-CNN model, or a combination thereof.

[0031] Preferably, the candidate box generation network adopts the Grounding DINO model.

[0032] Among them, the Grounding DINO model is a zero-shot object detection model that can detect specified targets in an image based only on text descriptions.

[0033] Among them, the Chinese full name of the Faster R-CNN model is: Faster Regional Convolutional Neural Network, and the English full name is: Faster Region-based Convolutional Neural Network.

[0034] Among them, the Faster R-CNN model realizes the rapid generation of candidate regions by introducing the region proposal network.

[0035] The step of obtaining the current object image and obtaining the candidate region output by the candidate frame generation network based on the current object image includes: Acquire a current object image, and input a feature vector of the current object image into a preset candidate box generation network; Obtain a candidate area output by the candidate box generation network based on the feature vector of the current object image.

[0036] Among them, Region Proposal is an important concept in the field of computer vision, especially in target detection tasks. In target detection tasks, objects in images need to be classified and their locations determined. By pre-selecting some regions that may contain objects as candidate regions and running more complex algorithms in these candidate regions, objects can be identified more accurately and their locations can be determined.

[0037] 23. Extract features from the region image corresponding to the candidate region to obtain features of the current object; S24, adding the feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories to obtain the sum of the similarities corresponding to each of the preset object categories, and dividing the sum of the similarities corresponding to each of the preset object categories by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories; The method of adding the feature similarities between the current object feature and the preset object features corresponding to each preset object category to obtain the sum of the similarities corresponding to each preset object category, and dividing the sum of the similarities corresponding to each preset object category by the number of features corresponding to each preset object category to generate the category similarities corresponding to each preset object category includes: Selecting the number of preset object features corresponding to the preset object category as the number of features corresponding to the preset object category; The feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories are added to obtain the sum of the similarities corresponding to each of the preset object categories, and the sum of the similarities corresponding to each of the preset object categories is divided by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories.

[0038] For the sake of illustration, an example is given below: For example, there are three preset object categories, and the three preset object categories are object category 1, object category 1, and object category 3; The number of preset object features corresponding to object category 1 is 5, the number of preset object features corresponding to object category 2 is 6, and the number of preset object features corresponding to object category 3 is 7; Among them, 5 are selected as the number of features corresponding to the object category 1.

[0039] Among them, 6 are selected as the number of features corresponding to the object category 2.

[0040] Among them, 7 are selected as the number of features corresponding to the object category 3.

[0041] Among them, according to the preset generation model, the feature similarities between the current object features and the preset object features corresponding to each of the preset object categories are added to obtain the sum of the similarities corresponding to each of the preset object categories, and the sum of the similarities corresponding to each of the preset object categories is divided by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories.

[0042] Exemplarily, the generation model is: ; in, is the category similarity of the p-th preset object category, pn is the number of features corresponding to the p-th preset object category, is the feature similarity between the current object feature and the i-th preset object feature corresponding to the p-th preset object category, p is the serial number of the preset object category, i is the serial number of the preset object feature, S is the category similarity, and K is the feature similarity.

[0043] The feature similarity is used to describe the similarity between the current object feature and the preset object feature corresponding to the preset object category.

[0044] Among them, the greater the feature similarity, the higher the similarity between the current object feature and the preset object feature corresponding to the preset object category; the smaller the feature similarity, the lower the similarity between the current object feature and the preset object feature corresponding to the preset object category.

[0045] The category similarity is used to describe the similarity between the current object feature and the preset object category.

[0046] The greater the category similarity, the higher the similarity between the current object feature and the preset object category; the smaller the category similarity, the lower the similarity between the current object feature and the preset object category.

[0047] S25, selecting the preset object category with the largest category similarity as the current object category corresponding to the current object feature; For the sake of illustration, an example is given below: For example, there are three preset object categories, and the three preset object categories are object category 1, object category 1, and object category 3; The category similarity corresponding to object category 1 is A1, the category similarity corresponding to object category 1 is A2, and the category similarity corresponding to object category 3 is A3; When A1 is the largest among A1, A2, and A3, object category 1 is selected as the current object category corresponding to the current object feature.

[0048] When A2 is the largest among A1, A2, and A3, object category 2 is selected as the current object category corresponding to the current object feature.

[0049] When A3 is the largest among A1, A2, and A3, object category 3 is selected as the current object category corresponding to the current object feature.

[0050] Among them, the preset object category with the greatest category similarity is the most consistent with the current object category. Selecting the preset object category with the greatest category similarity as the current object category corresponding to the current object feature is beneficial to improving the accuracy of the identified current object category.

[0051] S26, obtaining the polygonal envelope corresponding to the object in the current object image, combining the information of the polygonal envelope with the current object category to generate annotation information of the current object image, adding the annotation information to the current object image, and generating the annotated current object image.

[0052] The polygonal envelope of an object refers to the smallest circumscribed polygon that can fit tightly and completely cover the outline of the object. This polygon consists of a series of vertices connected in sequence, each vertex is located at the edge of the object to ensure that the polygon can accurately wrap the object.

[0053] Exemplarily, combining the information of the polygon envelope and the current object category to generate annotation information of the current object image, adding the annotation information to the current object image to generate the annotated current object image includes: Using a preset annotation format, combining the information of the polygon envelope and the current object category to obtain annotation information of the current object image; The annotation information is added to the current object image to generate the annotated current object image, wherein the annotation format includes one of a COCO format and a YOLO format or a combination thereof.

[0054] Among them, COCO format and YOLO format are two important annotation formats in the field of target detection.

[0055] The Chinese definition of COCO is: objects in common contexts, and the full English name of COCO is: CommonObjects in Context. The COCO format uses JSON files to store annotation information, which makes the COCO format extremely valuable in tasks such as object detection, segmentation, and scene understanding. The advantage of the COCO format is the richness and diversity of its annotation information, which can help improve the recognition accuracy and generalization ability of the algorithm.

[0056] The Chinese definition of YOLO is: You Only Look Once. The full name of YOLO is: You Only Look Once. The YOLO format uses a txt file to store annotation information, and each line represents the annotation result of an object. The advantage of this format is that it has a fast processing speed and can achieve real-time target detection. It is very suitable for application scenarios that require fast response, such as autonomous driving and real-time video surveillance.

[0057] For the sake of explanation, the application process of this application is as follows: (1) The preset object images corresponding to each preset object category are processed in sequence to obtain the preset object features of a single preset object category. The operation is repeated in the above manner to obtain a feature library corresponding to multiple preset object categories.

[0058] (2) With the help of the candidate box generation network, adjust the box generation parameter of the candidate box generation network to 0.18, use objects as the text prompt, let the candidate box generation network read the current object image to be labeled from the outside, and find the candidate area in the current object image.

[0059] (3) For the portion of the image in the candidate area, the dinov2 network is also used to extract the current object features, and the feature similarities between the current object features and the preset object features corresponding to each of the preset object categories are added to obtain the sum of the similarities corresponding to each of the preset object categories, and the sum of the similarities corresponding to each of the preset object categories is divided by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories. The preset object category with the largest category similarity is selected as the current object category corresponding to the current object feature.

[0060] (4) For the current object category and candidate area obtained in step (3), the SAM network is used to process them to obtain the polygonal envelope corresponding to the object; (5) Combining the information of the polygon envelope and the current object category to obtain labeling information of the current object image.

[0061] Wherein, after obtaining the polygon envelope corresponding to the object in the current object image, combining the information of the polygon envelope with the current object category to generate the annotation information of the current object image, adding the annotation information to the current object image to generate the annotated current object image, the automatic annotation method includes: A display window of the display page is created, and the annotation information is displayed through the display window.

[0062] In the embodiment of the present invention, the beneficial effects lie in two aspects. On the one hand, the polygonal envelope corresponding to the object in the current object image is obtained, the information of the polygonal envelope and the current object category are combined to generate the annotation information of the current object image, and the annotation information is added to the current object image to generate the annotated current object image. Since no manual annotation is required, the annotation time of the current object image is reduced, which is beneficial to improving the annotation efficiency of the current object image. On the other hand, since it will not be affected by manual intervention, it is beneficial to the reliability of the annotated current object image.

[0063] See also Figure 3 , Figure 3 yes Figure 1 A specific implementation flow diagram of step S23 is described in detail as follows: S31, performing feature extraction on the region image corresponding to the candidate region through the object feature extraction network to obtain image features of the region image; S32, selecting the image features of the regional image as the current object features.

[0064] In an embodiment of the present invention, since image features can directly capture the visual attributes of an object, such as shape, texture, color, etc., which are crucial for object recognition, selecting the image features of the regional image as the current object features is beneficial to improving the accuracy and reliability of identifying the regional image.

[0065] See also Figure 4 , Figure 4 yes Figure 1 A specific implementation flow diagram of step S25 is described in detail as follows: S41, obtaining a preset similarity, and determining whether the category similarity is greater than the preset similarity; S42: When the category similarity is greater than the preset similarity, selecting the preset object category with the greatest category similarity as the current object category corresponding to the current object feature.

[0066] In an embodiment of the present invention, when the category similarity is greater than the preset similarity, the preset object category with the greatest category similarity is selected as the current object category corresponding to the current object feature. Since the preset object category with the greatest category similarity is closest to the current object category, the preset object category with the greatest category similarity is selected as the current object category corresponding to the current object feature, which can reduce the risk of misidentification and is conducive to improving the accuracy of the identified current object category.

[0067] See also Figure 5 , Figure 5 yes Figure 1 A specific implementation flow diagram of step S26 is described in detail as follows: S51, inputting the current object image, the candidate region and the current object category into a preset image segmentation model, and performing image segmentation on the current object image according to the candidate region and the current object category through the image segmentation model to generate a polygonal envelope corresponding to the object in the current object image; The image segmentation model includes a SAM model, a Mask R-CNN model, or a combination thereof.

[0068] Preferably, the image segmentation model adopts a SAM model.

[0069] The Chinese name of the SAM model is: Segment Anything Model, and the English name is: Segment Anything Model. The SAM model is a model that can handle multiple segmentation tasks.

[0070] The Chinese name of the Mask R-CNN model is: Mask Region-based Convolutional Neural Network, and the English name is: Mask Region-based Convolutional Neural Network. The Mask R-CNN model is a deep learning model used in the fields of image segmentation and object detection.

[0071] S52, combining the information of the polygon envelope and the current object category to obtain the annotation information of the current object image, adding the annotation information to the current object image to generate the annotated current object image.

[0072] In an embodiment of the present invention, the information of the polygon envelope and the current object category are combined to obtain the annotation information of the current object image, and the annotation information is added to the current object image to generate the annotated current object image. Since no manual annotation is required, the annotation time of the current object image is reduced, which is beneficial to improving the annotation efficiency of the current object image.

[0073] See also Figure 6 , Figure 6 FIG. 1 is a schematic diagram of a structure of an automatic marking device in one embodiment of the present invention. Figure 6 As shown, the automatic labeling device includes a first extraction module 101, a first acquisition module 102, a second extraction module 103, a generation module 104, a selection module 105, and a labeling module 106. The functional modules are described in detail as follows: The first extraction module 101 is used to use a preset object feature extraction network to perform feature extraction on preset object images corresponding to each preset object category to obtain preset object features corresponding to each preset object category; A first acquisition module 102 is used to acquire a current object image and acquire a candidate region output by the candidate box generation network based on the current object image; The second extraction module 103 is used to extract features from the region image corresponding to the candidate region to obtain current object features; A generating module 104 is used to add the feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories to obtain the sum of the similarities corresponding to each of the preset object categories, and divide the sum of the similarities corresponding to each of the preset object categories by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories; A selection module 105, configured to select the preset object category with the greatest category similarity as the current object category corresponding to the current object feature; The labeling module 106 is used to obtain the polygon envelope corresponding to the object in the current object image, combine the information of the polygon envelope with the current object category, generate the labeling information of the current object image, add the labeling information to the current object image, and generate the labeled current object image.

[0074] In the embodiment of the present invention, the beneficial effects lie in two aspects. On the one hand, the polygonal envelope corresponding to the object in the current object image is obtained, the information of the polygonal envelope and the current object category are combined to generate the annotation information of the current object image, and the annotation information is added to the current object image to generate the annotated current object image. Since no manual annotation is required, the annotation time of the current object image is reduced, which is beneficial to improving the annotation efficiency of the current object image. On the other hand, since it will not be affected by manual intervention, it is beneficial to the reliability of the annotated current object image.

[0075] For the specific definition of the automatic marking device, please refer to the definition of the automatic marking method above, which will not be repeated here.

[0076] Each module in the above automatic annotation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0077] See also Figure 7 , Figure 7 1 is another structural diagram of a computer device in one embodiment of the present invention. In one embodiment, a computer device is provided. The computer device is a server device or a client device. The internal structure diagram thereof can be as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external database. When the computer program is executed by the processor, a function or step of an automatic labeling method can be implemented.

[0078] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor.

[0079] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant description of the aforementioned method embodiment. In order to avoid repetition, they will not be described one by one here.

[0080] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0081] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present disclosure.

[0082] Technicians can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0083] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. An automatic labeling method, characterized in that: include: Using a preset object feature extraction network, feature extraction is performed on preset object images corresponding to each preset object category to obtain preset object features corresponding to each preset object category; Acquire a current object image, and acquire a candidate region output by the candidate frame generation network based on the current object image; Extracting features from the region image corresponding to the candidate region to obtain features of the current object; Adding the feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories to obtain the sum of the similarities corresponding to each of the preset object categories, and dividing the sum of the similarities corresponding to each of the preset object categories by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories; Selecting the preset object category with the greatest category similarity as the current object category corresponding to the current object feature; Obtain a polygonal envelope corresponding to an object in the current object image, combine information of the polygonal envelope with the current object category, generate annotation information of the current object image, add the annotation information to the current object image, and generate an annotated current object image.

2. The automatic marking method according to claim 1, characterized in that: The obtaining of the current object image and the obtaining of the candidate region output by the candidate frame generation network based on the current object image include: Acquire a current object image, and input a feature vector of the current object image into a preset candidate box generation network; Obtain a candidate area output by the candidate frame generation network based on the feature vector of the current object image.

3. The automatic marking method according to claim 1, characterized in that: The extracting features of the region image corresponding to the candidate region to obtain the current object features includes: By using the object feature extraction network, feature extraction is performed on the regional image corresponding to the candidate region to obtain image features of the regional image; The image features of the region image are selected as the current object features.

4. The automatic marking method according to claim 1, characterized in that: The adding of the feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories to obtain the sum of the similarities corresponding to each of the preset object categories, and dividing the sum of the similarities corresponding to each of the preset object categories by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories, includes: Selecting the number of preset object features corresponding to the preset object category as the number of features corresponding to the preset object category; The feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories are added to obtain the sum of the similarities corresponding to each of the preset object categories, and the sum of the similarities corresponding to each of the preset object categories is divided by the number of features corresponding to each of the preset object categories to generate the category similarities corresponding to each of the preset object categories.

5. The automatic marking method according to claim 1, characterized in that: The selecting the preset object category with the largest category similarity as the current object category corresponding to the current object feature includes: Obtaining a preset similarity, and determining whether the category similarity is greater than the preset similarity; When the category similarity is greater than the preset similarity, the preset object category with the greatest category similarity is selected as the current object category corresponding to the current object feature.

6. The automatic marking method according to claim 1, characterized in that: The step of obtaining a polygon envelope corresponding to the object in the current object image, combining information of the polygon envelope with the current object category to generate annotation information of the current object image, and adding the annotation information to the current object image to generate the annotated current object image includes: Inputting the current object image, the candidate area and the current object category into a preset image segmentation model, and performing image segmentation on the current object image according to the candidate area and the current object category through the image segmentation model to generate a polygonal envelope corresponding to the object in the current object image; The information of the polygon envelope and the current object category are combined to obtain the annotation information of the current object image, and the annotation information is added to the current object image to generate the annotated current object image.

7. The automatic marking method according to claim 1, characterized in that: After obtaining a polygonal envelope corresponding to an object in the current object image, combining information of the polygonal envelope with the current object category to generate annotation information of the current object image, and adding the annotation information to the current object image to generate the annotated current object image, the automatic annotation method includes: A display window of the display page is created, and the annotation information is displayed through the display window.

8. An automatic labeling device, characterized in that: include: A first extraction module is used to use a preset object feature extraction network to perform feature extraction on preset object images corresponding to each preset object category to obtain preset object features corresponding to each preset object category; A first acquisition module is used to acquire a current object image and acquire a candidate region output by the candidate frame generation network based on the current object image; A second extraction module is used to extract features from the region image corresponding to the candidate region to obtain current object features; A generating module, configured to add the feature similarities between the current object feature and the preset object features corresponding to each of the preset object categories to obtain a sum of similarities corresponding to each of the preset object categories, and to divide the sum of similarities corresponding to each of the preset object categories by the number of features corresponding to each of the preset object categories to generate a category similarity corresponding to each of the preset object categories; A selection module, configured to select the preset object category with the greatest category similarity as the current object category corresponding to the current object feature; The labeling module is used to obtain the polygon envelope corresponding to the object in the current object image, combine the information of the polygon envelope with the current object category, generate the labeling information of the current object image, add the labeling information to the current object image, and generate the labeled current object image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the automatic annotation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the automatic annotation method according to any one of claims 1 to 7 are implemented.

Images