Training of a labeling model, labeling method and device, equipment and medium
By training a first object detection model and using bias to train a second object detection model, the accuracy judgment is increased, which solves the problems of low efficiency and unstable quality of manual annotation, realizes efficient object recognition and pre-annotation, and reduces the training cost of autonomous driving algorithms.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING CHANGAN AUTOMOBILE CO LTD
- Filing Date
- 2022-10-27
- Publication Date
- 2026-07-24
AI Technical Summary
In existing technologies, manual annotation is inefficient and of unstable quality, resulting in high training costs for autonomous driving algorithms and an inability to continuously iterate. Furthermore, existing pre-annotation algorithms have high accuracy requirements, which affects annotation efficiency.
By acquiring and labeling objects, a first object detection model is trained, and a second object detection model is trained using bias. An accuracy judgment model is added, and high-accuracy targets are selected for pre-labeling, reducing the reliance on pre-labeling algorithms.
It improves the efficiency of target identification and annotation, reduces the accuracy requirements of pre-annotation algorithms, and enhances annotation quality and efficiency.
Smart Images
Figure CN115620088B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data annotation technology, specifically relating to a training and annotation method, apparatus, equipment, and medium for an annotation model. Background Technology
[0002] Autonomous driving research and development requires a large amount of labeled, high-quality data to train algorithms and achieve excellent training results. Currently, manual annotation is inefficient, and its quality is greatly affected by the maturity of the annotators, resulting in inconsistent quality. The large-scale training demand leads to continuous and high costs for annotation services, increasing the financial burden on companies and hindering the sustainable iteration of algorithms. Therefore, emerging car manufacturers like Tesla and various AI companies are actively developing pre-annotation technologies.
[0003] Current annotation methods primarily involve training a single pre-annotation algorithm to identify targets in images or radar point clouds. The identification results are then automatically pre-loaded into the annotation system, followed by manual review. Qualified annotations are retained, while invalid ones are deleted or have their borders resized. However, this method requires the pre-annotation algorithm to achieve a certain level of accuracy. If the target bounding box is not accurately identified, the annotator needs to manually adjust the border position after automatic pre-annotation, which negatively impacts annotation efficiency. Summary of the Invention
[0004] In view of the shortcomings of the prior art described above, the present invention provides a training and annotation method, apparatus, device and medium for annotated models to solve the above-mentioned technical problems.
[0005] This invention provides a training method for an labeled model, the method comprising:
[0006] Get the first object to be labeled;
[0007] The first object to be labeled is labeled to obtain a first labeled object. The first labeled object contains a target label box, and the target label box contains the labeling target in the first object to be labeled.
[0008] Using the first object to be labeled and the first labeled object as the first training set, a first object detection model is trained to obtain the first object detection model.
[0009] The first target detection model is used to perform target detection on the first labeled object to obtain a target detection box;
[0010] The deviation between the target detection box and the target annotation box is obtained, and the second target detection model is trained using the first object to be annotated and the deviation as the second training set.
[0011] In one embodiment of the present invention, the first target detection model and the second target detection model adopt the same network structure; when training the second target detection model, the parameters of the first target detection model are transferred to the second target detection model.
[0012] In one embodiment of the present invention, the first object to be labeled includes an image or point cloud data.
[0013] In one embodiment of the present invention, the deviation in the second training set is a deviation greater than a set threshold.
[0014] In one embodiment of the present invention, the first target detection model is a target detection model based on the YOLO concept; after labeling the first object to be labeled to obtain the first labeled object, the method further includes:
[0015] The first labeled object is meshed so that one or more labeled targets in the first labeled object are distributed in different grids.
[0016] In one embodiment of the present invention, the deviation includes the intersection-union ratio of the target detection box and the target annotation box.
[0017] This invention provides a training device for annotated models, the device comprising:
[0018] The first acquisition module is used to acquire the first object to be labeled.
[0019] The annotation module is used to annotate the first object to be annotated to obtain a first annotation object. The first annotation object contains a target annotation box, and the target annotation box contains the annotation target in the first object to be annotated.
[0020] The first training module is used to train a first object detection model using the first object to be labeled and the first labeled object as a first training set.
[0021] The first detection module is used to perform target detection on the first labeled object using the first target detection model to obtain a target detection box;
[0022] The second training module is used to obtain the deviation between the target detection box and the target annotation box, and to train the second target detection model using the first object to be annotated and the deviation as the second training set.
[0023] This invention provides an annotation method, the method comprising:
[0024] Obtain the second object to be labeled;
[0025] The first target detection model and the second target detection model are used to detect the second object to be labeled, respectively, to obtain the first detection result and the second detection result.
[0026] The first prediction result and the second prediction result are concatenated to obtain a third prediction result, which is then used as the annotation result to complete the pre-annotation.
[0027] In one embodiment of the present invention, the third prediction result includes multiple prediction items, and the annotation result serves as an annotation item. The method further includes:
[0028] Obtain at least one of the plurality of prediction terms;
[0029] The at least one prediction term is compared with a reference value corresponding to the at least one prediction term;
[0030] The third prediction result to which the prediction term belongs is greater than or equal to the reference value corresponding to the at least one prediction term, and is used as the annotation result.
[0031] The present invention provides a labeling device, the device comprising:
[0032] The second acquisition module is used to acquire the second object to be labeled.
[0033] The second detection module is used to detect the second object to be labeled using the first target detection model and the second target detection model respectively, and obtain the first detection result and the second detection result.
[0034] The fusion annotation module is used to concatenate the first prediction result and the second prediction result to obtain a third prediction result, and use the third prediction result as the annotation result to complete the pre-annotation.
[0035] The present invention provides an electronic device, the electronic device comprising:
[0036] One or more processors;
[0037] A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the steps of the above-described training method or annotation method for the labeled model.
[0038] The present invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer processor, causes the computer to perform the steps of the above-described training method or annotation method for the labeled model.
[0039] The beneficial effects of the present invention are as follows: A training method for an annotation model according to the present invention includes: obtaining a first object to be annotated; annotating the first object to be annotated to obtain a first annotated object, wherein the first annotated object contains a target annotation box, and the target annotation box contains the annotated target in the first object to be annotated; training a first object detection model using the first object to be annotated and the first annotated object as a first training set; performing object detection on the first annotated object using the first object detection model to obtain a target detection box; obtaining the deviation between the target detection box and the target annotation box, and training a second object detection model using the first object to be annotated and the deviation as a second training set.
[0040] This invention adds a second target detection model, a target accuracy judgment model, to the first target detection model. This second model predicts the accuracy of the target objects identified by the first target detection model, thereby filtering out targets with high accuracy for pre-labeling. This method achieves target identification pre-labeling without relying on highly precise pre-labeling algorithms, thus improving the efficiency of target identification and labeling.
[0041] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:
[0043] Figure 1 This is a schematic diagram illustrating the implementation environment of a training method for a labeled model, as shown in an exemplary embodiment of this application.
[0044] Figure 2 This is a flowchart illustrating a training method for a labeled model, as shown in an exemplary embodiment of this application;
[0045] Figure 3 This is a schematic diagram illustrating a qualitative evaluation of the correlation trend between actual IOU and predicted IOU, as shown in an exemplary embodiment of this application.
[0046] Figure 4 This is a block diagram illustrating a training apparatus for a labeled model, as shown in an exemplary embodiment of this application;
[0047] Figure 5 This is a flowchart illustrating an annotation method in an exemplary embodiment of this application;
[0048] Figure 6 This is a block diagram illustrating a labeling device in an exemplary embodiment of this application;
[0049] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation
[0050] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.
[0051] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0052] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.
[0053] Figure 1 This is a schematic diagram illustrating the implementation environment of a training method for an exemplary labeled model according to this application. Please refer to... Figure 1The implementation environment includes a terminal device 101 and a server 102, which communicate with each other via a wired or wireless network. The terminal device acquires a first object to be labeled and sends it to the server. The server labels the first object to be labeled to obtain a first labeled object, which contains a target bounding box containing the labeled target from the first object to be labeled. A first object detection model is trained using the first object to be labeled and the first labeled object as a first training set. The first object detection model is then used to detect the first labeled object to obtain a target detection box. The deviation between the target detection box and the target labeled box is obtained, and a second object detection model is trained using the first object to be labeled and the deviation as a second training set.
[0054] This invention adds a second target detection model, a target accuracy judgment model, to the first target detection model. This second model predicts the accuracy of the target objects identified by the first target detection model, thereby filtering out targets with high accuracy for pre-labeling. This method achieves target identification pre-labeling without relying on highly precise pre-labeling algorithms, thus improving the efficiency of target identification, target detection labeling, and target classification.
[0055] Furthermore, the method of this invention is not only applicable to simple image target detection, but also, using the same approach, can be applied to the pre-annotation of various AI algorithms such as image-based lane line detection, 2D image semantic segmentation, 3D laser point cloud target detection, and 3D semantic segmentation.
[0056] It should be understood that Figure 1 The number of terminal devices 101 and servers 102 shown is merely illustrative. Any number of terminal devices 101 and servers 102 can be used depending on actual needs.
[0057] The terminal device 101 corresponds to the client, which can be any electronic device with a user input interface, including but not limited to smartphones, tablets, laptops, computers, etc. The user input interface includes but is not limited to touch screens, keyboards, physical buttons, audio pickup devices, etc.
[0058] Among them, server 102 corresponds to the server side, which can be a server that provides various services. It can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This section does not impose any restrictions on this.
[0059] Terminal device 101 can communicate with server 102 via wireless networks such as 3G (third-generation mobile information technology), 4G (fourth-generation mobile information technology), and 5G (fifth-generation mobile information technology), and this is not restricted here.
[0060] The embodiments of this application respectively propose a training method for a labeled model, a labeled method, a training device for a labeled model, a labeled device, an electronic device, and a computer-readable storage medium. These embodiments will be described in detail below.
[0061] Please see Figure 2 , Figure 2 This is a flowchart illustrating a training method for a labeled model, as shown in an exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown is specifically executed by the terminal device and server within that implementation environment. It should be understood that this method can also be applied to other exemplary implementation environments and executed by devices in other implementation environments; this embodiment does not limit the implementation environment to which the method is applicable.
[0062] Please see Figure 2 , Figure 2 The flowchart below illustrates an exemplary method for training a labeled model according to this application. This method includes at least steps S210 to S270, which are detailed below:
[0063] Step S210: Obtain the first object to be labeled;
[0064] Step S220: Label the first object to be labeled to obtain a first labeled object. The first labeled object contains a target label box, and the target label box contains the labeling target in the first object to be labeled.
[0065] Step S230: Using the first object to be labeled and the first labeled object as the first training set, train the first object detection model.
[0066] Step S240: Use the first target detection model to perform target detection on the first labeled object to obtain a target detection box;
[0067] Step S250: Obtain the deviation between the target detection box and the target annotation box, and use the first object to be annotated and the deviation as the second training set to train a second target detection model.
[0068] Based on the first target detection model, this invention adds a second target detection model for target accuracy judgment, which is used to predict the accuracy of the target objects identified by the first target detection model, thereby filtering out targets with high accuracy for pre-labeling.
[0069] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0070] The following provides a detailed explanation of each of the above steps.
[0071] In step S210, the first object to be labeled is obtained;
[0072] The first object to be labeled is the object that needs to be labeled. This object includes one or more targets to be labeled, such as people, vehicles, animals, trees, buildings, etc. The object to be labeled can be an image or radar point cloud data.
[0073] In step S220, the first object to be labeled is labeled to obtain a first labeled object. The first labeled object contains a target label box, and the target label box contains the labeling target in the first object to be labeled.
[0074] Taking image annotation as an example, image annotation refers to marking a bounding box (label box) containing the target in the image and / or assigning a label to the target. The methods for image annotation vary depending on the task. For example, for image target detection, image annotation involves marking the bounding box (label box) containing the target in the image (e.g., using a rectangle or ellipse to outline the area containing the target); for image target recognition, image annotation involves both marking the bounding box (label box) containing the target and assigning a label to the marked bounding box (label box) indicating the category of the target within the bounding box (e.g., apple, banana); for image classification, if each collected image contains only one type of target, image annotation only requires assigning a label to each image, which is the category to which the target in the image belongs.
[0075] The first annotation object refers to the image after it has been annotated. This annotated image includes a target annotation box, which contains the annotation target within the first object to be annotated. The annotation target can be a person, vehicle, animal, tree, building, etc. In this embodiment, the first object to be annotated is annotated manually.
[0076] Of course, the first object to be labeled can also be radar point cloud data. The specific labeling method can be found in the labeling of images, and will not be explained in detail here.
[0077] In step S230, the first object to be labeled and the first labeled object are used as the first training set to train the first object detection model.
[0078] Specifically, a deep learning algorithm is constructed, with the first object to be labeled as input and the first labeled object as output. The algorithm is trained to obtain the first object detection model.
[0079] After training the first object detection model, traditional evaluation methods (such as mAP, Mean Average Precision) are used to verify the results of the first object detection model. Once the model is confirmed to meet the standard, the next step is initiated.
[0080] In one embodiment, the first target detection model is a target detection model based on the YOLO concept; after labeling the object to be labeled to obtain the first labeled object, the method further includes:
[0081] The first labeled object is gridded so that one or more labeled targets within the first labeled object are distributed across different grids. For example, the first labeled object is divided into S*S grid cells. If the center of an object falls within a certain grid cell, then that grid cell is responsible for detecting the target. Each grid cell predicts and generates multiple target detection bounding boxes, along with their corresponding confidence scores and target type probabilities. These represent the probability that each grid cell contains a target category and the localization accuracy.
[0082] In step S240, the first target detection model is used to perform target detection on the first labeled object to obtain a target detection box;
[0083] Specifically, inputting the first labeled object into the first object detection model will result in multiple object detection boxes. Each object detection box will also include a confidence level and an object type probability. At this point, a probability threshold can be set; only object detection boxes with a probability greater than this threshold are considered for detection.
[0084] In step S250, the deviation between the target detection box and the target annotation box is obtained, and the second target detection model is trained using the first object to be annotated and the deviation as the second training set.
[0085] It should be noted that the target detection box output by the first target detection model is a predicted box. Therefore, it is compared with the target annotation box in the first labeled object to obtain the deviation between the target detection box and the target annotation box. This deviation and the first object to be labeled are used as the training set to train the second target detection model.
[0086] The first object detection model detects the first labeled object, obtaining the bounding box for each target in each grid. For each network, the bias of the grid without a target is set to 0. This bias can be directly used as the target variable for each grid, or a threshold can be set, with grids having a bias exceeding the threshold set to 1 and those below the threshold set to 0. The bias exceeding the threshold is then used as the target variable. The second object detection model is trained using the first labeled object as the training set.
[0087] In one embodiment, the deviation is the intersection-over-union ratio (IOU) of the target detection box and the target annotation box (IOU, the area or volume of the intersection of the target detection box and the target annotation box / the combined area or volume of the two).
[0088] When training the second object detection model, a new deep learning model is constructed using the original image / radar point cloud and the target variable (bias) as the training set. This model is then trained to generate the second object detection model. When constructing the second object detection model, a network similar to the first object detection model can generally be sampled, with adjustments made only to the target after the final fully connected layer. Furthermore, to accelerate the training speed of the second object detection model, a method similar to transfer learning can be used. The network model parameters before full connection directly use the parameters of the first object detection model without backpropagation, thereby significantly improving the training speed.
[0089] After training the second object detection model, it can be evaluated using the MSE (Mean Squared Error) metric, or qualitatively evaluated by the correlation trend between the actual IOU and the predicted IOU. Figure 3 As shown.
[0090] Figure 4 This is a block diagram illustrating a labeling device according to an exemplary embodiment of this application. The device can be applied to… Figure 1The implementation environment shown is specifically configured in a server. This device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.
[0091] like Figure 4 As shown, this application provides a training apparatus for annotated models, the apparatus comprising:
[0092] The first acquisition module 410 is used to acquire the first object to be labeled;
[0093] The annotation module 420 is used to annotate the first object to be annotated to obtain a first annotation object, wherein the first annotation object contains a target annotation box, and the target annotation box contains the annotation target in the first object to be annotated;
[0094] The first training module 430 is used to train a first object detection model using the first object to be labeled and the first labeled object as a first training set.
[0095] The first detection module 440 is used to perform target detection on the first labeled object using the first target detection model to obtain a target detection box;
[0096] The second training module 450 is used to obtain the deviation between the target detection box and the target annotation box, and to train the second target detection model using the first object to be annotated and the deviation as the second training set.
[0097] It should be noted that the first object to be labeled, i.e., the object that needs to be labeled, includes one or more targets to be labeled. These targets can be people, vehicles, animals, trees, buildings, etc. The object to be labeled can be an image or can include radar point clouds.
[0098] Let's take an image as an example to illustrate this.
[0099] Image annotation refers to marking a bounding box (label box) containing a target within an image and / or assigning a label to the target. The methods for image annotation vary depending on the task. For example, for image target detection, image annotation involves marking bounding boxes (label boxes) that include the target (e.g., using rectangles or ellipses to define the area containing the target). For image target recognition, image annotation involves both marking the bounding boxes and assigning a label to each bounding box, indicating the category of the target within that bounding box (e.g., apple, banana). For image classification, if each collected image contains only one type of target, image annotation only requires assigning a label to each image, which represents the category of the target in the image.
[0100] The first annotation object refers to the image after it has been annotated. This annotated image includes a target annotation box, which contains the annotation target within the first object to be annotated. The annotation target can be a person, vehicle, animal, tree, building, etc. In this embodiment, the first object to be annotated is annotated manually.
[0101] When training the first object detection model, a deep learning algorithm is constructed, with the first object to be labeled as input and the first labeled object as output. The model is trained based on the deep learning algorithm to obtain the first object detection model.
[0102] After training the first object detection model, traditional evaluation methods (such as mAP, Mean Average Precision) are used to verify the results of the first object detection model. Once the model is confirmed to meet the standard, the next step is initiated.
[0103] The first object detection model is an object detection model based on the YOLO concept; after labeling the object to be labeled to obtain the first labeled object, the method further includes:
[0104] The first labeled object is gridded so that one or more labeled targets within the first labeled object are distributed across different grids. For example, the first labeled object is divided into S*S grid cells. If the center of an object falls within a certain grid cell, then that grid cell is responsible for detecting the target. Each grid cell predicts and generates multiple target detection bounding boxes, along with their corresponding confidence scores and target type probabilities. These represent the probability that each grid cell contains a target category and the localization accuracy.
[0105] When using the first object detection model to detect the first labeled object, inputting the first labeled object into the first object detection model will result in multiple object detection boxes. Each object detection box will also include a confidence score and an object type probability. At this point, a probability threshold can be set; only object detection boxes with a probability greater than this threshold are considered valid detection boxes.
[0106] It should be noted that the target detection box output by the first target detection model is a predicted box. Therefore, it is compared with the target annotation box in the first labeled object to obtain the deviation between the target detection box and the target annotation box. This deviation and the first object to be labeled are used as the training set to train the second target detection model.
[0107] The first object detection model detects the first labeled object, obtaining the bounding box for each target in each grid. For each network, the bias of the grid without a target is set to 0. This bias can be directly used as the target variable for each grid, or a threshold can be set, with grids having a bias exceeding the threshold set to 1 and those below the threshold set to 0. The bias exceeding the threshold is then used as the target variable. The second object detection model is trained using the first labeled object as the training set.
[0108] In one embodiment, the deviation is the intersection-over-union ratio (IOU) of the target detection box and the target annotation box (IOU, the area or volume of the intersection of the target detection box and the target annotation box / the combined area or volume of the two).
[0109] When training the second object detection model, a new deep learning model is constructed using the original image / radar point cloud and the target variable (bias) as the training set. This model is then trained to generate the second object detection model. When constructing the second object detection model, a network similar to the first object detection model can generally be sampled, with adjustments made only to the target after the final fully connected layer. Furthermore, to accelerate the training speed of the second object detection model, a method similar to transfer learning can be used. The network model parameters before full connection directly use the parameters of the first object detection model without backpropagation, thereby significantly improving the training speed.
[0110] After training the second object detection model, it can be evaluated using the MSE (Mean Squared Error) metric, or qualitatively evaluated by the correlation trend between the actual IOU and the predicted IOU. Figure 3 As shown.
[0111] It should be noted that the training device for the labeled model provided in the above embodiments and the training method for the labeled model provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the training device for the labeled model provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation here.
[0112] Please see Figure 5 , Figure 5 A flowchart illustrating an annotation method as shown in an exemplary embodiment of this application is provided. The method includes:
[0113] Step S510: Obtain the second object to be labeled;
[0114] Step S520: The first target detection model and the second target detection model in the above embodiments are used to detect the second object to be labeled, respectively, to obtain the first detection result and the second detection result;
[0115] The first object detection model is used to detect the second object to be labeled, and the detection results for each grid are output as follows:
[0116]
[0117] Where: bbox represents the range of the target, confidence represents the confidence that the target exists in the grid, and cls represents target attribute information such as the probability of different target types.
[0118] The second target detection model is used to detect the second object to be labeled, and the detection results for each grid are output as follows:
[0119]
[0120] Where p_iou represents the accuracy of the target detection bounding box detected by the first target detection model in the grid.
[0121] Step S530: The first prediction result and the second prediction result are concatenated to obtain a third prediction result, and the third prediction result is used as the annotation result to complete the pre-annotation.
[0122] The detection results of the second target detection model correspond one-to-one with those of the first target detection model. By concatenating the detection results of the second target detection model and the first target detection model, the following result can be obtained:
[0123]
[0124] In one embodiment, the third prediction result includes multiple prediction items, and the annotation result serves as an annotation item. The method further includes:
[0125] Obtain at least one of the plurality of prediction items; compare the at least one prediction item with a reference value corresponding to the at least one prediction item; and take the third prediction result to which the prediction item to which the prediction item is greater than or equal to the reference value corresponding to the at least one prediction item belongs as the labeling result.
[0126] The prediction terms include bounding box, confidence, cls, and p_iou, with the confidence threshold λ serving as the reference value for each prediction term. conf Accuracy threshold λ iou If the confidence and p_iou in each third prediction result are both greater than their respective thresholds, that is, if the confidence is greater than the confidence threshold λ conf p_iou is greater than the accuracy threshold λ iou If so, this third prediction result will be used as the labeling result, thus completing the pre-labeling.
[0127] Please see Figure 6 , Figure 6 This is a block diagram illustrating an exemplary embodiment of the present application, the device comprising:
[0128] The second acquisition module 610 is used to acquire the second object to be labeled;
[0129] The second detection module 620 is used to detect the second object to be labeled using the first target detection model and the second target detection model in the foregoing embodiment, respectively, to obtain the first detection result and the second detection result.
[0130] The fusion annotation module 630 is used to concatenate the first prediction result and the second prediction result to obtain a third prediction result, and use the third prediction result as the annotation result to complete the pre-annotation.
[0131] It should be noted that the annotation device and the annotation method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the annotation device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.
[0132] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the annotation methods provided in the above embodiments.
[0133] Figure 7 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 7 The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0134] like Figure 7 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 702 or programs loaded from storage portion 708 into Random Access Memory (RAM) 703, such as performing the methods described in the above embodiments. The RAM 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An Input / Output (I / O) interface 705 is also connected to the bus 704.
[0135] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 707 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. The communication section 707 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.
[0136] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including methods for performing processes. Figure 2 The computer program for the method shown. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs various functions defined in the system of this application.
[0137] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.
[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0139] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.
[0140] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the annotation method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.
[0141] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the annotation methods provided in the various embodiments described above.
[0142] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.
Claims
1. A method for training a labeled model, characterized in that, The method includes: Obtain the first object to be labeled; the first object to be labeled may include an image or radar point cloud. The first object to be labeled is labeled to obtain a first labeled object. The first labeled object contains a target label box, and the target label box contains the labeling target in the first object to be labeled. Using the first object to be labeled and the first labeled object as the first training set, a first object detection model is trained to obtain the first object detection model. The first target detection model is used to perform target detection on the first labeled object to obtain a target detection box; The deviation between the target detection box and the target annotation box is obtained, and the first object to be annotated and the deviation are used as the second training set to train a second target detection model. The second target detection model is a target accuracy judgment model, which is used to predict the accuracy of the target objects identified by the first target detection model, and to select targets with high accuracy for pre-annotation.
2. The training method for the labeled model according to claim 1, characterized in that, The first object detection model and the second object detection model use the same network structure; when training the second object detection model, the parameters of the first object detection model are transferred to the second object detection model.
3. The training method for the labeled model according to claim 1, characterized in that, The bias in the second training set is the bias that is greater than a set threshold.
4. The training method for the labeled model according to claim 1, characterized in that, The first object detection model is an object detection model based on the YOLO concept; after annotating the first object to be labeled to obtain the first labeled object, the method further includes: The first labeled object is meshed so that one or more labeled targets in the first labeled object are distributed in different grids.
5. The training method for the labeled model according to any one of claims 1 to 4, characterized in that, The deviation includes the intersection-union ratio of the target detection box and the target annotation box.
6. A training device for a labeled model, characterized in that, The device includes: The first acquisition module is used to acquire a first object to be labeled; the first object to be labeled includes an image or radar point cloud. The annotation module is used to annotate the first object to be annotated to obtain a first annotation object. The first annotation object contains a target annotation box, and the target annotation box contains the annotation target in the first object to be annotated. The first training module is used to train a first object detection model using the first object to be labeled and the first labeled object as a first training set. The first detection module is used to perform target detection on the first labeled object using the first target detection model to obtain a target detection box; The second training module is used to obtain the deviation between the target detection box and the target annotation box, and to train the second target detection model using the first object to be annotated and the deviation as the second training set; the second target detection model is a target accuracy judgment model, used to predict the accuracy of the target objects identified by the first target detection model, and to filter out targets with high accuracy for pre-annotation.
7. A labeling method, characterized in that, The method includes: Obtain the second object to be labeled; The first target detection model and the second target detection model according to any one of claims 1 to 5 are used to detect the second object to be labeled, respectively, to obtain the first detection result and the second detection result; The first prediction result and the second prediction result are concatenated to obtain a third prediction result, which is then used as the annotation result to complete the pre-annotation.
8. The annotation method according to claim 7, characterized in that, The third prediction result includes multiple prediction items, and the annotation result serves as an annotation item. The method further includes: Obtain at least one of the plurality of prediction terms; The at least one prediction term is compared with a reference value corresponding to the at least one prediction term; The third prediction result to which the prediction term belongs is greater than or equal to the reference value corresponding to the at least one prediction term, and is used as the annotation result.
9. A labeling device, characterized in that, The device includes: The second acquisition module is used to acquire the second object to be labeled. The second detection module is used to detect the second object to be labeled by using the first target detection model and the second target detection model as described in any one of claims 1 to 5, respectively, and to obtain the first detection result and the second detection result. The fusion annotation module is used to concatenate the first prediction result and the second prediction result to obtain a third prediction result, and use the third prediction result as the annotation result to complete the pre-annotation.
10. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the steps of the training method of the labeled model as described in any one of claims 1 to 5 or the steps of the labeling method as described in claim 7.
11. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the steps of the training method for the labeled model as described in any one of claims 1 to 5 or the steps of the labeling method as described in claim 7.