Data annotation method and device

Abstract

The present application provides a data labeling method and apparatus. This method obtains at least one data frame to be processed, and for each data frame to be processed, determines at least one target calibration template for labeling the data frame based on the target type included in the data frame to be processed. The data frame to be processed is then labeled using the at least one target calibration template to obtain target labeling results, thereby obtaining information such as the type, attributes, and location of at least one target present in the data frame to be processed. In this technical solution, targets in the data frame to be processed are labeled based on the configured target calibration template. Because the calibration template is easy to establish and implement, it can accurately reflect the relationship between targets, eliminating the need to train a data labeling model, reducing manual labeling costs, and improving data labeling efficiency.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data labeling method and device. Background Art

[0002] With the rapid development of artificial intelligence (AI), computer vision recognition technology based on deep learning has been widely used in various industries. A high-performing deep learning model requires a large amount of high-quality annotated data to support it. Therefore, obtaining high-quality annotated data is crucial.

[0003] The current data labeling model is trained using a large amount of labeled data, and the labeled data is based on manual labeling. It not only consumes a lot of manpower and time, but the data labeling model can only be applied to the target type used during training, and there are problems with low labeling accuracy and low labeling efficiency. Summary of the Invention

[0004] The present application provides a data labeling method and device to overcome the problems of low labeling accuracy and low labeling efficiency existing in manual data labeling methods.

[0005] In a first aspect, an embodiment of the present application provides a data labeling method, comprising:

[0006] Obtain at least one data frame to be processed;

[0007] For each data frame to be processed, determining at least one target calibration template for marking the data frame to be processed based on the target type included in the data frame to be processed;

[0008] The at least one target calibration template is used to perform target marking on the data frame to be processed to obtain a target marking result.

[0009] In a possible design of the first aspect, the at least one target calibration template includes at least one of the following:

[0010] Calibration templates from the pre-configured calibration template library;

[0011] Calibration templates generated in real time based on historical target annotation results;

[0012] Calibration templates generated in real time based on template configuration instructions.

[0013] As an example, determining at least one target calibration template for marking the data frame to be processed based on the target type included in the data frame to be processed includes:

[0014] Based on the target type included in the data frame to be processed, a pre-configured calibration template library is queried, and at least one target calibration template for marking the data frame to be processed is determined in the calibration template library.

[0015] As another example, determining at least one target calibration template for marking the data frame to be processed based on the target type included in the data frame to be processed includes:

[0016] Acquire a template configuration instruction issued by a user, where the template configuration instruction is issued by the user based on a target type included in the data frame to be processed;

[0017] At least one target calibration template is generated according to the template configuration instruction for marking the data frame to be processed.

[0018] In another possible design of the first aspect, using the at least one target calibration template to perform target labeling on the data frame to be processed to obtain a target labeling result includes:

[0019] Determine at least one target in the data frame to be processed according to boundary information of each target in the data frame to be processed, wherein the at least one target includes: a regularly shaped target with a clear boundary and / or an irregularly shaped target with an unclear boundary;

[0020] For at least one target in the data frame to be processed, each target in the data frame to be processed is labeled using at least one target calibration template that matches the at least one target to obtain a target labeling result of the data frame to be processed.

[0021] Optionally, the labeling of each target in the data frame to be processed using at least one target calibration template matching the at least one target based on the at least one target in the data frame to be processed, to obtain a target labeling result for the data frame to be processed, includes:

[0022] For a regularly shaped target in the data frame to be processed, using a configured regular graphic in the at least one target calibration template that matches the regularly shaped target, to mark the regularly shaped target in the data frame to be processed; and / or

[0023] For irregularly shaped targets in the data frame to be processed, based on the key point positions marked by the user, the at least one target calibration template and the region expansion algorithm are called to mark the irregularly shaped targets in the data frame to be processed.

[0024] In another possible design of the first aspect, the method further includes:

[0025] Display the target labeling results of each data frame to be processed;

[0026] Acquire user review information on the target labeling result, where the review information includes a review pass indication or a modified target labeling result.

[0027] In another possible design of the first aspect, when the data frame to be processed includes at least two targets, the method further includes:

[0028] Determine the relationship between the objects in the data frame to be processed according to the at least one object calibration template and the object labeling result.

[0029] In a second aspect, the present application provides a data labeling device, comprising:

[0030] An acquisition module, configured to acquire at least one data frame to be processed;

[0031] a processing module, configured to determine, for each data frame to be processed, at least one target calibration template for marking the data frame to be processed based on a target type included in the data frame to be processed;

[0032] The labeling module is used to use the at least one target calibration template to perform target labeling on the data frame to be processed to obtain a target labeling result.

[0033] In a possible design of the second aspect, the at least one target calibration template includes at least one of the following:

[0034] Calibration templates from the pre-configured calibration template library;

[0035] Calibration templates generated in real time based on historical target annotation results;

[0036] Calibration templates generated in real time based on template configuration instructions.

[0037] As an example, the processing module is specifically configured to query a pre-configured calibration template library based on the target type included in the data frame to be processed, and determine at least one target calibration template for marking the data frame to be processed in the calibration template library.

[0038] As another example, the processing module is specifically used to obtain a template configuration indication issued by a user, where the template configuration indication is issued by the user based on the target type included in the data frame to be processed, and generate at least one target calibration template for marking the data frame to be processed according to the template configuration indication.

[0039] In another possible design of the second aspect, the labeling module is specifically configured to:

[0040] Determine at least one target in the data frame to be processed according to boundary information of each target in the data frame to be processed, wherein the at least one target includes: a regularly shaped target with a clear boundary and / or an irregularly shaped target with an unclear boundary;

[0041] For at least one target in the data frame to be processed, each target in the data frame to be processed is labeled using at least one target calibration template that matches the at least one target to obtain a target labeling result of the data frame to be processed.

[0042] Optionally, the labeling module is configured to label each target in the data frame to be processed based on at least one target in the data frame to be processed using at least one target calibration template that matches the at least one target, to obtain a target labeling result for the data frame to be processed, specifically:

[0043] The annotation module is specifically used to:

[0044] For a regularly shaped target in the data frame to be processed, using a configured regular graphic in the at least one target calibration template that matches the regularly shaped target, to mark the regularly shaped target in the data frame to be processed; and / or

[0045] For irregularly shaped targets in the data frame to be processed, based on the key point positions marked by the user, the at least one target calibration template and the region expansion algorithm are called to mark the irregularly shaped targets in the data frame to be processed.

[0046] In yet another possible design of the second aspect, the device further includes: a display module;

[0047] The display module is used to display the target labeling result of each data frame to be processed;

[0048] The acquisition module is further configured to acquire user review information on the target labeling result, wherein the review information includes a review pass indication or a modified target labeling result.

[0049] In another possible design of the second aspect, when the data frame to be processed includes at least two targets, the processing module is further used to determine the relationship between the targets in the data frame to be processed based on the at least one target calibration template and the target labeling result.

[0050] In a third aspect, the present application provides an electronic device comprising: a processor, a memory, and computer program instructions stored on the memory and executable on the processor, wherein when the processor executes the computer program instructions, the method described in the first aspect and each possible design of the first aspect is implemented.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods described in the above-mentioned first aspect and each possible design of the first aspect.

[0052] The data labeling method and apparatus provided in the embodiments of the present application obtain at least one data frame to be processed. For each data frame to be processed, based on the target type included in the data frame to be processed, at least one target calibration template is determined for labeling the data frame to be processed. The at least one target calibration template is then used to label the data frame to be processed, obtaining a target labeling result, thereby obtaining information about at least one target present in the data frame to be processed. In this technical solution, targets in the data frame to be processed are labeled based on the configured target calibration template. Since the calibration template is easy to establish and implement, it can accurately reflect the relationship between targets, eliminating the need to train a data labeling model, reducing manual labeling costs, and improving data labeling efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a schematic diagram of an application scenario of the data annotation method provided by this application;

[0054] Figure 2 A flowchart of the first embodiment of the data annotation method provided in this application;

[0055] Figure 3 A flowchart of the second embodiment of the data annotation method provided in this application;

[0056] Figure 4 A schematic diagram of the principle of marking regular-shaped objects in a data frame to be processed in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of the principle of marking irregularly shaped objects in a data frame to be processed in an embodiment of the present application;

[0058] Figure 6 A flowchart of a third embodiment of the data annotation method provided in an embodiment of the present application;

[0059] Figure 7 This is a flow chart of a user reviewing target annotation results in an embodiment of the present application;

[0060] Figure 8 A flowchart of a fourth embodiment of the data annotation method provided in an embodiment of the present application;

[0061] Figure 9 A schematic diagram of the structure of an embodiment of the data annotation device provided by this application;

[0062] Figure 10 A schematic diagram of the structure of an electronic device for implementing a data labeling method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0064] Before introducing the solution of the present application, the terms involved in the embodiments of the present application are first explained:

[0065] Calibration template: A general configuration set that is predefined or created when performing data annotation to describe a combination of multiple information such as a set of target information, non-target information, and other configuration information.

[0066] Region expansion: During the data annotation process, users are sometimes required to manually calibrate the key points of the target. In this case, it is necessary to cooperate with the target detection algorithm to accurately calibrate the target area.

[0067] Calibration attributes: A set of features that describe the target information in the data frame to be processed.

[0068] Correlation relationship: A description of the mutual influence and correlation between different goals.

[0069] With the development of computer technology, the application of machine learning algorithms has become increasingly widespread, and supervised learning algorithms are a commonly used type of algorithm. Supervised learning algorithms typically require large amounts of labeled data to train pre-established data annotation models. The quantity and accuracy of the labeled data directly affect the accuracy of the trained data annotation models.

[0070] Currently, data annotation primarily involves locating and labeling objects in the data frame being processed. Specifically, a pre-trained data annotation model is used to obtain attribute information such as the region, category, and confidence level of the objects of interest, as well as the relationships between objects. To achieve high data annotation accuracy, the data annotation model must be trained. Because labeled data contains a wealth of attribute information, a large amount of data must be labeled and used to train the model.

[0071] The inventors discovered that manual data labeling methods have the following problems: 1. The regional positioning of targets depends on the actual performance of the data labeling model and cannot achieve single-target positioning of the target of interest; 2. Deep learning methods can only locate the position of the target and do not address how to label the target's attributes and features; 3. Deep learning methods are only suitable for labeling single targets and do not address how to label the relationships between targets; 4. The repetitive data labeling work during the training of the data labeling model requires a large amount of manpower and time. In other words, the data labeling model established in manual data labeling solutions can only be applied to the target types used during training, resulting in limited labeling scope, low labeling accuracy, and low labeling efficiency.

[0072] The inventors have discovered in practice that: based on the target types and the relationships between targets included in each data frame in the historical annotation data set, some calibration templates can be pre-configured, and the calibration templates can be used to characterize the target's location information, attribute information, and the association relationship between targets. Therefore, by using target area intelligent detection and multi-target template association calibration technology, the individual characteristics of the target and the association relationship between different targets can be described from multiple dimensions, so that a data annotation method can be designed that can quickly annotate data and obtain rich annotation results to support the normal progress of algorithm iteration.

[0073] An embodiment of the present application provides a data labeling method, which obtains at least one data frame to be processed, and for each data frame to be processed, determines at least one target calibration template for labeling the data frame to be processed based on the target type included in the data frame to be processed, and then uses the at least one target calibration template to perform target labeling on the data frame to be processed, obtaining a target labeling result, thereby obtaining information about at least one target present in the data frame to be processed. In this technical solution, the targets in the data frame to be processed are labeled based on the configured target calibration template. Because the calibration template is easy to establish and implement, it can accurately reflect the relationship between targets, eliminating the need to train a data labeling model, reducing manual labeling costs, and improving data labeling efficiency.

[0074] For example, Figure 1 This is a schematic diagram of the application scenario of the data annotation method provided by this application. Figure 1 As shown, the application scenario may include: an electronic device 11 and at least one data source 12 that can communicate with the electronic device 11. The electronic device 11 can obtain a data frame to be processed from any data source and perform processing such as data labeling.

[0075] For example, in Figure 1 In the application scenario shown, each data source 12 can be a data generating device or a data storage device, which can be set according to the actual scenario and is not further set here.

[0076] In this embodiment, the electronic device 11 may obtain data to be processed from at least one data source. Optionally, the data to be processed may be image data or video data, and each video data may include multiple image data frames.

[0077] Optionally, in actual applications, the data generated or stored by the data source may carry a large amount of noise. In order to improve the efficiency of subsequent data annotation, after the electronic device 11 obtains the data to be processed from at least one data source 12, it can first perform pre-processing such as filtering and denoising on the data to be processed to obtain at least one cleaned data frame to be processed. Then, based on the target type included in the data frame to be processed, a target calibration template for data annotation is determined, and finally the target is labeled using the template. For the specific process of data annotation, please refer to the description in the following embodiment and will not be repeated here.

[0078] Exemplarily, the electronic device 11 may be a device integrating processing and display functions, for example, including: a processor 111 and a display 112 , wherein the processor 111 is used to perform data annotation on the data frame to be processed, and the display 112 is used to present the processing results of the processor 111 . Figure 1 The electronic device in the figure only shows a processor and a display as an example. The actual composition of the electronic device can be determined according to actual conditions and will not be described in detail here.

[0079] It should be noted that Figure 1 This is only a schematic diagram of an application scenario provided by the embodiment of the present application. Figure 1 The equipment included in the Figure 1 The application scenarios shown may also include data storage devices, etc.

[0080] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0081] Figure 2 This is a flow chart of the first embodiment of the data annotation method provided by this application. Figure 2 As shown, the method may include the following steps:

[0082] S201: Obtain at least one data frame to be processed.

[0083] In an embodiment of the present application, when a user has a need for data annotation, the data to be processed is first acquired, and then the data to be processed is frame-processed to obtain at least one data frame to be processed corresponding to the data to be processed.

[0084] Exemplarily, the above-mentioned data frame to be processed can be a frame of image data, or it can be multiple frames of image data obtained by frame processing of video data. Usually, each data frame to be processed has at least one target. Therefore, the electronic device can obtain the target information in the data frame to be processed by performing target positioning and labeling on the data frame to be processed.

[0085] Optionally, the data to be processed may be data obtained by the electronic device from at least one data source based on pre-configured information, or may be data imported into the electronic device by a user from a data storage device. The embodiment of the present application does not limit the method of obtaining the data to be processed, which may be set according to actual needs and will not be elaborated here.

[0086] Optionally, the data source can be of different types, for example, a relational database management system (RMDB), a non-relational database (NoSQL), a file system (file), and a distributed file system (Hadoop distributed file system, HDFS). The present embodiment does not limit the type of data source, which can be determined according to actual conditions and will not be further described here.

[0087] Furthermore, in the embodiments of the present application, the data to be processed obtained by the electronic device may contain low-quality data such as inaccurate data and missing data. For such low-quality data, the electronic device can adopt strict filtering conditions, first cleaning the data to be processed and retaining only data that meets the requirements, so as to minimize the impact of data noise on the subsequent annotation process. The embodiments of the present application do not limit the specific implementation of data cleaning, which can be set according to actual needs and will not be repeated here.

[0088] Optionally, the electronic device may perform target labeling according to the following steps S202 and S203 for each acquired data frame to be processed.

[0089] S202: Determine at least one target calibration template for marking the data frame to be processed based on the target type included in the data frame to be processed.

[0090] In an embodiment of the present application, the electronic device can process each data frame to be processed separately. Specifically, it can automatically detect the target type existing in the data frame to be processed, or it can determine the target type included in the data frame to be processed based on the user's target selection indication, and then determine at least one target calibration template for marking the data frame to be processed based on the target type of the calibration template.

[0091] Optionally, in an embodiment of the present application, each calibration template may represent target type, target location information, attribute characteristics, etc. Target location information may be represented by coordinate values, and attribute characteristics may refer to target category, occupied area, etc.

[0092] For example, the user may pre-establish multiple calibration templates as needed or by analyzing historical data to generate a calibration template library, and then load the calibration template library into the electronic device before performing data annotation.

[0093] As an example, when it is determined that there is no calibration template matching the data frame to be processed in the pre-loaded calibration template library, the electronic device can generate a new calibration template in real time based on the user's template configuration instructions, and can also generate a new calibration template based on historical target annotation results such as historical target annotation types or historical target annotation trajectories, for target annotation of the data frame to be processed.

[0094] From the above analysis, it can be seen that in the embodiment of the present application, the at least one target calibration template may include at least one of the following:

[0095] Calibration templates from the pre-configured calibration template library;

[0096] Calibration templates generated in real time based on historical target annotation results;

[0097] Calibration templates generated in real time based on template configuration instructions.

[0098] As an example, when the at least one target calibration template includes a calibration template in a pre-configured calibration template library, S202 may be implemented by the following steps:

[0099] Based on the target type included in the data frame to be processed, a pre-configured calibration template library is queried, and at least one target calibration template for marking the data frame to be processed is determined in the calibration template library.

[0100] Specifically, when the electronic device analyzes the data frame to be processed and determines the target type included in the data frame to be processed, it can first query a pre-configured calibration template library to determine whether there is a calibration template in the calibration template library for calibrating the target in the data frame to be processed. If so, at least one target calibration template in the calibration template library is selected to facilitate subsequent target labeling. In other words, in embodiments of the present application, the electronic device can obtain the required target calibration template from the configured calibration template library on demand.

[0101] Optionally, in an embodiment of the present application, each calibration template in the calibration template library may include target type information, location information, and attribute information, and may also include a marking algorithm. The embodiment of the present application does not limit the specific information of each calibration template, which can be determined according to actual needs.

[0102] By querying in a pre-configured calibration template library, at least one target calibration template for marking the data frame to be processed is determined. Accordingly, the type information, location information and attribute information of the target in the data frame to be processed can be determined, which simplifies the target labeling process and improves the target labeling efficiency.

[0103] As another example, when the at least one target calibration template includes a calibration template generated in real time based on a template configuration instruction, S202 may be implemented by the following steps:

[0104] A1. Obtain a template configuration instruction issued by a user, where the template configuration instruction is issued by the user based on a target type included in a data frame to be processed;

[0105] A2. Generate at least one target calibration template for marking the data frame to be processed according to the template configuration instruction.

[0106] In an embodiment of the present application, the electronic device discovers through retrieval that for certain target types in the data frame to be processed, the corresponding calibration template cannot be determined in the configured calibration template library. At this time, the user can determine the calibration template creation plan for the target type based on the target type included in the data frame to be processed, thereby issuing a template configuration instruction on the user interface of the electronic device.

[0107] Correspondingly, after obtaining the template configuration indication issued by the user, the electronic device can generate at least one target calibration template for marking the data frame to be processed on the user interaction interface, and mark the target information in the target calibration template, such as the target type, target location information, attribute feature information, the relationship between targets, etc.

[0108] For example, there are two ways for an electronic device to generate a new calibration template: one is to create a new calibration template, and the other is to update the existing target template. The two ways are explained below.

[0109] As an example, when an electronic device needs to create a new calibration template, it first initializes the template creation interface according to the user's instructions. In a possible design, the electronic device can obtain the template name entered by the user on the template creation interface and the relationship description between the target type, and then determine whether the template name is the same as the existing template in the target template library. If it is the same, the user is prompted to re-enter the relationship description between the template name and the target type. If it is not the same, the attribute detection algorithm is configured for the template based on the user's instructions, and a new calibration template is generated. Finally, the template list in the target template library is updated.

[0110] As another example, when an electronic device needs to update an existing calibration template, it first initializes the template creation interface according to the user's instructions, selects the calibration template to be updated, and performs any one of the operations of adding target types, deleting target types, adjusting the order of target types, and corresponding property pages on the calibration template according to the user's instructions. Then, it determines whether the target type list in the calibration template is empty. If so, the user is prompted to re-operate. If not, the attribute detection algorithm is configured for the template based on the user's instructions, and an updated calibration template is generated. Finally, the template list in the target template library is updated.

[0111] At least one target calibration template is generated for marking the data frame to be processed according to the template configuration instruction of the user. The marking template can be generated in real time on demand, thereby improving the accuracy of target marking in the data frame to be processed.

[0112] S203 : Use at least one target calibration template to perform target marking on the data frame to be processed to obtain a target marking result.

[0113] In an embodiment of the present application, after determining the above-mentioned at least one target calibration template, the electronic device locates and labels the targets existing in the data frame to be processed in sequence according to the target type existing in each target calibration template, and then determines the type, position, attributes and other information of at least one target existing in the data frame to be processed based on the target type, target position information and target attribute characteristics represented by each target calibration template, so that the target labeling result can be output.

[0114] Optionally, the target labeling result records the type information, location information, and attribute characteristics of each target. When the data frame to be processed includes at least two targets, the electronic device may further determine the relationship between the targets in the data frame to be processed based on the at least one target calibration template and the target labeling result, so that the target labeling result ultimately output also includes the association relationship between the targets.

[0115] Optionally, since each data frame to be processed can correspond to multiple target calibration templates, that is, the targets in the data frame to be processed can be calibrated using different target calibration templates. Optionally, each target calibration template can also correspond to multiple data frames to be processed, that is, it can be used to mark targets in multiple data frames to be processed.

[0116] Accordingly, in an embodiment of the present application, after an electronic device has marked a target in a data frame to be processed based on a target calibration template, if the current target calibration template still contains areas for calibrating other targets in the data frame to be processed, the target calibration template can be used to continue marking the target. If the current target calibration template does not contain areas for calibrating other targets in the data frame to be processed, the target calibration template can be selected to mark the target until all targets in the data frame to be processed are marked.

[0117] Optionally, after all the targets in the data frame to be processed are marked, other data frames to be processed may be marked in a similar manner, which will not be described in detail here.

[0118] The data labeling method provided in an embodiment of the present application obtains at least one data frame to be processed. For each data frame to be processed, based on the target type included in the data frame to be processed, at least one target calibration template is determined for labeling the data frame to be processed. The data frame to be processed is then labeled with the at least one target calibration template to obtain a target labeling result, thereby obtaining information such as the type, attributes, and location of at least one target present in the data frame to be processed. In this technical solution, the targets in the data frame to be processed are labeled based on the configured target calibration template. Since the calibration template is easy to establish and implement, it can accurately reflect the relationship between targets, eliminating the need to train a data labeling model, reducing manual labeling costs, and improving data labeling efficiency.

[0119] Figure 3 This is a flow chart of the second embodiment of the data annotation method provided by this application. Figure 3 As shown, the above S203 can be implemented by the following steps:

[0120] S301: Determine at least one target in the data frame to be processed according to boundary information of each target in the data frame to be processed.

[0121] The at least one target includes: a regularly shaped target with a clear boundary and / or an irregularly shaped target with an unclear boundary.

[0122] In an embodiment of the present application, the electronic device can employ different processing schemes for different targets in the data frame to be processed. Thus, when the data frame to be processed is labeled using at least one determined target calibration template, at least one target present in the data frame to be processed can first be determined based on the boundary information of the target in the data frame to be processed. Alternatively, if at least two targets are present in the data frame to be processed, all targets present in the data frame to be processed can be classified.

[0123] For example, the at least one target in the data frame to be processed may include a regularly shaped target with clear boundaries, an irregularly shaped target with unclear boundaries, or a combination of regularly shaped targets with clear boundaries and irregularly shaped targets with unclear boundaries. The present embodiment does not limit the type of target included in the data frame to be processed; the type can be determined based on actual scenarios and will not be further elaborated herein.

[0124] S302 : For at least one target in the data frame to be processed, label each target in the data frame to be processed using at least one target calibration template that matches the at least one target, to obtain a target labeling result for the data frame to be processed.

[0125] In an embodiment of the present application, the electronic device can label each target in the data frame to be processed in turn based on the shape of each target and the configuration of the labeling template, using a target calibration template that matches each target according to at least one target in the data frame to be processed determined in S301.

[0126] For example, as an example, S302 may be implemented by the following steps:

[0127] B1. For a regularly shaped target in a data frame to be processed, use a configured regular graphic that matches the regularly shaped target in at least one target calibration template to mark the regularly shaped target in the data frame to be processed.

[0128] Optionally, in an embodiment of the present application, when an electronic device marks a regular-shaped target in a data frame to be processed, it can determine a configured regular graphic that matches the regular-shaped target in at least one target calibration template, and then use the configured regular graphic to mark the area of the regular-shaped target in the data frame to be processed, and finally perform attribute calibration on each regular-shaped target based on the attribute information of the target in the target calibration template.

[0129] For example, Figure 4 Schematic diagram of the principle of marking regular shape objects in the data frame to be processed in the embodiment of the present application. Figure 4As shown, the data frame to be processed is displayed on the human-computer interaction interface of the electronic device. For example, there is a tree, a rabbit and multiple stars (taking 3 as an example) in the data frame to be processed, where the stars are regular-shaped targets with clear boundaries. For the stars, the regular graphics of stars in the target template library can be used to calibrate each star, and then the attribute detection algorithm is called to determine the attribute information of each star, such as the correlation between the position information and each attribute.

[0130] It is understandable that if the electronic device does not call the attribute detection algorithm to calibrate the attribute information, it can be calibrated according to the user's instructions. The specific implementation of attribute calibration can be determined according to actual needs and will not be repeated here.

[0131] B2. For irregularly shaped targets in the data frame to be processed, based on the key point positions marked by the user, at least one target calibration template and a region expansion algorithm are called to mark the irregularly shaped targets in the data frame to be processed.

[0132] For example, in an embodiment of the present application, the irregularly shaped targets in the data frame to be processed mainly refer to complex targets with unclear boundaries. When these irregularly shaped targets need to be labeled, the user usually first calibrates the positions of multiple key points of the irregularly shaped targets. In this way, the electronic device can be based on the key point positions of the target labeled by the user, combined with the target shape in the above-mentioned at least one target calibration template, and call the area expansion algorithm (that is, the area positioning algorithm) to intelligently detect and label the area of the irregularly shaped target in the data frame to be processed, and finally perform attribute calibration on each irregularly shaped target based on the attribute information of the target in the target calibration template.

[0133] Optionally, the region expansion algorithm is also called the automatic region filling algorithm. Region filling is to give a region boundary and assign a specified color code to all pixel units within the boundary. The most commonly used region filling is polygon filling.

[0134] For example, Figure 5 Schematic diagram of the principle of marking irregular shaped objects in the data frame to be processed in the embodiment of the present application. Figure 4 The data to be processed is described as follows: Figure 5 As shown, the data frame to be processed displayed on the human-computer interaction interface of the electronic device includes a tree, a rabbit and multiple stars (taking 3 stars as an example), among which the tree and the rabbit are irregular-shaped targets with unclear boundaries. For the tree and the rabbit, the electronic device first obtains the positions of multiple key points calibrated by the user for the tree and the rabbit, and then calls the target area detection algorithm to determine the area range of the tree and the rabbit, and then annotates the attribute information by calling the attribute detection algorithm, and feeds back the results with relatively low confidence to manual review.

[0135] It is understandable that if the electronic device does not call the attribute detection algorithm to calibrate the attribute information, it can be calibrated according to the user's instructions. The specific implementation of attribute calibration can be determined according to actual needs and will not be repeated here.

[0136] The data labeling method provided in an embodiment of the present application determines at least one target in the data frame to be processed based on the boundary information of each target in the data frame to be processed. For each target in the data frame to be processed, at least one target calibration template matching the at least one target is used to label each target in the data frame to obtain target labeling results for the data frame to be processed. This technical solution can produce relatively accurate target labeling results, laying the foundation for obtaining high-quality labeled data.

[0137] For example, based on the above embodiment, Figure 6 This is a flow chart of the third embodiment of the data annotation method provided in the embodiment of this application. Figure 6 As shown, the method may further include the following steps:

[0138] S601: Display the target labeling result of each data frame to be processed.

[0139] In an embodiment of the present application, since the electronic device automatically labels based on the target calibration template, there may be a problem of inaccurate labeling results. Therefore, in order to further improve the accuracy of target labeling, the electronic device can present the target labeling results of each data frame to be processed on the human-computer interaction interface of the electronic device after obtaining them so that the user can review them.

[0140] S602: Obtain user review information on the target annotation result, where the review information includes a review pass indication or a modified target annotation result.

[0141] In an embodiment of the present application, after the electronic device displays the target labeling result of each data frame to be processed, the user can review the target labeling result to determine whether the target labeling result is correct. If it is correct, a review pass indication is issued. If it is incorrect, the target labeling result is directly modified, that is, the modified target labeling result is issued, so that the electronic device directly saves the modified target labeling result.

[0142] Exemplarily, the modification made by the user to the target annotation result may be a process in which the user refines the target area in the target detection result, so that the obtained target annotation result is closer to the area of the real target in the data frame to be processed.

[0143] For example, Figure 7Schematic diagram of the process of the user reviewing the target marking result in the embodiment of the present application. Figure 7 As shown, in the user interface of the electronic device, assuming that the targets existing in the data frame to be processed include a rabbit, the corresponding area is the area included by the thin solid line. In the target labeling result obtained by the above-mentioned scheme of the present application, the area corresponding to the rabbit is the area included by the dotted line.

[0144] exist Figure 7 In the human-computer interaction interface, the user can see that there is a certain error between the area included by the dotted line and the actual area of the rabbit in the data frame to be processed. Therefore, the user can modify the dotted area by selecting the boundary with the error and moving the boundary of the area corresponding to the dotted line, thereby obtaining a result with a high degree of consistency with the actual area. Specifically, Figure 7 The area corresponding to the medium-thick dashed line.

[0145] The data annotation method provided in the embodiments of this application displays the target annotation results for each data frame to be processed and obtains user review information on the target annotation results. This review information includes a review pass indication or a revised target annotation result. This technical solution further improves the accuracy of target annotation by displaying and obtaining user review information on the target annotation results, laying the foundation for subsequent high-quality annotated data.

[0146] The technical solution of the present application is described above through various embodiments. The technical solution of the present application is explained below through an overall process.

[0147] Figure 8 This is a flow chart of the fourth embodiment of the data annotation method provided in the embodiment of this application. Figure 8 As shown, in an embodiment of the present application, the method may include the following steps:

[0148] S801. Load the calibration template library configured by the user.

[0149] S802: Obtain the data to be processed imported by the user.

[0150] S803: Preprocess the data to be processed to obtain at least one data frame to be processed corresponding to the data to be processed.

[0151] For each data frame to be processed, perform the following steps in sequence:

[0152] S804: Create a new calibration template based on the user's template configuration instruction and store it in the calibration template library.

[0153] S805 : Select at least one target calibration template corresponding to the data frame to be processed from a calibration template library according to the target type in the data frame to be processed.

[0154] S806: Mark the target in the data frame to be processed according to the selected target calibration template.

[0155] Specifically, for a simple target with a clear boundary (a target with a regular shape), S807 is executed; for a complex target with an unclear boundary (a target with an irregular shape), S808 is executed.

[0156] S807: Use the configured regular graphics in the target calibration template to mark the area of the regular-shaped target in the data frame to be processed.

[0157] S808 : Based on the key point positions of the target calibrated by the user, the region expansion algorithm and the target calibration template are called to mark the region of the irregularly shaped target in the data frame to be processed.

[0158] S809: Attribute calibration is performed on the features of the located target.

[0159] S810 , determining whether there is any graphic used to mark the target in the data frame to be processed in the selected target calibration template; if so, proceed to S806 ; if not, proceed to S811 .

[0160] S811, confirm that the calibration of the selected calibration template is completed, and determine whether the current data frame to be processed is labeled. If not, select the next target calibration template and go to S804; if so, go to S812;

[0161] S812: Determine that the marking of the current data frame to be processed is completed. On the one hand, the process may go to S803 to continue marking the next data frame to be processed. On the other hand, the process may execute S813.

[0162] S813: Export the target labeling result of the data frame to be processed.

[0163] The target labeling result includes information about at least one target in the data frame to be processed. Optionally, the information may include: location information, attribute characteristics, and relationships between targets.

[0164] The technical solution of the present application can achieve accurate positioning of the target area, calibration of attribute features and description of the relationship between targets by associating the configured calibration template with the data to be processed for target labeling. For complex targets with unclear boundaries, the region expansion algorithm can be integrated to automatically detect and support users to manually refine the precise region where the target is located, thereby significantly improving the data labeling efficiency and significantly improving the data labeling dimension, such as the region positioning of the target, attribute feature calibration, and the relationship between targets.

[0165] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0166] Figure 9 This is a schematic diagram of the structure of an embodiment of the data annotation device provided by this application. The device can be integrated into an electronic device or implemented by an electronic device. Figure 9 As shown, the device may include: an acquisition module 901, a processing module 902 and a labeling module 903.

[0167] The acquisition module 901 is configured to acquire at least one data frame to be processed;

[0168] The processing module 902 is configured to determine, for each data frame to be processed, at least one target calibration template for marking the data frame to be processed based on the target type included in the data frame to be processed;

[0169] The labeling module 903 is configured to perform target labeling on the data frame to be processed using the at least one target labeling template to obtain a target labeling result, wherein the target labeling result includes information of at least one target in the data frame to be processed.

[0170] In one possible design of an embodiment of the present application, the at least one target calibration template includes at least one of the following:

[0171] Calibration templates from the pre-configured calibration template library;

[0172] Calibration templates generated in real time based on historical target annotation results;

[0173] Calibration templates generated in real time based on template configuration instructions.

[0174] As an example, the processing module 902 is specifically configured to query a pre-configured calibration template library based on the target type included in the data frame to be processed, and determine at least one target calibration template for marking the data frame to be processed in the calibration template library.

[0175] As another example, the processing module 902 is specifically used to obtain a template configuration indication issued by a user, where the template configuration indication is issued by the user based on the target type included in the data frame to be processed, and generate at least one target calibration template for marking the data frame to be processed according to the template configuration indication.

[0176] In another possible design of the embodiment of the present application, the marking module 903 is specifically configured to:

[0177] Determine at least one target in the data frame to be processed according to boundary information of each target in the data frame to be processed, wherein the at least one target includes: a regularly shaped target with a clear boundary and / or an irregularly shaped target with an unclear boundary;

[0178] For at least one target in the data frame to be processed, each target in the data frame to be processed is labeled using at least one target calibration template that matches the at least one target to obtain a target labeling result of the data frame to be processed.

[0179] Optionally, the labeling module 903 is configured to label each target in the data frame to be processed using at least one target calibration template that matches the at least one target, to obtain a target labeling result for the data frame to be processed, specifically:

[0180] The marking module 903 is specifically used to:

[0181] For a regularly shaped target in the data frame to be processed, using a configured regular graphic in the at least one target calibration template that matches the regularly shaped target, to mark the regularly shaped target in the data frame to be processed; and / or

[0182] For irregularly shaped targets in the data frame to be processed, based on the key point positions marked by the user, the at least one target calibration template and the region expansion algorithm are called to mark the irregularly shaped targets in the data frame to be processed.

[0183] Reference Figure 9 As shown, in another possible design of the embodiment of the present application, the apparatus further includes: a display module 904;

[0184] Display module 904, used to display the target labeling result of each data frame to be processed;

[0185] The acquisition module 901 is further configured to acquire user review information on the target labeling result, where the review information includes a review pass indication or a modified target labeling result.

[0186] In another possible design of an embodiment of the present application, when the data frame to be processed includes at least two targets, the processing module 902 is further used to determine the relationship between the targets in the data frame to be processed based on the at least one target calibration template and the target labeling result.

[0187] The device provided in the embodiment of the present application can be used to perform Figures 2 to 8 The implementation principles and technical effects of the methods in the illustrated embodiments are similar and will not be described in detail here.

[0188] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by a processing element; they can also all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the processing module can be a separately established processing element, or it can be integrated into a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called by a processing element of the above device to perform the functions of the above processing module. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0189] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code on a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0190] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)).

[0191] Figure 10 This is a schematic diagram of the structure of an electronic device for implementing a data tagging method provided in an embodiment of the present application. Figure 10 As shown, the electronic device may include: a processor 1001, a memory 1002, a communication interface 1003 and a system bus 1004. The memory 1002 and the communication interface 1003 are connected to the processor 1001 via the system bus 1004 and communicate with each other. The memory 1002 is used to store computer-executable instructions, and the communication interface 1003 is used to communicate with other devices. When the processor 1001 executes the above-mentioned computer-executable instructions, the above-mentioned Figures 2 to 8 A scheme of the embodiment shown.

[0192] Optionally, the electronic device may further include: a human-computer interaction interface 1005. The human-computer interaction interface 1005 is used to display the target marking result of the data frame to be processed, and to receive user review information on the target marking result.

[0193] Should Figure 10The system bus mentioned in the figure may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory (non-volatile memory), such as at least one disk storage.

[0194] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0195] Optionally, the embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which, when executed on a computer, enable the computer to execute the above-mentioned Figures 2 to 8 The method of the embodiment shown.

[0196] Optionally, the embodiment of the present application further provides a chip for executing instructions, the chip being used to execute the above Figures 2 to 8 The method of the embodiment shown.

[0197] The embodiment of the present application further provides a computer program product, wherein the computer program product includes a computer program, wherein the computer program is stored in a computer-readable storage medium, and at least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, the above-mentioned Figures 2 to 8 The method of the embodiment shown.

[0198] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula, the character " / " indicates that the previous and next associated objects are in a "division" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items.

[0199] It is understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not intended to limit the scope of the embodiments of the present application. In the embodiments of the present application, the order of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data annotation method, characterized in that: include: Obtain at least one data frame to be processed; For each data frame to be processed, determining at least one target calibration template for marking the data frame to be processed based on the target type included in the data frame to be processed; Performing target labeling on the data frame to be processed using the at least one target labeling template to obtain a target labeling result; The at least one target calibration template includes at least one of the following: Calibration templates from the pre-configured calibration template library; Calibration templates generated in real time based on historical target annotation results; Calibration templates generated in real time based on template configuration instructions; When the target in the data frame to be processed is a regularly shaped target, marking the regularly shaped target in the data frame to be processed using a configured regular graphic that matches the regularly shaped target in the at least one target calibration template; and / or When the target in the data frame to be processed is an irregularly shaped target, the at least one target calibration template and the region expansion algorithm are called based on the key point positions marked by the user to mark the irregularly shaped target in the data frame to be processed.

2. The method according to claim 1, characterized in that The determining, based on the target type included in the data frame to be processed, at least one target calibration template for marking the data frame to be processed includes: Based on the target type included in the data frame to be processed, a pre-configured calibration template library is queried, and at least one target calibration template for marking the data frame to be processed is determined in the calibration template library.

3. The method according to claim 1, characterized in that The determining, based on the target type included in the data frame to be processed, at least one target calibration template for marking the data frame to be processed includes: Acquire a template configuration instruction issued by a user, where the template configuration instruction is issued by the user based on a target type included in the data frame to be processed; At least one target calibration template is generated according to the template configuration instruction for marking the data frame to be processed.

4. The method according to claim 1, wherein The using the at least one target calibration template to perform target labeling on the data frame to be processed to obtain a target labeling result includes: Determine at least one target in the data frame to be processed according to boundary information of each target in the data frame to be processed, wherein the at least one target includes: a regularly shaped target with a clear boundary and / or an irregularly shaped target with an unclear boundary; For at least one target in the data frame to be processed, each target in the data frame to be processed is labeled using at least one target calibration template that matches the at least one target to obtain a target labeling result of the data frame to be processed.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Display the target labeling results of each data frame to be processed; Acquire user review information on the target labeling result, where the review information includes a review pass indication or a modified target labeling result.

6. The method according to any one of claims 1 to 4, characterized in that When the data frame to be processed includes at least two targets, the method further includes: Determine the relationship between the objects in the data frame to be processed according to the at least one object calibration template and the object labeling result.

7. A data labeling device, characterized in that: include: An acquisition module, configured to acquire at least one data frame to be processed; a processing module, configured to determine, for each data frame to be processed, at least one target calibration template for marking the data frame to be processed based on a target type included in the data frame to be processed; a labeling module, configured to perform target labeling on the data frame to be processed using the at least one target labeling template to obtain a target labeling result; The at least one target calibration template includes at least one of the following: a calibration template in a pre-configured calibration template library; a calibration template generated in real time based on historical target annotation results; a calibration template generated in real time based on a template configuration instruction; The labeling module is specifically used to, when the target in the data frame to be processed is a regularly shaped target, use the configured regular graphics in the at least one target calibration template that matches the regularly shaped target to label the regularly shaped target in the data frame to be processed; and / or, when the target in the data frame to be processed is an irregularly shaped target, based on the key point positions marked by the user, call the at least one target calibration template and the region expansion algorithm to label the irregularly shaped target in the data frame to be processed.

8. The device according to claim 7, characterized in that The processing module is specifically configured to query a pre-configured calibration template library based on the target type included in the data frame to be processed, and determine at least one target calibration template in the calibration template library for marking the data frame to be processed.

9. The device according to claim 7, characterized in that The processing module is specifically used to obtain a template configuration instruction issued by a user, where the template configuration instruction is issued by the user based on the target type included in the data frame to be processed, and generate at least one target calibration template for marking the data frame to be processed according to the template configuration instruction.

Images