Data labeling method and related products

Abstract

The application discloses a data labeling method and related products, and belongs to the technical field of data processing. The method comprises the following steps: obtaining a first edge line and a second edge line of a target object contained in a target image; the first edge line and the second edge line are different; obtaining a first contour of the target object according to the first edge line; and correcting the edge line of the first contour by using the second edge line to obtain a second contour of the target object. The method can greatly improve the accuracy and efficiency of the labeling result by proposing a deep learning method based on a preselected frame to perform edge detection on the target object in the target image, obtaining the first contour of the target object, and then introducing weak semantic information to convert the correction of the edge points of the first contour of the target object into the correction of the edge line of the first contour of the target object, and finally obtaining the second contour of the target object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data labeling method and related products. BACKGROUND

[0002] A label is a data form used to describe the characteristics of a business entity. Through the label, the business entity can be characterized and its characteristics can be reflected from multiple angles. The label can be obtained through data labeling. Data labeling is the act of labeling data by data labeling personnel with the help of a marking tool. The types of data labeling include image labeling, speech labeling, text labeling, etc. Among them, image labeling can be applied to face recognition and automatic driving vehicle recognition application scenarios. Data labeling personnel need to outline the target markers with different colors, and then label the corresponding outlines. The label is used to summarize the target markers in the outline so that the model can automatically identify different markers in the image. With the wide use of data-driven image processing technology in various fields, more and more application scenarios need to use labels to automatically identify different markers contained in the image. However, labels need to be obtained through data labeling. Therefore, how to optimize the accuracy and efficiency of data labeling is crucial.

[0003] At present, an interactive image segmentation labeling method is usually used for data labeling. Specifically, an initial region is given manually, a segmentation algorithm is used to segment the target object in the initial region to obtain the edge points of the target object, and then the segmentation result of the target object is corrected by correcting the edge points.

[0004] However, the above labeling method needs to manually correct the edge points one by one, which is time-consuming and laborious, and the accuracy and efficiency of the labeling result are low. SUMMARY

[0005] Embodiments of the present application provide a data labeling method and related products. The method includes proposing a pre-selected frame-based deep learning method to perform edge detection on a target object in a target image to obtain a first outline of the target object, and introducing weak semantic information to convert the correction of the edge points of the first outline of the target object into the correction of the edge lines of the first outline of the target object, and correct the first outline of the target object to obtain a second outline of the target object. The accuracy and efficiency of the labeling result can be greatly improved.

[0006] In a first aspect, embodiments of the present application provide a data labeling method, which includes:

[0007] obtaining a first edge line and a second edge line of a target object contained in a target image; the first edge line and the second edge line are different;

[0008] obtaining a first outline of the target object according to the first edge line;

[0009] correct the edge line of the first contour by using the second edge line to obtain the second contour of the target object.

[0010] In the embodiments of the present application, different edge lines of a target object contained in a target image can be obtained by processing the target image, such as a first edge line and a second edge line. Then, the target object is labeled based on the first edge line to obtain a first contour of the target object. Then, the edge line of the first contour obtained by the labeling is corrected by using the second edge line to obtain a second contour of the target object.

[0011] In the current data labeling method, a segmentation algorithm is used to segment a target object to obtain edge points of the target object, and then the contour of the target object is corrected by correcting the edge points. Therefore, the edge points need to be corrected manually, which is time-consuming and laborious, and the accuracy and efficiency of the labeling result are low.

[0012] Compared with the current commonly used data labeling method, the data labeling method provided in the embodiments of the present application labels the target object by using the first edge line, and then corrects the edge line of the first contour obtained by the labeling by using the second edge line, so that the correction of the edge points of the first contour of the target object is converted into the correction of the edge line of the first contour of the target object, and the second contour of the target object is obtained by correction, which can greatly improve the accuracy and efficiency of the labeling result.

[0013] In a possible implementation, the correcting the edge line of the first contour by using the second edge line comprises:

[0014] The contour formed by the second edge line and the edge line of the first contour is determined as the second contour.

[0015] In the embodiments of the present application, a possible implementation of correcting the edge line of the first contour by using the second edge line is provided. The contour formed by the second edge line and the edge line of the first contour is determined as the second contour of the target object, and the correction method improves the accuracy and efficiency of the labeling result.

[0016] In a possible implementation, before the first edge line and the second edge line of the target object contained in the target image are obtained, the method further comprises:

[0017] The target image is segmented to obtain the first edge point of the target object; the segmentation processing is used to separate the target object contained in the target image;

[0018] The target object is edge detected to obtain the second edge point of the target object; the edge detection is used to separate the edge of the target object;

[0019] obtaining a first edge line and a second edge line of a target object contained in a target image, comprising:

[0020] determining the first edge line and the second edge line according to the first edge point and the second edge point.

[0021] In the embodiments of the present application, a possible implementation of obtaining a first edge line and a second edge line of a target object contained in a target image is provided. That is, before determining the first edge line and the second edge line of the target object, the target object in the target image is first separated out through segmentation processing of the target image to obtain a first edge point of the target object. The segmentation processing here includes a non-deep learning based image segmentation method (such as a segmentation method based on a watershed algorithm, graph theory, etc.) and a deep learning based image segmentation method. The main difference between the two is that the deep learning based image segmentation method needs to use a convolutional neural network. After the target object in the target image is separated out, edge detection is performed on the target object to separate the edges of the target object and obtain a second edge point of the target object. The edge detection here includes non-preselected frame deep learning based edge detection and preselected frame deep learning based edge detection. The difference between the two is whether a preselected frame is used. The preselected frame deep learning based edge detection is not to detect whether a pixel point on the target image is an edge point, but to detect the position and category of the edge point corresponding to each preselected frame. The first edge point and the second edge point described above are both edge points of the target object, but they are obtained in different ways. Therefore, the first edge point and the second edge point described above are not exactly the same. According to the first edge point and the second edge point, the first edge line and the second edge line of the target object described above can be determined. Since the first edge point and the second edge point described above are not exactly the same, the first edge line and the second edge line obtained accordingly are also not exactly the same.

[0022] Through the implementation of obtaining the first edge line and the second edge line provided in the embodiments of the present application, the accuracy of the edge line of the target object obtained can be improved, and it is beneficial to subsequently convert the correction of the edge points of the first contour of the target object into the correction of the edge line of the first contour of the target object, thereby improving the accuracy and efficiency of the labeling result.

[0023] In a possible implementation, the determining the first edge line and the second edge line according to the first edge point and the second edge point comprises:

[0024] obtaining the first edge line according to the first edge point and the second edge point; the points on the first edge line are the edge points coinciding with the first edge point and the second edge point;

[0025] The second edge line is obtained according to the first edge point, the second edge point and the first edge line.

[0026] In the embodiments of the present application, a possible specific implementation of determining a first edge line and a second edge line according to a first edge point and a second edge point is provided. A line formed by edge points in which the first edge point and the second edge point coincide is determined as the first edge line, that is, the first edge line is located on the edge result obtained by the above-mentioned segmentation processing and edge detection, and the first edge line can be obtained by the above-mentioned segmentation processing and edge detection. The second edge line is determined according to the first edge point, the second edge point and the first edge line. The first edge line and the second edge line obtained by the embodiments of the present application have higher precision, and the accuracy and efficiency of the result labeled according to the first edge line and the second edge line are also higher.

[0027] In a possible implementation, the second edge line is obtained according to the first edge point, the second edge point and the first edge line, and the method comprises the following steps.

[0028] A third edge line is obtained according to the first edge point and the second edge point; the points on the third edge line are the first edge point or the second edge point, and are not the edge points in which the first edge point and the second edge point coincide;

[0029] The third edge line having two or more intersection points with the first edge line is determined as the second edge line.

[0030] In the embodiments of the present application, a possible specific implementation of determining a second edge line according to a first edge point, a second edge point and a first edge line is provided. A line formed by removing the edge points in which the first edge point and the second edge point coincide from the union of the first edge point and the second edge point is determined as a third edge line, that is, the third edge line is located on the edge result obtained by the above-mentioned segmentation processing or on the edge result obtained by the above-mentioned edge detection, but not on the edge results obtained by the segmentation processing and the edge detection. The third edge line having two or more intersection points with the first edge line is determined as the second edge line. The second edge line obtained by the embodiments of the present application has higher precision, and the first contour edge point of the target object can be corrected to the first contour edge line of the target object by using the second edge line, which is beneficial to improving the accuracy and efficiency of the correction.

[0031] In a possible implementation, the edge detection is performed on the target object to obtain a second edge point of the target object, and the method comprises the following steps.

[0032] A label of the target object is determined according to the position of the edge point on the edge line of the target object.

[0033] According to the label of the target object, the edge points on the edge line of the target object are adjusted to obtain the second edge points.

[0034] In the embodiments of the present application, a possible implementation of edge detection is provided. The edge detection is performed on the target object to separate the edge of the target object to obtain the second edge points of the target object. In the edge detection based on the preselected frame deep learning, the position and category of the edge points corresponding to each preselected frame are detected instead of detecting whether the pixel points on the target image are edge points. That is, the category label of the target object is determined according to the position of the edge points corresponding to each preselected frame on the edge line, and then the edge points corresponding to each preselected frame on the edge line are adjusted according to the category label, the edge direction and edge distance are assigned to the edge points corresponding to each preselected frame, the edge points corresponding to each preselected frame are moved by the edge distance in the edge direction, and the second edge points are obtained. The edge detection based on the preselected frame deep learning can make the position of the obtained second edge points more accurate, and the edge line of the target object obtained accordingly is also more accurate, which is beneficial to improve the accuracy and efficiency of the labeling result.

[0035] In a possible implementation, the determining of the label of the target object according to the position of the edge points on the edge line of the target object comprises:

[0036] In a case where the number of edge points located on the first label is greater than the number of edge points located on the second label, the first label is determined as the label of the target object.

[0037] Alternatively, in a case where the number of edge points located on the first label is less than the number of edge points located on the second label, the second label is determined as the label of the target object.

[0038] Alternatively, in a case where the number of edge points located on the first label is equal to the number of edge points located on the second label, the label of the target object is determined according to the key degree of the first label and the key degree of the second label.

[0039] In the embodiments of the present application, a possible implementation of determining the category label of the target object is provided. The label of the category in which the edge points corresponding to each preselected frame on the edge line of the target object are located is determined as the category label of the target object, that is, the category of the preselected frame is the category in which the points on the edge line are located. For example, in the case where the number of edge points located on the first label is greater than the number of edge points located on the second label, the first label is determined as the category label of the target object, and by analogy, in the case where the number of edge points located on the first label is less than the number of edge points located on the second label, the second label is determined as the category label of the target object. In the case where the number of edge points located on the first label is equal to the number of edge points located on the second label, the category label of the target object is further determined according to the key degree of the label. The embodiments of the present application can improve the accuracy and efficiency of determining the category of the target object by assigning categories to the preselected frame to determine the category label of the target object, and adjusting the edge points on the edge line of the target object according to the category label, so that the position accuracy of the obtained second edge points is higher.

[0040] In a possible implementation, the segmentation processing on the target image is performed to obtain the first edge points of the target object, including:

[0041] The target image is input into a convolutional neural network for the segmentation processing to obtain the first edge points.

[0042] In the embodiments of the present application, the target image is input into a convolutional neural network for segmentation processing by using an image segmentation method based on deep learning. Compared with the commonly used interactive image segmentation method, the edge points of the target object obtained by the present method are more accurate, and the segmentation efficiency is higher.

[0043] In a possible implementation, the label of the target object is determined according to the key degree of the first label and the key degree of the second label, including:

[0044] In the case where the key degree of the first label is higher than the key degree of the second label, the first label is determined as the label of the target object;

[0045] In the case where the key degree of the second label is higher than the key degree of the first label, the second label is determined as the label of the target object.

[0046] In the embodiments of the present application, a possible implementation of determining the category label of the target object is provided. That is, in the case that the number of edge points located on the first label is equal to the number of edge points located on the second label, the category label of the target object is further determined according to the key degree of the label. For example, in the case that the key degree of the first label is higher than the key degree of the second label, the first label is determined as the category label of the target object, and similarly, in the case that the key degree of the second label is higher than the key degree of the first label, the second label is determined as the category label of the target object. The embodiments of the present application can improve the accuracy and efficiency of determining the category of the target object by further comparing the key degrees of the labels to determine the category label of the target object, and accordingly adjusting the edge points on the edge line of the target object can make the position of the second edge point more accurate.

[0047] In a possible implementation, the method further includes:

[0048] obtaining a first bounding box and a second bounding box of the target object according to target data of the target radar; the first bounding box and the second bounding box are different, and the target data includes data obtained by the target radar detecting the target object;

[0049] obtaining a first contour of the target object according to the first bounding box;

[0050] correcting the bounding box of the first contour by using the second bounding box to obtain a second contour of the target object.

[0051] In the embodiments of the present application, different processing is performed on the target data collected by the target radar detecting the target object, and different bounding boxes of the target object, such as the first bounding box and the second bounding box, can be obtained. Then, the target object is initially labeled based on the first bounding box to obtain the first contour of the target object. Then, the bounding box of the first contour obtained by the initial labeling is corrected by using the second bounding box to obtain the second contour of the target object.

[0052] The data labeling method provided by the embodiments of the present application is not only suitable for labeling two-dimensional image data, but also suitable for labeling three-dimensional radar data. The correction of the edge points of the first contour of the target object is converted into the correction of the bounding box of the first contour of the target object, and the second contour of the target object is obtained by correction, which can greatly improve the accuracy and efficiency of the labeling result.

[0053] In a possible implementation, the correcting the bounding box of the first contour by using the second bounding box includes:

[0054] the contour formed by the second bounding box and the bounding box of the first contour is determined as the second contour.

[0055] In a possible implementation, before the target object first and second bounding boxes are obtained according to the target data of the target radar, the method further includes:

[0056] performing segmentation processing on the target object to obtain a first edge surface of the target object; the segmentation processing is used to separate the target object from a detection area of the target radar;

[0057] performing edge detection on the target object to obtain a second edge surface of the target object; the edge detection is used to separate edges of the target object;

[0058] the target object first and second bounding boxes are obtained according to the target data of the target radar, and include:

[0059] the first and second bounding boxes are determined according to the first and second edge surfaces.

[0060] In a possible implementation, the first and second bounding boxes are determined according to the first and second edge surfaces, and include:

[0061] the first bounding box is obtained according to the first and second edge surfaces; the edge surface on the first bounding box is an edge surface on which the first and second edge surfaces coincide;

[0062] the second bounding box is obtained according to the first edge surface, the second edge surface, and the first bounding box.

[0063] In a possible implementation, the second bounding box is obtained according to the first edge surface, the second edge surface, and the first bounding box, and includes:

[0064] a third bounding box is obtained according to the first and second edge surfaces; the surface on the third bounding box is the first edge surface or the second edge surface, and is not an edge surface on which the first and second edge surfaces coincide;

[0065] the third bounding box with two or more surfaces intersecting with the first bounding box is determined as the second bounding box.

[0066] In a possible implementation, the edge detection on the target object to obtain the second edge surface of the target object includes:

[0067] a label of the target object is determined according to a position of an edge surface on a bounding box of the target object;

[0068] the edge surface on the bounding box of the target object is adjusted to obtain the second edge surface according to the label of the target object.

[0069] In a possible implementation, the label of the target object is determined according to positions of the edge faces on the bounding box of the target object, including:

[0070] In a case where the number of edge faces located on the first label is greater than the number of edge faces located on the second label, the first label is determined as the label of the target object;

[0071] Or, in a case where the number of edge faces located on the first label is less than the number of edge faces located on the second label, the second label is determined as the label of the target object;

[0072] Or, in a case where the number of edge faces located on the first label is equal to the number of edge faces located on the second label, the label of the target object is determined according to the key degree of the first label and the key degree of the second label.

[0073] In a second aspect, an embodiment of the present application provides a data labeling device, which includes:

[0074] A determination unit is configured to obtain a first edge line and a second edge line of a target object contained in a target image; the first edge line and the second edge line are different;

[0075] The determination unit is further configured to obtain a first contour of the target object according to the first edge line;

[0076] A correction unit is configured to correct an edge line of the first contour by using the second edge line, to obtain a second contour of the target object.

[0077] In a possible implementation, the correction unit is specifically configured to determine a contour formed by the second edge line and the edge line of the first contour as the second contour.

[0078] In a possible implementation, the device further includes:

[0079] A segmentation unit is configured to perform segmentation processing on the target image to obtain a first edge point of the target object; the segmentation processing is configured to separate the target object contained in the target image;

[0080] An edge detection unit is configured to perform edge detection on the target object to obtain a second edge point of the target object; the edge detection is configured to separate an edge of the target object;

[0081] The determination unit is specifically configured to determine the first edge line and the second edge line according to the first edge point and the second edge point.

[0082] In a possible implementation, the determining unit is further configured to obtain the first edge line according to the first edge point and the second edge point; and a point on the first edge line is an edge point where the first edge point and the second edge point coincide.

[0083] The determining unit is further configured to obtain the second edge line according to the first edge point, the second edge point, and the first edge line.

[0084] In a possible implementation, the determining unit is further configured to obtain a third edge line according to the first edge point and the second edge point; a point on the third edge line is the first edge point or the second edge point, and is not an edge point where the first edge point and the second edge point coincide.

[0085] The determining unit is further configured to determine the third edge line that has two or more intersection points with the first edge line as the second edge line.

[0086] In a possible implementation, the edge detecting unit is configured to determine a label of the target object according to positions of edge points on edge lines of the target object.

[0087] The edge detecting unit is further configured to adjust the edge points on the edge lines of the target object to obtain the second edge point according to the label of the target object.

[0088] In a possible implementation, the edge detecting unit is further configured to determine a first label as the label of the target object in a case where a number of edge points located on the first label is greater than a number of edge points located on a second label.

[0089] Alternatively, the edge detecting unit is further configured to determine the second label as the label of the target object in a case where the number of edge points located on the first label is less than the number of edge points located on the second label.

[0090] Alternatively, the edge detecting unit is further configured to determine the label of the target object according to a key degree of the first label and a key degree of the second label in a case where the number of edge points located on the first label is equal to the number of edge points located on the second label.

[0091] In a possible implementation, the segmenting unit is configured to input the target image into a convolutional neural network to perform the segmentation processing, and obtain the first edge point.

[0092] In a possible implementation, the determining unit is specifically further configured to determine the first label as the label of the target object in a case where the key degree of the first label is higher than the key degree of the second label.

[0093] The determining unit is specifically further configured to determine the second label as the label of the target object in a case where the key degree of the second label is higher than the key degree of the first label.

[0094] In a possible implementation, the determining unit is configured to obtain a first bounding box and a second bounding box of a target object according to target data of a target radar, wherein the first bounding box and the second bounding box are different, and the target data includes data obtained by the target radar detecting the target object.

[0095] The determining unit is further configured to obtain a first contour of the target object according to the first bounding box.

[0096] The correcting unit is configured to correct a bounding box of the first contour by using the second bounding box to obtain a second contour of the target object.

[0097] In a possible implementation, the correcting unit is specifically configured to determine a contour formed by the second bounding box and the bounding box of the first contour as the second contour.

[0098] In a possible implementation, the segmenting unit is configured to perform segmentation processing on the target object to obtain a first edge surface of the target object, wherein the segmentation processing is configured to separate the target object from a detection area of the target radar.

[0099] The edge detecting unit is configured to perform edge detection on the target object to obtain a second edge surface of the target object, wherein the edge detection is configured to separate edges of the target object.

[0100] The determining unit is specifically configured to determine the first bounding box and the second bounding box according to the first edge surface and the second edge surface.

[0101] In a possible implementation, the determining unit is specifically further configured to obtain the first bounding box according to the first edge surface and the second edge surface, wherein an edge surface on the first bounding box is an edge surface on which the first edge surface and the second edge surface coincide.

[0102] The determining unit is specifically further configured to obtain the second bounding box according to the first edge surface, the second edge surface, and the first bounding box.

[0103] In a possible implementation, the determining unit is specifically further configured to obtain a third edge frame according to the first edge face and the second edge face; a face on the third edge frame is the first edge face or the second edge face, and is not an edge face that coincides with the first edge face and the second edge face.

[0104] The determining unit is specifically further configured to determine the third edge frame with two or more faces intersecting with the first edge frame as the second edge frame.

[0105] In a possible implementation, the edge detecting unit is specifically configured to determine a label of the target object according to positions of edge faces on the edge frame of the target object.

[0106] The edge detecting unit is specifically further configured to adjust the edge faces on the edge frame of the target object to obtain the second edge face according to the label of the target object.

[0107] In a possible implementation, the edge detecting unit is specifically further configured to determine a first label as the label of the target object in a case where a number of edge faces located on the first label is greater than a number of edge faces located on a second label.

[0108] Alternatively, the edge detecting unit is specifically further configured to determine the second label as the label of the target object in a case where the number of edge faces located on the first label is less than the number of edge faces located on the second label.

[0109] Alternatively, the edge detecting unit is specifically further configured to determine the label of the target object according to a key degree of the first label and a key degree of the second label in a case where the number of edge faces located on the first label is equal to the number of edge faces located on the second label.

[0110] As to the technical effects brought by the second aspect and any possible implementation, reference can be made to the introduction of the technical effects corresponding to the first aspect and the corresponding implementation.

[0111] In a third aspect, an embodiment of the present application provides a data labeling apparatus, which comprises a processor and a memory; the memory is configured to store computer programs or instructions; the processor is configured to execute the computer programs or instructions stored in the memory, so that the data labeling apparatus executes the method according to the first aspect and any possible implementation.

[0112] Optionally, the data labeling apparatus further comprises a transceiver, which is configured to receive data or send data.

[0113] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which is configured to store a computer program. When the computer program is executed, the method in the first aspect and any possible implementation manner thereof is implemented.

[0114] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the method in the first aspect and any possible implementation manner thereof is implemented.

[0115] In a sixth aspect, an embodiment of the present application provides a chip, which includes a processor. When the processor executes an instruction, the chip executes the method in the first aspect and any possible implementation manner thereof.

[0116] Optionally, the chip further includes a communication interface, which is configured to input or output data.

[0117] In a seventh aspect, an embodiment of the present application provides a terminal, which includes at least one data labeling apparatus in the second aspect or the third aspect, or the chip in the sixth aspect.

[0118] In an eighth aspect, an embodiment of the present application provides a server, which includes at least one data labeling apparatus in the second aspect or the third aspect, or the chip in the sixth aspect.

[0119] Optionally, in the process of executing the method in the first aspect and any possible implementation manner thereof, the processor can be a processor specially configured to execute the method, or a processor configured to execute the computer program in the memory, such as a general processor. The memory can be a non-transitory memory, such as a read only memory (ROM), which can be integrated on the same chip with the processor, or respectively arranged on different chips. The type of the memory and the arrangement manner of the memory and the processor are not limited in the embodiments of the present application.

[0120] In a possible implementation manner, the at least one memory is located outside the apparatus.

[0121] In another possible implementation manner, the at least one memory is located inside the apparatus.

[0122] In another possible implementation manner, part of the at least one memory is located inside the apparatus, and another part of the at least one memory is located outside the apparatus.

[0123] In the present application, the processor and the memory can also be integrated in one device, that is, the processor and the memory can also be integrated together.

[0124] In the embodiments of the present application, by proposing a deep learning method based on a preselected frame to perform edge detection on a target object in a target image, a first contour of the target object is obtained, and weak semantic information is introduced to convert the correction of edge points of the first contour of the target object into the correction of edge lines of the first contour of the target object, and a second contour of the target object is obtained by correction, which can greatly improve the accuracy and efficiency of the labeling result. BRIEF DESCRIPTION OF DRAWINGS

[0125] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly introduced as follows. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art.

[0126] Figure 1 An effect diagram of data labeling provided by the embodiments of the present application;

[0127] Figure 2 An architecture diagram of a data labeling system provided by the embodiments of the present application;

[0128] Figure 3 An application scenario diagram of data labeling provided by the embodiments of the present application;

[0129] Figure 4 A flow diagram of a data labeling method provided by the embodiments of the present application;

[0130] Figure 5a An effect diagram of data labeling provided by the embodiments of the present application;

[0131] Figure 5b An effect diagram of data labeling provided by the embodiments of the present application;

[0132] Figure 5c An effect diagram of data labeling provided by the embodiments of the present application;

[0133] Figure 6a An architecture diagram of a data labeling system provided by the embodiments of the present application;

[0134] Figure 6b An architecture diagram of a data labeling system provided by the embodiments of the present application;

[0135] Figure 6c An architecture diagram of a data labeling system provided by the embodiments of the present application;

[0136] Figure 7 Another data labeling method provided by an embodiment of the present application;

[0137] Figure 8 A structural schematic diagram of a data labeling device provided by an embodiment of the present application;

[0138] Figure 9 A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0139] In order to make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described below with reference to the drawings.

[0140] The terms "first" and "second" and the like in the specification of the present application, claims and drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device, etc. including a series of steps or units, is not limited to the listed steps or units, but can optionally include other steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device, etc.

[0141] In this document, "embodiment" means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor is it independent or alternative to other embodiments. Those skilled in the art can explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0142] It should be understood that in the present application, "at least one" means one or more, "multiple" means two or more, "at least two" means two or three and three or more, and "and / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases of only A, only B and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b and c can be single or multiple.

[0143] The application provides a data labeling method. In order to more clearly describe the scheme of the application, some knowledge related to data labeling will be introduced first, and then the embodiments of the application will be described in combination with the accompanying drawings in the embodiments of the application.

[0144] Please refer to Figure 1 , Figure 1 An effect diagram of data labeling provided by the embodiments of the application.

[0145] Edge surface: Figure 1 (A) in FIG. 1 is a three-dimensional model of a target object (a target vehicle) obtained by a vehicle-mounted radar according to collected target data. The three-dimensional model of the target vehicle is composed of a plurality of edge surfaces, which can be obtained by labeling the collected target data, such as (B) in FIG. 1. Figure 1

[0146] Edge line: if (B) in FIG. 1 is taken as a two-dimensional image model of the target object (the target vehicle), the two-dimensional image model of the target vehicle is composed of a plurality of edge lines, which can be obtained by labeling the collected two-dimensional image data, such as (C) in FIG. 1. Figure 1 Figure 1

[0147] Edge point: a plurality of edge points are obtained by labeling the collected two-dimensional image data, and the edge line (C) in FIG. 1 can be obtained from the edge points. The black points labeled on the edge line are edge points. Figure 1

[0148] Image labeling: refers to a process of adding text feature information reflecting the content of an image to the image by a machine learning method according to the visual content of the image. The basic idea is to automatically learn the potential association or mapping relationship between the semantic concept space and the visual feature space by using a labeled image set or other available information, and add text keywords to unknown images. After the image automatic labeling technology is processed, the image information problem can be converted into a relatively mature text information processing problem.

[0149] Image data labeling is widely used in the fields of automatic driving, portrait recognition, medical image recognition and the like. Among them, the image data labeling in the field of automatic driving mainly has two labeling modes, one is bounding box labeling, and the other is fine segmentation labeling.

[0150] ​​​​The box labeling method is to label the edges of the box close to the edges of the target object and mark the attributes of each box when labeling the image. For an algorithm, each box is a small image, and each small image corresponds to an object category. For example, by using the above box labeling method, different categories of vehicles such as cars, vans, and small trucks can be obtained in the image. Special attention should be paid to the tangency of the box and the edge of the vehicle during labeling. If it is not tangent, such as selecting the part that does not belong to the vehicle into the box, the machine may identify the selected part as a vehicle during learning, thereby causing inaccurate or even incorrect identification of the machine. In addition, image data labeling requires a large amount of labeled data. The machine algorithm learns from a large amount of data and summarizes the high-dimensional features of these objects. When identifying new images, the summarized high-dimensional features can be used to identify new images, and a probability is given for each possible result.

[0151] The segmentation labeling method is to use an interactive image segmentation method to segment the target object in the initial region of the given original image using a segmentation algorithm to generate a series of edge points, and then correct the segmentation result by correcting the edge points.

[0152] The following will briefly describe the segmentation labeling method. It is assumed that the original image (a) contains multiple categories of objects such as people, vehicles, street trees, and buildings. The segmentation labeling needs to identify different categories of objects in the image and detect their positions to segment them.

[0153] First, the original image (a) is subjected to semantic segmentation, i.e., each pixel point in the original image (a) is classified to determine the category of each pixel point (such as belonging to the background, a person, or a vehicle), thereby labeling each object in the original image (a) with a category label and dividing the image into regions to obtain image (b).

[0154] Then, the image (b) obtained by semantic segmentation is subjected to instance segmentation. Instance segmentation is a combination of target detection and semantic segmentation, i.e., the target object is detected in the image, and then the category label of the target object is determined by combining the category of each pixel point determined by the semantic segmentation to obtain image (c). For example, taking a vehicle as a target object, semantic segmentation does not distinguish different instances of the same vehicle category (such as all vehicle objects being labeled red), while instance segmentation distinguishes different instances of the same category (such as using different colors to distinguish different vehicle categories) by combining the results of semantic segmentation and target object detection.

[0155] Finally, panoramic segmentation is performed on the image (c) segmented by instance segmentation. Panoramic segmentation is a combination of semantic segmentation and instance segmentation, that is, all targets in the image are detected, and different instances in the same category are distinguished to obtain the image (d). By comparing the image (c) and the image (d), it can be seen that the instance segmentation only detects and classifies the target objects (such as the car in the figure) in the image by pixels, and distinguishes different instances (using different colors), while the panoramic segmentation detects and classifies all objects in the image including the background to distinguish different instances (using different colors).

[0156] No matter the above box annotation method or the segmentation annotation method, the edge points of the target object generated finally need to be corrected, and the edge points are corrected one by one, which is time-consuming and laborious, and the accuracy and efficiency of the annotation result are low.

[0157] In view of the problem of low accuracy and efficiency of the annotation result in the above data annotation method, the embodiment of the application provides a data annotation system and a new data annotation method based on the data annotation system. The method detects the edge of the target object in the target image by proposing a deep learning method based on a preselected box to obtain a first contour of the target object, and then introduces weak semantic information to convert the correction of the edge points of the first contour of the target object into the correction of the edge lines of the first contour of the target object, and corrects the first contour of the target object to obtain a second contour of the target object, which can greatly improve the accuracy and efficiency of the annotation result.

[0158] The data annotation system and the data annotation method based on the data annotation system will be described below.

[0159] Please refer to Figure 2 , Figure 2 The architecture schematic diagram of the data annotation system provided by the embodiment of the application is shown in the figure.

[0160] As Figure 2As shown, the system architecture mainly includes three aspects: a data collection module, a storage module, and a data labeling module. Among them, the data collection module is used to collect data to be labeled, which can specifically collect image data through a camera, or collect target data through a radar, etc. The storage module is used to store the data to be labeled collected by the data collection module. The storage module can be a cloud or a local server, and data communication is realized with the data collection module through a mobile network. The data labeling module can be an independent module, or an intelligent driving platform such as a mobile data center (MDC), or an intelligent cockpit platform such as a cockpit domain controller (CDC), or a vehicle control platform such as a vehicle domain controller (VDC), etc., which is used to obtain the data to be labeled from the cloud or the server for labeling to obtain a labeling result, and store the labeling result to the cloud or the server, so as to be used for subsequent data recognition.

[0161] It can be understood that, Figure 2 The data labeling system architecture in the above embodiment is only an exemplary embodiment in the present application, and the data labeling system architecture in the present application includes but is not limited to the above data labeling system architecture.

[0162] Correspondingly, based on the data labeling system provided in the present application, a new data labeling method is also provided, and the data labeling process of the method is completed by the data labeling module in the above data labeling system. The method can be applied to various scenes such as automatic driving, portrait recognition, medical image recognition, etc., and the data labeling in the automatic driving application scene will be taken as an example for description.

[0163] Please refer to Figure 3 , Figure 3 An application scene diagram of data labeling provided in the present application.

[0164] As Figure 3As shown, the application scenario can include a cloud network and a plurality of vehicle driving control devices (test vehicle, user vehicle a, user vehicle b, user vehicle c). The plurality of vehicle driving control devices can communicate data with the cloud network, thereby realizing data interaction between the test vehicle and the user vehicles, and data interaction between different user vehicles (user vehicle a, user b, user vehicle c). In particular, the vehicle driving control device is an intelligent vehicle that can sense the road environment through a camera or a vehicle-mounted sensing system, automatically plan a driving route, and control the vehicle to reach a predetermined target. The intelligent vehicle is a high-tech comprehensive system that integrates environment perception, planning and decision-making, multi-level assisted driving, and other functions, which mainly uses computer, modern sensing, information fusion, communication, artificial intelligence, and automatic control technologies. In the present application, the vehicle (such as the test vehicle, user vehicle a, user vehicle b, and user vehicle c) can be a vehicle that mainly relies on an intelligent driving instrument in the vehicle to achieve the purpose of unmanned driving, can be an intelligent vehicle with an assisted driving system or a full-automatic driving system, and can also be a wheeled mobile robot.

[0165] Based on the above application scenario, in one possible implementation, the data labeling method provided by the embodiments of the present application is completed by the test vehicle. The test vehicle collects image data of the target road section in the driving process through a camera in the corresponding driving scenario of the target road section, labels the image data by relying on the computer system in the vehicle, obtains the corresponding data label, and transmits the data label to the cloud network. In actual driving, the user vehicle collects image data of the driving road section through a camera, and obtains the data label labeled by the test vehicle from the cloud network. The data label is used in combination with an image recognition algorithm to automatically identify the objects existing in the collected image, so as to achieve the purpose of sensing the road environment, automatically planning a driving route, and controlling the vehicle to avoid obstacle objects and reach a predetermined address.

[0166] In another possible implementation, the data labeling method provided by the embodiments of the present application is completed by the user vehicle. Since the road conditions are complex and changeable in the actual driving process of the vehicle, it is difficult to rely only on the data label obtained by the test vehicle in the test of the target road section to complete the identification of all collected image data. At this time, the user vehicle needs to label the image data by relying on the computer system in the vehicle, obtain the corresponding data label, and use the data label in combination with an image recognition algorithm to automatically identify the objects existing in the collected image, so as to achieve the purpose of sensing the road environment, automatically planning a driving route, and controlling the vehicle to avoid obstacle objects and reach a predetermined address. In addition, the user vehicle also transmits the data label to the cloud network, so that it can be used for subsequent image recognition of other user vehicles.

[0167] It can be understood that, Figure 3The data labeling in the automatic driving application scenario is only an exemplary embodiment in the embodiments of the present application, and the data labeling application scenario in the embodiments of the present application includes but is not limited to the above automatic driving application scenario.

[0168] Please refer to Figure 4 , Figure 4 A flowchart of a data labeling method provided in the embodiments of the present application is shown in the figure, and the method includes but is not limited to the following steps:

[0169] Step 401: The electronic device obtains a first edge line and a second edge line of a target object contained in a target image.

[0170] The electronic device in the embodiments of the present application is a device equipped with a processor that can be used to execute computer instructions. The electronic device can be a terminal such as a computer, a controller, etc., or a server, etc. Specifically, it can be a vehicle-mounted device equipped with a computer system in the above Figure 3 The vehicle-mounted device equipped with a computer system in the above

[0171] In the embodiments of the present application, the electronic device obtains a first edge line and a second edge line of a target object contained in a target image by processing the target image differently, and the first edge line and the second edge line are different.

[0172] The electronic device processes the target image differently, specifically, by processing the target image by segmentation to separate the target object in the target image to obtain the first edge point of the target object. The segmentation here includes a non-deep learning based image segmentation method (such as a segmentation method based on a watershed algorithm, graph theory, etc.) and a deep learning based image segmentation method. The main difference between the two is that the deep learning based image segmentation method needs to use a convolutional neural network to input the target image into the convolutional neural network for segmentation. After separating the target object in the target image, the edge of the target object is detected to separate the edge of the target object to obtain the second edge point of the target object. The edge detection here includes non-preselected frame deep learning based edge detection and preselected frame deep learning based edge detection. The difference between the two is whether a preselected frame is used. The preselected frame deep learning based edge detection is not to detect whether a pixel point on the target image is an edge point, but to detect the position and category of the edge point corresponding to each preselected frame.

[0173] The first edge point and the second edge point are both edge points of the target object, but are obtained by different manners, and thus the first edge point and the second edge point are not completely identical. According to the first edge point and the second edge point, the first edge line and the second edge line of the target object can be determined. Since the first edge point and the second edge point are not completely identical, the first edge line and the second edge line obtained accordingly are also not completely identical.

[0174] In a possible implementation, the first edge line and the second edge line of the target object can be determined according to the first edge point and the second edge point, specifically, a line formed by edge points that coincide with the first edge point and the second edge point is determined as the first edge line, that is, the first edge line is located on the edge result obtained by the segmentation processing and the edge detection, and the segmentation processing and the edge detection can both obtain the first edge line. Then, a line formed by removing, from the union of the first edge point and the second edge point, edge points that coincide with the first edge point and the second edge point is determined as the third edge line, that is, the third edge line is located on the edge result obtained by the segmentation processing or on the edge result obtained by the edge detection, but not on the edge results obtained by both the segmentation processing and the edge detection. Finally, the third edge line that has two or more intersection points with the first edge line is determined as the second edge line.

[0175] The specific implementation of obtaining the first edge line and the second edge line provided in the embodiments of the present application can improve the accuracy of the edge line of the target object obtained, and is conducive to converting the correction of the edge points of the first contour of the target object into the correction of the edge line of the first contour of the target object, thereby improving the accuracy and efficiency of the labeling result.

[0176] Exemplarily, the implementation of obtaining the first edge line and the second edge line of the target object included in the target image can be further described through an effect diagram of data labeling.

[0177] Please refer to Figure 5a , Figure 5a an effect diagram of data labeling provided by the embodiments of the present application.

[0178] As Figure 5a shown, the target object in the target image is separated out through segmentation processing to obtain the first edge point of the target object. Then, the edge of the target object is separated out through edge detection to obtain the second edge point of the target object. A line formed by edge points that coincide with the first edge point and the second edge point (that is, the edge points marked in Figure 5a , is determined as the first edge line of the target object (that is, Figure 5aThe first edge line is located on the edge result obtained by the segmentation processing or the edge result obtained by the edge detection, but not on both.

[0179] The third edge line is determined by the line formed by the union of the first edge points and the second edge points after removing the edge points coinciding with the first edge points and the second edge points (i.e. Figure 5a the edge points marked in the edge line a) in FIG. 6). Figure 5a The third edge line is located on the edge result obtained by the segmentation processing or the edge result obtained by the edge detection, but not on both. Figure 5a The third edge line with two or more intersection points with the first edge line (i.e. the edge line a) in FIG. 6) is determined as the second edge line (i.e.

[0180] the edge line b) in FIG. 6).

[0181] An exemplary data annotation effect diagram is shown in FIG. 6. Figure 5b Figure 5b An exemplary data annotation effect diagram is shown in FIG. 6.

[0182] As shown in FIG. 6, the content contained in a given target image is briefly divided into regions, the hexagon is the actual contour of the target object, the closed region inside the hexagon is the internal region of the target object, and the elliptical region containing the hexagon is the sensitive region of the contour of the target object. The target object needs to be annotated in the sensitive region to obtain the accurate contour. In the process of annotating the target object in the sensitive region, the edge of the target object can be separated based on the edge detection of the preselected frame deep learning to obtain the edge points of the target object. Figure 5b The edge detection based on the preselected frame deep learning can be specifically referred to in the edge detection based on the preselected frame deep learning.

[0183] Figure 5c An exemplary data annotation effect diagram is shown in FIG. 6. Figure 5c

[0184] As shown in FIG. 6, in the process of annotating the target image, the target object needs to be divided into a plurality of grid units, and the preselected content in the preselected frame is all the horizontal axis segments of the grid divided by the target image. Figure 5c Figure 5c ​​​The line segment is marked by the middle elliptical region. The edge detection based on the preselected frame deep learning is not to detect whether a pixel point on the target image is an edge point, but to detect the position and category of the edge point on the horizontal axis line segment corresponding to each preselected frame. That is, according to the position of the edge point on the horizontal axis line segment corresponding to each preselected frame on the edge line, the category label of the target object is determined, and the category of the label on which the edge point on the horizontal axis line segment corresponding to each preselected frame on the edge line of the target object is located is determined as the category label of the target object. For example, in the case where the number of edge points located on the first label is greater than the number of edge points located on the second label, the first label is determined as the category label of the target object, and similarly, in the case where the number of edge points located on the first label is less than the number of edge points located on the second label, the second label is determined as the category label of the target object. In the case where the number of edge points located on the first label is equal to the number of edge points located on the second label, the category label of the target object is further determined according to the key degree of the label. For example, in the case where the key degree of the first label is higher than the key degree of the second label, the first label is determined as the category label of the target object, and similarly, in the case where the key degree of the second label is higher than the key degree of the first label, the second label is determined as the category label of the target object. Specifically, the key degree of the label is not a fixed value, and can be different according to different application scenarios. In the automatic driving application scenario, by default, the importance of a person is higher than that of a car, and the importance of a car is higher than that of a street tree, so the key degree of a person is higher than that of a car, and the key degree of a car is higher than that of a street tree. After determining the category label of the target object, the edge points corresponding to each preselected frame on the edge line are adjusted according to the category label, and the edge direction and edge distance of the edge points corresponding to each preselected frame are assigned, and the edge distance of the edge points corresponding to each preselected frame is moved according to the edge direction, so as to obtain the edge points of the target object, i.e. the second edge points.

[0185] The embodiment of the present application can improve the accuracy and efficiency of determining the category of the target object by assigning the category label to the preselected frame based on the edge detection of the preselected frame deep learning, and further comparing the key degree of the label to determine the category label of the target object, and adjusting the edge points on the edge line of the target object according to the category label, so that the position accuracy of the obtained second edge points is higher, and the edge line of the target object obtained according to the category label is more accurate, which is beneficial to improve the accuracy and efficiency of the labeling result.

[0186] Step 402: obtaining a first contour of the target object according to the first edge line.

[0187] After the electronic device obtains the first edge line of the target object, the target object is initially labeled based on the first edge line to obtain a first contour of the target object. The first contour can be used as a final labeled contour result of the target object, or can be used as a candidate labeled contour result of the target object, and on this basis, the edges of the candidate contour are corrected to improve the accuracy of the labeling result.

[0188] For example, in the above Figure 5a , the target object is initially labeled using the obtained edge line a to obtain a first contour of the target object. It can be seen that the actual contour of the target object is slightly different from the obtained first contour, and the first contour does not completely cover the target object and is tangent to the edge of the target object. It is necessary to correct the edges of the first contour using the edge line b on the basis of the initial labeling to improve the accuracy of the labeling result.

[0189] Step 403: correcting the edge line of the first contour using the second edge line to obtain a second contour of the target object.

[0190] The electronic device corrects the edge line of the first contour obtained by initial labeling using the second edge line to obtain a second contour of the target object. Specifically, the contour formed by the second edge line and the edge line of the first contour is determined as the second contour of the target object, and the correction method improves the accuracy and efficiency of the labeling result.

[0191] For example, in the above Figure 5a , on the basis of initially labeling the target object using the edge line a to obtain the first contour, the edges of the first contour are corrected using the edge line b to improve the accuracy of the labeling result, and the second contour of the target object obtained after correction completely covers the target object and is tangent to the edge of the target object.

[0192] The current data labeling method usually uses a segmentation algorithm to segment the target object to obtain edge points of the target object, and then corrects the contour of the target object by correcting the edge points. Therefore, the edge points need to be manually corrected one by one, which is time-consuming and laborious, and the accuracy and efficiency of the labeling result are low.

[0193] The data labeling method provided by the embodiment of the present application can greatly improve the accuracy and efficiency of the labeling result.

[0194] Correspondingly, the data labeling method provided by the above Figure 4 can correspond toFigure 6a to Figure 6c The data labeling system architecture is shown. The data labeling method in Figure 4 will be described below. Figure 6a to Figure 6c The data labeling system architecture in will be described below.

[0195] Please refer to Figure 6a , Figure 6a The architecture schematic diagram of a data labeling system provided by the embodiment of the present application is shown.

[0196] As shown in Figure 6a , first, the image data of the target image is obtained, and then the image data is processed based on non-deep learning image segmentation, deep learning image segmentation, pre-frame deep learning edge detection, and non-pre-frame deep learning edge detection, to obtain the results of different edge information about the target object, and according to the results obtained by the above different processing, strong semantic information and weak semantic information are generated.

[0197] It can be understood that the strong semantic information is the edge line a in the above Figure 5a or the first edge line in the above Figure 4 , that is, it is located on the results of the edge information obtained by the above different processing, in other words, the overlapping part of the edge information obtained by the above different processing is the strong semantic information. Correspondingly, the weak semantic information is the edge line b in the above Figure 5a or the second edge line in the above Figure 4 , that is, it is located on the results of the edge information obtained by the above different processing, in other words, the non-overlapping part of the edge information obtained by the above different processing is the weak semantic information.

[0198] Finally, the target object is initially labeled based on the generated strong semantic information, and the initially labeled result is corrected based on the generated weak semantic information. In combination with the data labeling method in the above Figure 4 , the above steps 402: obtaining the first contour of the target object according to the first edge line, and step 403: correcting the edge line of the first contour using the second edge line to obtain the second contour of the target object can be respectively corresponded to. In combination with the specific embodiment of the data labeling in the above Figure 5a , the above steps of initially labeling the target object using the edge line a to obtain the first contour of the target object, and correcting the edge line of the first contour using the edge line b to obtain the second contour of the target object can be respectively corresponded to.

[0199] Further, the strong semantic information is generated according to the results obtained by the above different processing, and the specific process can be referred to in the above Figure 6b , Figure 6b The architecture schematic diagram of a data labeling system provided by the embodiment of the present application is shown.

[0200] AsFigure 6b As shown, the image data of the target image is respectively subjected to deep learning-based image segmentation and non-deep learning-based image segmentation, the results of the edge information about the target object obtained by the two are fused to generate the edge points of the target object. Then the image data of the target image is respectively subjected to pre-frame deep learning-based edge detection and non-pre-frame deep learning-based edge detection, the edge detection results obtained by the two are fused, and the edge detection fusion result is used for correction of the generated edge points of the target object, so as to obtain strong semantic information representing the first edge line of the target object.

[0201] Further, weak semantic information is generated according to the results obtained by the above different processing. For details, please refer to Figure 6c , Figure 6c The figure shows the architecture of a data labeling system provided by an embodiment of the present application.

[0202] As Figure 6c shown, in combination with the generated strong semantic information, the image data is respectively subjected to non-deep learning-based image segmentation, deep learning-based image segmentation, pre-frame deep learning-based edge detection, and non-pre-frame deep learning-based edge detection, which can generate weak semantic information.

[0203] Please refer to Figure 7 , Figure 7 The figure shows the flowchart of another data labeling method provided by an embodiment of the present application. The method includes but is not limited to the following steps:

[0204] Step 701: The electronic device obtains a first edge frame and a second edge frame of a target object according to target data of a target radar.

[0205] The electronic device in the embodiment of the present application is a device equipped with a processor that can be used to execute computer instructions. The electronic device can be a terminal such as a computer, a controller, etc., or a server, etc. Specifically, it can be a vehicle-mounted device equipped with a computer system in the above Figure 3 , which is used to label the target data collected by the vehicle-mounted target radar to obtain corresponding data labels, and use the data labels in combination with recognition algorithms to automatically recognize the objects existing in the collected target data, so as to realize sensing of the road environment, automatically plan the driving route and control the vehicle to avoid obstacles, and reach the predetermined address.

[0206] In the embodiment of the present application, the electronic device obtains a first edge frame and a second edge frame of a target object by performing different processing on the target data collected by the target radar in detecting the target object. The first edge frame and the second edge frame are different.

[0207] The electronic device processes the target data collected by the target radar in different manners. Specifically, the target object can be separated from the detection area of the target radar through segmentation processing to obtain a first edge surface of the target object. Then, edge detection is performed on the target object to separate the edge of the target object to obtain a second edge surface of the target object.

[0208] The first edge surface and the second edge surface are both edge surfaces of the target object, but are obtained in different manners. Therefore, the first edge surface and the second edge surface are not completely the same. According to the first edge surface and the second edge surface, the first frame and the second frame of the target object can be determined. Since the first edge surface and the second edge surface are not completely the same, the first frame and the second frame obtained therefrom are also not completely the same.

[0209] In a possible implementation, the first frame and the second frame of the target object can be determined according to the first edge surface and the second edge surface. Specifically, a frame formed by an edge surface where the first edge surface and the second edge surface coincide can be determined as the first frame, that is, the first frame is located on the edge result obtained by the segmentation processing and the edge detection. The segmentation processing and the edge detection can both obtain the first frame. Then, a frame formed by the union of the first edge surface and the second edge surface after removing the edge surface where the first edge surface and the second edge surface coincide can be determined as the third frame, that is, the third frame is located on the edge result obtained by the segmentation processing or the edge detection, but not on the edge result obtained by both the segmentation processing and the edge detection. Finally, the third frame that has two or more intersections with the first frame can be determined as the second frame.

[0210] In a possible implementation, the edge detection on the target object is performed to obtain a second edge surface of the target object. Specifically, a label of the target object can be determined according to positions of the edge surfaces on the edge frame of the target object. For example, if the number of edge surfaces located on a first label is greater than the number of edge surfaces located on a second label, the first label is determined as the label of the target object. Similarly, if the number of edge surfaces located on the first label is less than the number of edge surfaces located on the second label, the second label is determined as the label of the target object. In a case where the number of edge surfaces located on the first label is equal to the number of edge surfaces located on the second label, the label of the target object can be further determined according to a key degree of the first label and a key degree of the second label. For example, if the key degree of the first label is higher than the key degree of the second label, the first label is determined as the class label of the target object. Similarly, if the key degree of the second label is higher than the key degree of the first label, the second label is determined as the class label of the target object. Specifically, the key degree of the label is not a fixed value, and can be different according to different application scenarios. In an automatic driving application scenario, by default, the importance of a person is higher than that of a vehicle, and the importance of the vehicle is higher than that of a street tree. Therefore, the key degree of the person is higher than that of the vehicle, and the key degree of the vehicle is higher than that of the street tree. After the class label of the target object is determined, the positions of the edge surfaces on the edge frame of the target object are adjusted according to the class label, to obtain the second edge surface.

[0211] Step 702: obtaining a first contour of the target object according to the first edge frame.

[0212] After the electronic device obtains the first edge frame of the target object, the target object is labeled based on the first edge frame to obtain a first contour of the target object. The first contour can be used as a final labeling contour result of the target object, or can be used as a candidate labeling contour result of the target object, and the edge frame of the candidate contour is corrected to improve the accuracy of the labeling result.

[0213] Step 703: correcting the edge frame of the first contour by using the second edge frame to obtain a second contour of the target object.

[0214] The electronic device corrects the edge frame of the first contour obtained by the initial labeling by using the second edge frame to obtain a second contour of the target object. Specifically, a contour formed by the second edge frame and the edge frame of the first contour is determined as the second contour of the target object. The correction method improves the accuracy and efficiency of the labeling result.

[0215] The embodiments of the present application can obtain different bounding boxes of a target object, such as a first bounding box and a second bounding box, by performing different processing on target data collected by the target radar in detecting the target object. Then, the target object is preliminarily labeled based on the first bounding box to obtain a first contour of the target object. Then, the edge of the first contour obtained by the preliminary labeling is corrected by using the second bounding box to obtain a second contour of the target object.

[0216] The data labeling method provided by the embodiments of the present application is not only suitable for labeling two-dimensional image data, but also suitable for labeling three-dimensional radar data. The correction of the edge points of the first contour of the target object is converted into the correction of the edge of the first contour of the target object, and the second contour of the target object is obtained by correction, which can greatly improve the accuracy and efficiency of the labeling result.

[0217] The above describes the method of the embodiments of the present application in detail, and the device of the embodiments of the present application is provided below.

[0218] Please refer to Figure 8 , Figure 8 FIG. 1 is a structural schematic diagram of a data labeling device provided by the embodiments of the present application. The data labeling device 80 can include a determination unit 801 and a correction unit 802, and the descriptions of the units are as follows.

[0219] The determination unit 801 is configured to obtain a first edge line and a second edge line of a target object contained in a target image; the first edge line and the second edge line are different.

[0220] The determination unit 801 is further configured to obtain a first contour of the target object according to the first edge line.

[0221] The correction unit 802 is configured to correct an edge line of the first contour by using the second edge line to obtain a second contour of the target object.

[0222] In a possible implementation, the correction unit 802 is specifically configured to determine the contour formed by the second edge line and the edge line of the first contour as the second contour.

[0223] In a possible implementation, the device further includes:

[0224] The segmentation unit 803 is configured to perform segmentation processing on the target image to obtain first edge points of the target object; the segmentation processing is configured to separate the target object contained in the target image.

[0225] The edge detection unit 804 is configured to perform edge detection on the target object to obtain second edge points of the target object; the edge detection is configured to separate the edge of the target object.

[0226] The determining unit 801 is specifically configured to determine the first edge line and the second edge line according to the first edge point and the second edge point.

[0227] In a possible implementation, the determining unit 801 is specifically further configured to obtain the first edge line according to the first edge point and the second edge point; and a point on the first edge line is an edge point at which the first edge point and the second edge point coincide.

[0228] The determining unit 801 is specifically further configured to obtain the second edge line according to the first edge point, the second edge point, and the first edge line.

[0229] In a possible implementation, the determining unit 801 is specifically further configured to obtain a third edge line according to the first edge point and the second edge point; and a point on the third edge line is the first edge point or the second edge point, and is not an edge point at which the first edge point and the second edge point coincide.

[0230] The determining unit 801 is specifically further configured to determine the third edge line having two or more intersection points with the first edge line as the second edge line.

[0231] In a possible implementation, the edge detecting unit 804 is specifically configured to determine a label of the target object according to positions of edge points on an edge line of the target object.

[0232] The edge detecting unit 804 is specifically further configured to adjust the edge points on the edge line of the target object to obtain the second edge point according to the label of the target object.

[0233] In a possible implementation, the edge detecting unit 804 is specifically further configured to determine a first label as the label of the target object in a case where a number of edge points located on the first label is greater than a number of edge points located on a second label.

[0234] Alternatively, the edge detecting unit 804 is specifically further configured to determine a second label as the label of the target object in a case where a number of edge points located on the first label is less than a number of edge points located on the second label.

[0235] Alternatively, the edge detecting unit 804 is specifically further configured to determine the label of the target object according to a key degree of the first label and a key degree of the second label in a case where a number of edge points located on the first label is equal to a number of edge points located on the second label.

[0236] In a possible implementation, the segmentation unit 803 is specifically configured to input the target image into a convolutional neural network to perform the segmentation processing, to obtain the first edge point.

[0237] In a possible implementation, the determination unit 801 is specifically further configured to determine the first label as the label of the target object in a case where the key degree of the first label is higher than the key degree of the second label.

[0238] The determination unit 801 is specifically further configured to determine the second label as the label of the target object in a case where the key degree of the second label is higher than the key degree of the first label.

[0239] In a possible implementation, the determination unit 801 is configured to obtain a first bounding box and a second bounding box of a target object according to target data of a target radar, the first bounding box and the second bounding box are different, and the target data includes data obtained by the target radar detecting the target object.

[0240] The determination unit 801 is further configured to obtain a first contour of the target object according to the first bounding box.

[0241] The correction unit 802 is configured to correct a bounding box of the first contour by using the second bounding box, to obtain a second contour of the target object.

[0242] In a possible implementation, the correction unit 802 is specifically configured to determine the contour formed by the second bounding box and the bounding box of the first contour as the second contour.

[0243] In a possible implementation, the segmentation unit 803 is configured to perform segmentation processing on the target object, to obtain a first edge surface of the target object, and the segmentation processing is configured to separate the target object from a detection region of the target radar.

[0244] The edge detection unit 804 is configured to perform edge detection on the target object, to obtain a second edge surface of the target object, and the edge detection is configured to separate edges of the target object.

[0245] The determination unit 801 is specifically configured to determine the first bounding box and the second bounding box according to the first edge surface and the second edge surface.

[0246] In a possible implementation, the determination unit 801 is specifically further configured to obtain the first bounding box according to the first edge surface and the second edge surface, and the edge surface on the first bounding box is an edge surface on which the first edge surface and the second edge surface coincide.

[0247] The determining unit 801 is further configured to obtain the second bezel according to the first edge surface, the second edge surface, and the first bezel.

[0248] In a possible implementation, the determining unit 801 is further configured to obtain a third bezel according to the first edge surface and the second edge surface; a surface on the third bezel is the first edge surface or the second edge surface, and is not an edge surface that coincides with the first edge surface and the second edge surface.

[0249] The determining unit 801 is further configured to determine the third bezel with two or more surfaces intersecting with the first bezel as the second bezel.

[0250] In a possible implementation, the edge detecting unit 804 is specifically configured to determine a label of the target object according to positions of edge surfaces on a bezel of the target object.

[0251] The edge detecting unit 804 is further configured to adjust the edge surfaces on the bezel of the target object to obtain the second edge surface according to the label of the target object.

[0252] In a possible implementation, the edge detecting unit 804 is further configured to determine a first label as the label of the target object in a case where a number of edge surfaces located on the first label is greater than a number of edge surfaces located on a second label.

[0253] Alternatively, the edge detecting unit 804 is further configured to determine the second label as the label of the target object in a case where the number of edge surfaces located on the first label is less than the number of edge surfaces located on the second label.

[0254] Alternatively, the edge detecting unit 804 is further configured to determine the label of the target object according to a key degree of the first label and a key degree of the second label in a case where the number of edge surfaces located on the first label is equal to the number of edge surfaces located on the second label.

[0255] For the technical effects brought by any of the possible implementation manners in the embodiments of the present application, reference can be made to the introduction of the technical effects of the first aspect of the present application and the corresponding implementation manners.

[0256] According to the embodiments of the present application, Figure 8Each unit in the apparatus shown can be combined into one or several other units respectively or all, or some of the units can be further split into a plurality of units with smaller functions to constitute, which can achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In actual application, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the network device can also include other units. In actual application, these functions can also be assisted by other units, and can be implemented by multiple units in cooperation.

[0257] It should be noted that the implementation of each unit can also correspond to the above Figure 4 and Figure 7 The corresponding description of the method embodiments shown.

[0258] In the data labeling apparatus 80 described Figure 8 , the edge of the target object in the target image is detected by proposing a deep learning method based on the preselected frame, the first contour of the target object is obtained, and the weak semantic information is introduced to convert the correction of the edge points of the first contour of the target object into the correction of the edge lines of the first contour of the target object, and the second contour of the target object is obtained by correction, which can greatly improve the accuracy and efficiency of the labeling result.

[0259] Please refer to Figure 9 , Figure 9 A structural schematic diagram of an electronic device 90 provided by the embodiments of the present application. The electronic device 90 can include a memory 901, a processor 902. Further optionally, it can also include a communication interface 903 and a bus 904, wherein the memory 901, the processor 902 and the communication interface 903 are connected with each other through the bus 904. The communication interface 903 is used for data interaction with the above-mentioned data labeling apparatus 80.

[0260] The memory 901 is used to provide a storage space, and the storage space can store data such as operating systems and computer programs. The memory 901 includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0261] The processor 902 is a module for performing arithmetic operations and logical operations, and can be a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), and the like processing module.

[0262] The memory 901 stores a computer program, and the processor 902 invokes the computer program stored in the memory 901 to perform the above Figure 4 The data labeling method shown in the figure:

[0263] Obtain a first edge line and a second edge line of a target object contained in a target image; the first edge line and the second edge line are different;

[0264] According to the first edge line, a first contour of the target object is obtained;

[0265] The edge line of the first contour is corrected using the second edge line to obtain a second contour of the target object.

[0266] The specific content of the method executed by the processor 902 can be referred to the above Figure 4 , which will not be repeated here.

[0267] On the other hand, the processor 902 invokes the computer program stored in the memory 901 to perform the above Figure 7 The data labeling method shown in the figure:

[0268] According to the target data of the target radar, a first edge frame and a second edge frame of the target object are obtained; the first edge frame and the second edge frame are different, and the target data includes data obtained by the target radar detecting the target object;

[0269] According to the first edge frame, a first contour of the target object is obtained;

[0270] The edge frame of the first contour is corrected using the second edge frame to obtain a second contour of the target object.

[0271] The specific content of the method executed by the processor 902 can be referred to the above Figure 7 , which will not be repeated here.

[0272] Correspondingly, the processor 902 invokes the computer program stored in the memory 901, and can also be used to execute the method steps performed by each unit in the data labeling device 80 shown in the above Figure 8 The specific content of the method executed by the processor 902 can be referred to the above Figure 8 , which will not be repeated here.

[0273] In Figure 9 In the electronic device 90 described, by proposing a deep learning method based on a preselected frame to perform edge detection on a target object in a target image, a first contour of the target object is obtained, and then weak semantic information is introduced to convert the correction of edge points of the first contour of the target object into the correction of edge lines of the first contour of the target object, and a second contour of the target object is obtained by correction, which can greatly improve the accuracy and efficiency of the labeling result.

[0274] The embodiment of the application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and when the computer program runs on one or more processors, the method shown in the above Figure 4 and Figure 7 The embodiment of the application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and when the computer program runs on one or more processors, the method shown in the above

[0275] The embodiment of the application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program runs on a processor, the method shown in the above Figure 4 and Figure 7 The embodiment of the application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program runs on a processor, the method shown in the above

[0276] The embodiment of the application further provides a chip, wherein the chip comprises a processor, and the processor is used to execute instructions, and when the processor executes the instructions, the method shown in the above Figure 4 and Figure 7 The embodiment of the application further provides a chip, wherein the chip comprises a processor, and the processor is used to execute instructions, and when the processor executes the instructions, the method shown in the above

[0277] Optionally, the chip further comprises a communication interface, and the communication interface is used to input data or output data.

[0278] The embodiment of the application further provides a terminal, wherein the terminal comprises at least one data labeling device 80, or an electronic device 90, or a chip.

[0279] The embodiment of the application further provides a server, wherein the server comprises at least one data labeling device 80, or an electronic device 90, or a chip.

[0280] In summary, by proposing a deep learning method based on a preselected frame to perform edge detection on a target object in a target image, a first contour of the target object is obtained, and then weak semantic information is introduced to convert the correction of edge points of the first contour of the target object into the correction of edge lines of the first contour of the target object, and a second contour of the target object is obtained by correction, which can greatly improve the accuracy and efficiency of the labeling result.

[0281] Those skilled in the art can understand that all or part of the processes in the above-mentioned method embodiments can be implemented by computer program related hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed, the computer program can include the processes of the above-mentioned method embodiments. The storage medium includes read only memory (ROM) or random access memory (RAM), magnetic disc or optical disc, and various media that can store computer program codes.

Claims

1. A data labeling method, characterized by, The method comprises the following steps: obtaining a first edge line and a second edge line of a target object contained in a target image; the first edge line and the second edge line are different; obtaining a first contour of the target object according to the first edge line; correcting the edge line of the first contour by using the second edge line to obtain a second contour of the target object; before the step of obtaining the first edge line and the second edge line of the target object contained in the target image, the method further comprises the following steps: segmenting the target image to obtain first edge points of the target object; the segmentation is used to separate the target object contained in the target image; detecting edges of the target object to obtain second edge points of the target object; the edge detection is used to separate the edges of the target object; the step of obtaining the first edge line and the second edge line of the target object contained in the target image comprises the following steps: determining the first edge line and the second edge line according to the first edge points and the second edge points.

2. The method of claim 1, wherein, the step of correcting the edge line of the first contour by using the second edge line comprises the following steps: determining the contour formed by the second edge line and the edge line of the first contour as the second contour.

3. The method according to claim 1 or 2, characterized in that, the step of determining the first edge line and the second edge line according to the first edge points and the second edge points comprises the following steps: obtaining the first edge line according to the first edge points and the second edge points; the points on the first edge line are the edge points coinciding with the first edge points and the second edge points; obtaining the second edge line according to the first edge points, the second edge points and the first edge line.

4. The method of claim 3, wherein, the step of obtaining the second edge line according to the first edge points, the second edge points and the first edge line comprises the following steps: obtaining a third edge line according to the first edge points and the second edge points; the points on the third edge line are the first edge points or the second edge points, and are not the edge points coinciding with the first edge points and the second edge points; determining the third edge line having two or more intersection points with the first edge line as the second edge line.

5. The method according to any one of claims 1 to 4, characterized in that, the step of detecting edges of the target object to obtain the second edge points of the target object comprises the following steps: determining a label of the target object according to the positions of the edge points on the edge line of the target object; adjusting the edge points on the edge line of the target object to obtain the second edge points according to the label of the target object.

6. The method of claim 5, wherein, the step of determining the label of the target object according to the positions of the edge points on the edge line of the target object comprises the following steps: in the case that the number of the edge points on a first label is greater than the number of the edge points on a second label, determining the first label as the label of the target object; or, in the case that the number of the edge points on the first label is less than the number of the edge points on the second label, determining the second label as the label of the target object. Or, in the case that the number of edge points located on the first label is equal to the number of edge points located on the second label, determining the label of the target object according to the key degree of the first label and the key degree of the second label.

7. The method according to any one of claims 1 to 6, characterized in that, The segmentation processing on the target image comprises: The segmentation processing on the target image comprises:

8. A data labeling apparatus, comprising: The segmentation processing on the target image comprises: The segmentation processing on the target image comprises: The determining unit is configured to obtain a first edge line and a second edge line of a target object in a target image; the first edge line and the second edge line are different; The determining unit is further configured to obtain a first contour of the target object according to the first edge line; The correcting unit is configured to correct an edge line of the first contour by using the second edge line to obtain a second contour of the target object; The device further comprises: The segmentation unit is configured to perform segmentation processing on the target image to obtain first edge points of the target object; the segmentation processing is configured to separate the target object in the target image; The edge detection unit is configured to perform edge detection on the target object to obtain second edge points of the target object; the edge detection is configured to separate edges of the target object; 9. The apparatus of claim 8, wherein, The determining unit is specifically configured to determine the first edge line and the second edge line according to the first edge points and the second edge points.

10. The apparatus of claim 8 or 9, wherein, The correcting unit is specifically configured to determine a contour formed by the second edge line and the edge line of the first contour as the second contour. The determining unit is specifically further configured to obtain the first edge line according to the first edge points and the second edge points; points on the first edge line are edge points that coincide with the first edge points and the second edge points; 11. The apparatus of claim 10, wherein, The determining unit is specifically further configured to obtain the second edge line according to the first edge points, the second edge points, and the first edge line. The determining unit is specifically further configured to obtain a third edge line according to the first edge points and the second edge points; points on the third edge line are the first edge points or the second edge points, and are not edge points that coincide with the first edge points and the second edge points; 12. The apparatus of any one of claims 8-11, wherein, The determining unit is specifically further configured to determine the third edge line that has two or more intersection points with the first edge line as the second edge line. The edge detection unit is specifically configured to determine a label of the target object according to positions of edge points on the edge line of the target object; 13. The apparatus of claim 12, wherein, The edge detection unit is specifically further configured to adjust the edge points on the edge line of the target object to obtain the second edge points according to the label of the target object. The edge detection unit is specifically further configured to determine a first label as the label of the target object in the case that the number of edge points located on the first label is greater than the number of edge points located on a second label. Or, the edge detection unit is further configured to determine the second label as the label of the target object if the number of edge points located on the first label is less than the number of edge points located on the second label. Or, the edge detection unit is further configured to determine the label of the target object according to the key degree of the first label and the key degree of the second label if the number of edge points located on the first label is equal to the number of edge points located on the second label.

14. The apparatus of any one of claims 8-13, wherein, The segmentation unit is specifically configured to input the target image into a convolutional neural network for the segmentation processing to obtain the first edge points.

15. A data labeling apparatus, comprising: The data labeling device comprises: a processor and a memory; the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory, so that the data labeling device executes the method in any one of claims 1 to 7.

16. A computer readable storage medium characterized by: The computer readable storage medium is configured to store a computer program; when the computer program is executed, the method in any one of claims 1 to 7 is implemented. The computer program is executed, so that the method in any one of claims 1 to 7 is implemented.

17. A computer program product, characterised in that, The computer program is executed, so that the method in any one of claims 1 to 7 is implemented. The computer program is executed, so that the method in any one of claims 1 to 7 is implemented. The computer program is executed, so that the method in any one of claims 1 to 7 is implemented.

18. A chip, characterized by The data labeling device comprises: the processor is configured to execute instructions; when the instructions are executed, the method in any one of claims 1 to 7 is implemented. The data labeling device comprises:

19. A terminal, characterized by the processor is configured to execute instructions; when the instructions are executed, the method in any one of claims 1 to 7 is implemented.

20. A server, comprising: The data labeling device comprises: the processor is configured to execute instructions; when the instructions are executed, the method in any one of claims 1 to 7 is implemented.

Images