A system and method for improving annotation attribute quality inspection efficiency based on an automatic driving scene

Abstract

The application relates to a system and method for improving annotation attribute quality inspection efficiency based on an automatic driving scene, which comprises a configuration module, a parsing module and a rendering module.The configuration module is used for responding to the input of a user in real time and determining one or more label templates according to real-time input information; each label template comprises the color configuration information of one or more types of annotation objects and annotation attributes; the annotation attributes include occlusion, confidence and direction; the parsing module is used for loading annotation data to be inspected, matching the one or more label templates determined by the configuration module with the annotation data to be inspected, and transmitting the matching results to the configuration module and the rendering module respectively; and the rendering module is used for rendering one or more annotation objects in real time according to the color configuration information and the matching results.The color configuration of the label templates containing different types of annotation objects and annotation attributes improves the quality inspection efficiency of different annotation attributes.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of vehicle auxiliary driving and image labeling, and particularly relates to a labeling attribute quality inspection and efficiency improvement method based on an automatic driving scene. BACKGROUND

[0002] Data labeling is a key link for most artificial intelligence algorithms to run effectively. The more accurate the data labeling is and the larger the amount of labeled data is, the better the performance of the algorithm is. Data labeling is to provide a high-quality, safe and reliable data set for the algorithm. At present, there is no clear reference standard for data labeling in the industry, so data quality inspection based on an automatic driving scene is the most important link in the production of a labeled data set. Efficient data quality inspection can guarantee the accuracy and timeliness of the data set, and therefore it is necessary to improve the review efficiency of data labeling attribute quality inspection items. The existing labeling attribute quality inspection is artificial attribute quality inspection.

[0003] Artificial attribute quality inspection is a process in which a quality inspector reviews the attribute dimensions of a labeled object according to labeling specifications. Especially when there are a large number of labeled objects or target objects overlapping in an image, the quality inspection task will be very complex and difficult, and the quality inspector needs to review each object, which is very time-consuming and tedious. In addition, due to the different knowledge backgrounds of the quality inspectors, there are differences in understanding the quality inspection specifications, which may result in different degrees of quality inspection errors and further affect the quality of the data set. For example, the quality inspection task needs to check whether the occlusion attribute values of all labeled objects are correct, and then the artificial needs to judge and record the attributes of each object, which is very time-consuming. SUMMARY

[0004] To improve the quality inspection efficiency of image labeling tasks, in a first aspect of the present application, a labeling attribute quality inspection and efficiency improvement system based on an automatic driving scene is provided, which comprises: a configuration module configured to respond to the input of a user in real time and determine one or more label templates according to real-time input information; wherein each label template comprises color configuration information of one or more types of labeled objects and their labeling attributes; the labeling attributes comprise occlusion, confidence and direction; an analysis module configured to load labeled data to be inspected, and match the one or more label templates determined by the configuration module with the labeled data to be inspected, and transmit the matching results to the configuration module and the rendering module respectively; and a rendering module configured to render one or more labeled objects in real time according to the color configuration information and the matching results.

[0005] In some embodiments of the present application, the configuration module comprises: a determination unit configured to determine one or more label templates according to real-time input information; and a color configuration unit configured to configure or generate color configuration information of one or more types of labeled objects and their labeling attributes according to the label templates.

[0006] Further, a sending unit is configured to send the label template to the parsing module.

[0007] In some embodiments of the present application, the parsing module comprises: a loading unit configured to load the annotation data to be inspected; a matching unit configured to match the annotation data to be inspected with one or more label templates determined by the configuration module; and a transmission unit configured to transmit the matching result to the configuration module and the rendering module respectively.

[0008] Further, the matching unit comprises: a first matching unit configured to match the annotation objects in the annotation data to be inspected with one or more label templates determined by the configuration module; and a second matching unit configured to match the annotation attributes in the annotation data to be inspected with one or more label templates determined by the configuration module.

[0009] Further, the transmission unit comprises: a first transmission unit configured to parse the matching result into color configuration information of one or more types of annotation objects and their annotation attributes, and transmit the color configuration information to the configuration module as a label template; and a second transmission unit configured to directly transmit the color configuration information of one or more types of annotation objects and their annotation attributes to the rendering module.

[0010] In a second aspect of the present application, a method for improving annotation attribute inspection efficiency based on an autonomous driving scene is provided, comprising: responding to user input in real time, and determining one or more label templates according to real-time input information; wherein each label template comprises color configuration information of one or more types of annotation objects and their annotation attributes; the annotation attributes include occlusion, confidence and direction; loading annotation data to be inspected, and matching the annotation data to be inspected with one or more determined label templates; and rendering one or more annotation objects in real time according to the color configuration information and the matching result.

[0011] In a third aspect of the present application, an electronic device is provided, comprising: one or more processors; and a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method for improving annotation attribute inspection efficiency based on an autonomous driving scene provided in the first aspect of the present application.

[0012] In a fourth aspect of the present application, a computer readable medium is provided, which stores a computer program, wherein the computer program is executed by a processor to implement the method for improving annotation attribute inspection efficiency based on an autonomous driving scene provided in the first aspect of the present application.

[0013] The present application has the following beneficial effects:

[0014] The application realizes self-defined rapid quality inspection of labeled attributes, and can be applied to rapid review of different attributes and attribute values in an automatic driving scene. Quality inspection personnel can select and configure attributes of quality inspection items that are complex and difficult to directly judge manually, render according to colors corresponding to attribute values, quickly determine whether the attribute values are accurate according to the colors, and improve the accuracy of labeled data. When multiple attributes are reviewed, a shortcut key is provided to switch rendering in real time, and rapid attribute review is supported. In an automatic driving scene, attributes such as occlusion, confidence, and direction need to be inspected, and the method can quickly improve the overall inspection efficiency for different attribute inspection items. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A basic flow diagram of the automatic driving scene-based labeled attribute quality inspection efficiency improvement system in some embodiments of the application is shown in the figure.

[0016] Figure 2 A working flow diagram of the automatic driving scene-based labeled attribute quality inspection efficiency improvement system in some embodiments of the application is shown in the figure.

[0017] Figure 3 A flow diagram of the automatic driving scene-based labeled attribute quality inspection efficiency improvement method in some embodiments of the application is shown in the figure.

[0018] Figure 4 A structural diagram of an electronic device in some embodiments of the application is shown in the figure. DETAILED DESCRIPTION

[0019] The principles and characteristics of the application are described below in conjunction with the accompanying drawings, and the examples are only used to explain the application and not to limit the scope of the application.

[0020] REFERENCE Figure 1 With Figure 2 In a first aspect of the application, an automatic driving scene-based labeled attribute quality inspection efficiency improvement system 1 is provided, which includes: a configuration module 11 configured to respond to user input in real time and determine one or more label templates according to real-time input information; each label template includes color configuration information of one or more types of labeled objects and their labeled attributes; the labeled attributes include occlusion, confidence, and direction; an analysis module 12 configured to load labeled data to be inspected, and match the labeled data to be inspected with one or more label templates determined by the configuration module, and transmit the matching results to the configuration module and the rendering module, respectively; and a rendering module 13 configured to render one or more labeled objects in real time according to color configuration information and matching results.

[0021] It can be understood that the above-mentioned label template generally corresponds to a text markup language (XML) or a web2, web3 or html corresponding label-based configuration file, each label (mark) can mark an object or a class of attributes (such as color), an object can correspond to multiple attributes, and an attribute can correspond to multiple objects; in the present disclosure, the object includes but is not limited to an image to be marked, one or more targets composed of multiple pixels in the image; the attribute includes the color, size, position occlusion, confidence and direction of each object, etc.

[0022] In some embodiments of the present application, the configuration module 11 comprises: a determination unit for determining one or more label templates according to real-time input information; a color configuration unit for configuring or generating color configuration information of one or more classes of marked objects and their marked attributes according to the label template. Further, a sending unit for sending the label template to the parsing module.

[0023] In some embodiments of the present application, the parsing module 12 comprises: a loading unit for loading the marked data to be inspected; a matching unit for matching the one or more label templates determined by the configuration module with the marked data to be inspected; a transmission unit for transmitting the matching result to the configuration module and the rendering module respectively.

[0024] Further, the matching unit comprises: a first matching unit for matching the one or more label templates determined by the configuration module with the marked objects in the marked data to be inspected; and a second matching unit for matching the one or more label templates determined by the configuration module with the marked attributes in the marked data to be inspected.

[0025] Further, the transmission unit comprises: a first transmission unit for parsing the matching result into color configuration information of one or more classes of marked objects and their marked attributes, and transmitting the color configuration information to the configuration module as a label template; and a second transmission unit for directly transmitting the color configuration information of one or more classes of marked objects and their marked attributes to the rendering module.

[0026] Based on the above-mentioned embodiments, in another embodiment of the system for improving annotation attribute inspection efficiency based on automatic driving scene in the present application, it comprises five parts of engineering architecture module, review mode module, UI interface construction module, annotation task information parsing module and canvas drawing engine module. The specific implementation method is as follows:

[0027] S1: Construction of engineering architecture module:

[0028] S11: Build code and related responsive logic through native js;

[0029] S12: Import and compile framework template requirements;

[0030] S13: Analyze and process the less syntax written CSS style requirements.

[0031] S2: Review the construction of the mode module (business layer):

[0032] S21: Add label classification select drop-down options to select label classification and related business logic;

[0033] S22: Add options radio single selector to select attribute items and related business logic;

[0034] S23: Add color configurator to configure colors for certain properties in attribute items and related business logic;

[0035] S24: Add message sending module to implement unified message sending for various operations within the component and related business logic;

[0036] S25: Shortcut key response function module to implement Ctrl+up and down arrow shortcut key function logic.

[0037] S3: Implementation of the UI interface construction module:

[0038] S31: Build review mode buttons and review mode list to add "review mode" and "review mode list" to the annotation tool interface and related business logic;

[0039] S32: Mount checklist subcomponent to implement the function of mounting the checklist subcomponent written by the framework to the review mode list.

[0040] S4: Implementation of the annotation task information parsing module:

[0041] S41: Add parsing and transform label information in tasks to implement the related business logic of parsing and transforming label templates in tasks and passing values to the checklist subcomponent;

[0042] S42: Add a method to respond to checklist subcomponent messages to implement the related business logic of filtering and setting annotation data color values based on the exported attribute configuration;

[0043] S43: Add a method to call the canvas drawing engine to execute the redraw method to call the canvas drawing engine to render the annotation based on the filtered and configured annotation data.

[0044] S5: Construction and invocation of the canvas drawing engine module:

[0045] S51: Adjust the drawing engine color rendering part of the logic in conjunction with the review mode;

[0046] S52: According to the actual development situation, the configuration adjustment related to the color in the canvas drawing engine is adjusted, such as the color of the edge line, the filling color, etc.

[0047] Embodiment 2

[0048] Reference Figure 3 In a second aspect, the application provides a method for improving annotation attribute quality inspection based on an automatic driving scene, comprising: S100. Real-time responding to the input of a user, and determining one or more label templates according to the real-time input information; wherein each label template comprises color configuration information of one or more types of annotation objects and their annotation attributes; the annotation attributes comprise occlusion, confidence, and direction; S200. Loading annotation data to be inspected, and matching the annotation data to be inspected according to the determined one or more label templates; S300. Real-time rendering one or more annotation objects according to the color configuration information and the matching result.

[0049] Embodiment 3

[0050] Reference Figure 4 In a third aspect, the application provides an electronic device, comprising: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of the first aspect of the application.

[0051] The electronic device 500 can include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 502 or programs loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0052] Generally, the following devices can be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a hard disk, etc.; and communication devices 509. The communication devices 509 can allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4The electronic device 500 is illustrated with various means, but it is to be understood that not all of the illustrated means need be present in every embodiment. A greater or lesser number of means can alternatively be present. Figure 4 Each block shown in the flowchart can represent one or more of the means, as appropriate.

[0053] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed. It should be noted that the computer readable medium described in embodiments of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In embodiments of the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, the computer readable signal medium can include a data signal that is propagated in baseband or that is propagated as part of a carrier wave. Such propagated data signals can take a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the above.

[0054] The computer readable medium described above can be included within the electronic device described above; alternatively, the computer readable medium can exist as a separate entity, which is not incorporated in the electronic device. The computer readable medium bears one or more computer programs that, when executed by the electronic device, cause the electronic device to:

[0055] Computer program code for carrying out operations of embodiments of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++, Python, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0056] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of a code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in some cases, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by a dedicated hardware-based system that carries out specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0057] The above descriptions are only the preferred embodiments of the present application, and are not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A system for improving the efficiency of attribute quality inspection in autonomous driving scenarios, characterized in that, include: The configuration module is used to respond to user input in real time and determine one or more label templates based on the real-time input information; Each label template includes one or more types of labeled objects and their color configuration information for the labeled attributes; The annotation attributes include occlusion, confidence level, and orientation; The parsing module is used to load the annotation data to be inspected, and to match the annotation data to be inspected with one or more label templates determined by the configuration module, and pass the matching results to the configuration module and the rendering module respectively. The parsing module includes: a loading unit for loading the annotation data to be inspected; a matching unit for matching the annotation data to be inspected with one or more tag templates determined by the configuration module; and a transmission unit for transmitting the matching results to the configuration module and the rendering module respectively. The matching unit includes: a first matching unit for matching the annotation objects in the annotation data to be inspected with one or more tag templates determined by the configuration module; and a second matching unit for matching the annotation attributes in the annotation data to be inspected with one or more tag templates determined by the configuration module. The transmission unit includes: a first transmission unit for parsing the matching results into color configuration information of one or more types of annotation objects and their annotation attributes, and transmitting the color configuration information as a tag template to the configuration module; and a second transmission unit for directly transmitting the color configuration information of one or more types of annotation objects and their annotation attributes to the rendering module. The rendering module is used to render one or more labeled objects and their labeled attributes in real time based on color configuration information and matching results.

2. The system for improving the efficiency of attribute quality inspection based on autonomous driving scenarios according to claim 1, characterized in that, The configuration module includes: The determining unit is used to determine one or more label templates based on real-time input information; The color configuration unit is used to configure or generate color configuration information for one or more types of labeled objects and their labeled attributes based on the label template.

3. The system for improving the efficiency of attribute quality inspection based on autonomous driving scenarios according to claim 2, characterized in that, Also includes: The sending unit is used to send the tag template to the parsing module.

4. A method for improving the efficiency of attribute quality inspection in autonomous driving scenarios, characterized in that, include: It responds to user input in real time and determines one or more label templates based on the real-time input information; Each label template includes one or more types of labeled objects and their color configuration information for the labeled attributes; The annotation attributes include occlusion, confidence level, and orientation; Load the annotation data to be inspected, and match it with one or more determined label templates: match the annotation objects in the annotation data with one or more label templates; match the annotation attributes in the annotation data with one or more label templates determined by the configuration module; parse the matching results into color configuration information of one or more types of annotation objects and their annotation attributes, and pass the color configuration information as a label template to the configuration module; pass the color configuration information of one or more types of annotation objects and their annotation attributes directly to the rendering module. Based on color configuration information and matching results, render one or more labeled objects and their labeled attributes in real time.

5. The method for improving the efficiency of attribute quality inspection based on autonomous driving scenarios according to claim 4, characterized in that, It also includes updating one or more label templates based on color configuration information and matching results.

6. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the method for improving the efficiency of labeled attribute quality inspection based on autonomous driving scenarios as described in any one of claims 4 to 5.

7. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the method for improving the quality inspection efficiency of labeled attributes based on autonomous driving scenarios as described in any one of claims 4 to 5.

Images