An image labeling system and an image labeling method
By implementing modular design and component libraries, the problems of low efficiency and high cost in image annotation tool development have been solved, enabling rapid iteration and efficient image annotation tool development.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN KOTEI INFORMATICS
- Filing Date
- 2022-06-23
- Publication Date
- 2026-07-21
AI Technical Summary
Existing image annotation tools suffer from low development efficiency, high cost, and difficulty in rapid iteration and updates when faced with various image AI training models and high accuracy requirements.
Adopting a modular design, we develop general operation modules and annotation component objects, which are connected through mounting bridges to form a component library. This supports standard data formats and algorithm libraries for different image annotation types, enabling rapid development and iteration.
It improves the development efficiency of image annotation tools, reduces development costs, supports rapid iteration, and adapts to changing needs for various image annotation types.
Smart Images
Figure CN115311487B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image annotation, and more specifically, to an image annotation system and an image annotation method. Background Technology
[0002] In recent years, with the rapid development of the AI industry, the demand for data annotation tools from many AI teams has exploded, leading to the emergence of four main categories of annotation tools: image, video, audio, and text. In particular, the rapid advancement of AI image technology has resulted in an increasing number of image recognition models. These models require different types of structured image data for AI training, leading to a proliferation of image annotation tools. Currently, common image annotation tool types include bounding box annotation, 2.5D bounding box annotation, polygon annotation, keypoint recognition, semantic segmentation, and image classification. As the types of AI training models increase, new annotation types will emerge based on existing ones. Furthermore, as image AI training models demand higher accuracy from labeled data, existing annotation tools are rapidly updating and iterating in terms of annotation methods and algorithms. Therefore, annotation tool development has become a continuous investment. Improving the efficiency of annotation tool development, reducing development costs, and accelerating iteration frequency have become important indicators of a labeling company's competitiveness. Summary of the Invention
[0003] This invention addresses the technical problems existing in the prior art by providing an image annotation system and an image annotation method.
[0004] According to a first aspect of the present invention, an image annotation system is provided, comprising an annotation scene rendering layer, an image annotation sprite, a mounting bridge, and a component library; A general operation module is developed based on the common parts of different image annotation types in the annotation process, and corresponding annotation component objects are developed based on different image annotation types. The general operation module and multiple annotation component objects are stored in the component library. The image annotation sprite binds multiple annotation component objects by binding a mounting bridge; Each annotation component object is used to perform corresponding type of annotation on the image to be annotated and return the annotation data to the image annotation sprite; The image annotation sprite is used to store the images to be annotated and the annotation data; The annotation scene rendering layer is used to draw and display the images to be annotated and the annotation data stored in the image annotation sprite.
[0005] Based on the above technical solution, the present invention can also be improved as follows.
[0006] Optionally, the annotation scene rendering layer includes an annotation scene rendering module, a gesture event module, a local storage module, a snapshot module, and an AI interface module; The annotation scene rendering module is used to draw and display the images to be annotated and the annotation data; The gesture event module is used for processing operation data from the mouse and keyboard; The local storage module is used to store the image annotation sprites; The snapshot module is used to store each operation as the rendering layer changes; The AI interface module is a reserved interface for expanding semi-automation, enabling data communication with external input devices and image annotation wizards.
[0007] Optionally, the image annotation sprite is used to store the images to be annotated and the annotation data, including: The image annotation sprite stores the images to be annotated and their corresponding annotation data in a tree data structure; wherein, the parent node in the tree data structure stores the image information to be annotated, and the child node stores the annotation data corresponding to the image; or, the parent node in the tree data structure stores the upper layer, and the child node stores the lower layer.
[0008] Optionally, the image information to be labeled stored in the parent node of the tree data structure includes the scaling value and coordinate value of the image to be labeled.
[0009] Optionally, each of the annotation component objects integrates an annotation algorithm library corresponding to different image annotation types. The annotation algorithm library includes a 2.5D algorithm library, a 2D algorithm library, a semantic segmentation algorithm library, a data classification algorithm library, and a key point algorithm library, which are used to perform 2.5D bounding box annotation, 2D bounding box annotation, object polygon outline annotation, data classification annotation, and key point recognition annotation on the image to be annotated, respectively.
[0010] Optionally, each of the annotation component objects outputs corresponding annotation data based on standard data format specifications.
[0011] According to a second aspect of the present invention, an image annotation method is provided, comprising: A general operation module is developed based on the common parts of different image annotation types in the annotation process, and corresponding annotation component objects are developed based on different image annotation types. The general operation module and multiple annotation component objects are stored in a component library. Image annotation sprites bind multiple annotation component objects by using a binding bridge; Each annotation component object performs corresponding annotations on the image to be annotated, returns the annotation data to the image annotation sprite for storage, and draws and displays the image to be annotated and the annotation data in the annotation scene rendering layer.
[0012] Optionally, the image annotation sprite stores the image to be annotated and the annotation data in the following manner: The image annotation sprite stores the images to be annotated and their corresponding annotation data in a tree data structure; wherein, the parent node in the tree data structure stores the image information to be annotated, and the child node stores the annotation data corresponding to the image; or, the parent node in the tree data structure stores the upper layer, and the child node stores the lower layer.
[0013] Optionally, each of the annotation component objects integrates an annotation algorithm library corresponding to different image annotation types. The annotation algorithm library includes a 2.5D algorithm library, a 2D algorithm library, a semantic segmentation algorithm library, a data classification algorithm library, and a key point algorithm library, which are used to perform 2.5D bounding box annotation, 2D bounding box annotation, object polygon outline annotation, data classification annotation, and key point recognition annotation on the image to be annotated, respectively.
[0014] Optionally, each of the annotation component objects outputs corresponding annotation data based on standard data format specifications.
[0015] This invention provides an image annotation system and image labeling method that identifies the commonalities in the annotation process of different image annotation types and develops a general operation module. Different parts are developed independently. If it is necessary to change or add to different image annotation types, only the corresponding annotation component objects need to be modified or redeveloped. Only the development of a certain module needs to be focused on, without having to familiarize oneself with the structure of the entire annotation tool, thereby achieving quick start and improving development efficiency. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the structure of an image annotation system provided by the present invention; Figure 2 A schematic diagram of the tree data structure for image annotation sprites; Figure 3 This is a flowchart illustrating an image identification method provided by the present invention. Detailed Implementation
[0017] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.
[0018] The problems mentioned in the background section can be addressed from multiple perspectives, including the following: 1. How to implement modular classification in image annotation tools? This reduces code coupling, allowing developers to focus on specific modules without needing to understand the entire annotation tool's structure, thus enabling quick learning and improving development efficiency.
[0019] 2. During feature iteration or new feature development, components that need to be replaced or are missing can be developed and replaced by mounting them. This ensures that using the replaced component will not affect other components, achieving functional isolation between components.
[0020] 3. A unified data standard is adopted for different image annotation types. This solves the compatibility issues caused by different data formats used by different developers.
[0021] Based on the above aspects, the present invention provides an image annotation system and an image identification method.
[0022] Example 1 An image identification system architecture, see [link to documentation]. Figure 1 The image annotation system mainly includes an annotation scene rendering layer, image annotation sprites, mounting bridges, and component libraries.
[0023] A general operation module is developed based on the common parts of the annotation process for different image annotation types, and corresponding annotation component objects are developed based on different image annotation types. The general operation module and multiple annotation component objects are stored in the component library. The image annotation sprite binds multiple annotation component objects by binding a mounting bridge.
[0024] Each annotation component object is used to annotate the image to be annotated with the corresponding type and return the annotation data to the image annotation sprite; the image annotation sprite is used to store the image to be annotated and the annotation data; the annotation scene rendering layer is used to draw and display the image to be annotated and the annotation data stored by the image annotation sprite.
[0025] It is understood that the application scenario of this invention is to annotate images, that is, to annotate target objects in images, which is called image annotation. Specifically, there are different types of image annotation, such as 2.5D bounding box annotation, 2D bounding box annotation, object polygon outline annotation, data classification annotation, and key point recognition annotation of the image to be annotated.
[0026] For each type of image annotation, there is a corresponding image annotation algorithm, which is used to annotate the image. Based on the characteristics of image annotation, this invention abstracts the common parts (such as data input) of different image annotation types in the annotation process and designs them in a modular fashion, called a general operation module, thus generating a basic overall architecture.
[0027] For the differences, namely specific image annotations, different annotation algorithms are used. Therefore, corresponding annotation component objects are developed according to different image annotation types, and the general operation module and multiple annotation component objects are stored in a component library. Interfaces for data input and output are defined for the functional characteristics of different annotation component objects, isolating different functional modules and generating a standard data format specification. That is, for different annotation component objects, when outputting annotation data after image annotation, a standard data format specification is followed.
[0028] For the differences between various image annotation types, corresponding image annotation component objects are developed. A general operation module and all annotation component objects form a component library. The image annotation sprite attaches multiple annotation component objects via a mounting bridge. The mounting bridge acts as middleware connecting the sprite and the component library; a sprite can only be bound to one mounting bridge object, while a mounting bridge object can be bound to multiple component objects. The mounting bridge executes multiple annotation component objects according to the principle of "first-bound, first-executed." Using the mounting bridge pattern, when updating or developing new features, only the component needs to be developed or modified. Then, the developed component is mounted through the mounting module, completing the feature development.
[0029] For details, please refer to the overall framework design diagram, such as... Figure 1 The overall framework mainly includes a scene annotation rendering layer, an image annotation sprite, a mounting bridge, and a component library. The scene annotation rendering layer includes a scene annotation rendering module, a gesture event module, a local storage module, a snapshot module, and an AI interface module.
[0030] The system includes: a scene rendering module for drawing and displaying images and annotation data; a gesture event module for processing mouse and keyboard operation data; a local storage module for storing image annotation sprites; a snapshot module for storing each step of the operation when the rendering layer changes; and an AI interface module, which is a reserved interface for expanding semi-automation to enable data communication with external input devices and image annotation sprites.
[0031] As an example, an image annotation sprite is used to store images to be annotated and annotation data, including: the image annotation sprite stores the images to be annotated and the corresponding annotation data in a tree data structure; wherein, the parent node in the tree data structure stores the information of the images to be annotated, and the child nodes store the annotation data corresponding to the images; or, the parent node in the tree data structure stores the upper layer, and the child nodes store the lower layer.
[0032] It is understandable that image annotation sprites mainly use scaling values, image coordinate values, and annotation data as basic elements. Through the management and design of a tree data structure, a sprite can become the parent sprite of multiple sprites, thereby realizing the concept of layers.
[0033] For different application scenarios, the image annotation sprite stores different data in the parent and child nodes of the tree data structure. For example, for an image with 50 bounding boxes for the target objects, the parent node in the tree data structure stores the original image information, such as the scaling and coordinates of the image to be annotated. The child nodes store the information of the 50 bounding boxes, such as their positions and the classification of the target objects. In this case, the parent node has 50 child nodes. For image layers, the parent node stores the layer above, and the child nodes store the layer below, storing layer information through a tree structure.
[0034] For each type of image annotation, a corresponding annotation component object is developed. This component object integrates an image annotation algorithm library, including libraries for 2.5D, 2D, semantic segmentation, data classification, and keypoint recognition. These libraries are used for 2.5D bounding box annotation, 2D bounding box annotation, object polygon outline annotation, data classification annotation, and keypoint recognition annotation of the images to be annotated, respectively. These annotation component objects form a component library, which is mainly divided into three parts: user input data, algorithm calculation, and result output. User-input gesture data drives the algorithm to perform calculations, returns the results to the image annotation sprite for storage, and simultaneously triggers the rendering layer to draw and display the image.
[0035] Without using this architecture, the varying skill levels of developers and the lack of a standardized module design can lead to repetitive development due to the inability to reuse different annotation types. In severe cases, the changes may be so significant that modifying the existing code is more difficult than redeveloping the entire project.
[0036] This framework, through its standardized modular design, allows developers to focus solely on the business logic of their assigned modules, thus reducing development complexity. The development and iteration of the annotation tool leads to the accumulation of components across different modules. When new requirements arise, readily available annotation components can be used directly, while components not yet in the library can be developed independently. Finally, the entire development process is completed through component assembly. These advantages significantly accelerate development iteration, reduce development costs, and enable the company to adapt to the ever-changing demands of the image annotation field, thereby enhancing its competitiveness.
[0037] Example 2 An image annotation method, see [link to relevant documentation] Figure 2 The image annotation method includes: A general operation module is developed based on the common parts of the annotation process for different image annotation types, and corresponding annotation component objects are developed based on different image annotation types. The general operation module and multiple annotation component objects are stored in a component library. The image annotation sprite binds multiple annotation component objects by binding a mounting bridge.
[0038] It is understandable that each type of image annotation has a corresponding image annotation algorithm, and the image is annotated using the appropriate algorithm. Based on the characteristics of image annotation, this invention abstracts the common parts (such as data input) of different image annotation types during the annotation process and designs them in a modular fashion, called a general operation module, thus generating a basic overall architecture.
[0039] For the differences, namely specific image annotations, different annotation algorithms are used. Therefore, corresponding annotation component objects are developed according to different image annotation types, and the general operation module and multiple annotation component objects are stored in a component library. Interfaces for data input and output are defined for the functional characteristics of different annotation component objects, isolating different functional modules and generating a standard data format specification. That is, for different annotation component objects, when outputting annotation data after image annotation, a standard data format specification is followed.
[0040] For the differences between various image annotation types, corresponding image annotation component objects are developed. A general operation module and all annotation component objects form a component library. The image annotation sprite attaches multiple annotation component objects via a mounting bridge. The mounting bridge acts as middleware connecting the sprite and the component library; a sprite can only be bound to one mounting bridge object, while a mounting bridge object can be bound to multiple component objects. The mounting bridge executes multiple annotation component objects according to the principle of "first-bound, first-executed." Using the mounting bridge pattern, when updating or developing new features, only the component needs to be developed or modified. Then, the developed component is mounted through the mounting module, completing the feature development.
[0041] Each annotation component object performs the corresponding type of annotation on the image to be annotated, returns the annotation data to the image annotation sprite for storage, and draws and displays the image to be annotated and the annotation data in the annotation scene rendering layer.
[0042] Understandably, when an image needs to be labeled, the image to be labeled is input into the corresponding labeling component object in the component based on the general operation module, the image is labeled using the labeling component object, and the labeling data is output according to the standard data format specification. The labeling data is then sent to the image labeling sprite for storage, and the labeling scene rendering layer draws and displays the image to be labeled and the corresponding labeling data.
[0043] The image annotation sprite stores the images to be annotated and the annotation data in the following way: The image annotation sprite stores the images to be annotated and the corresponding annotation data in a tree data structure; wherein, the parent node in the tree data structure stores the information of the image to be annotated, and the child node stores the annotation data corresponding to the image; or, the parent node in the tree data structure stores the upper layer, and the child node stores the lower layer.
[0044] Each of the annotation component objects integrates an annotation algorithm library corresponding to different image annotation types. The annotation algorithm library includes a 2.5D algorithm library, a 2D algorithm library, a semantic segmentation algorithm library, a data classification algorithm library, and a key point algorithm library, which are used to perform 2.5D bounding box annotation, 2D bounding box annotation, object polygon outline annotation, data classification annotation, and key point recognition annotation on the image to be annotated, respectively.
[0045] The image annotation system and method provided in this invention identify the commonalities in the annotation process of different image annotation types and develop a general operation module. Only different parts are developed independently. If it is necessary to change or add to different image annotation types, only the corresponding annotation component objects need to be modified or redeveloped. Only the development of a certain module is required, without having to be familiar with the structure of the entire annotation tool, thereby achieving quick start and improving development efficiency.
[0046] By implementing standardized modular design, developers can focus solely on the business logic of their assigned modules, thus reducing development complexity. The development and iteration of annotation tools accumulate components across different modules. When new requirements arise, readily available components can be used directly, while components not found in the library can be developed independently. Finally, the entire development process is completed through component assembly. These advantages significantly accelerate development iteration and reduce development costs.
[0047] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0048] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0049] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0050] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0051] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0052] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0053] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. An image annotation system, characterized in that, This includes a scene annotation rendering layer, image annotation sprites, mounting bridges, and component libraries; A general operation module is developed based on the common parts of different image annotation types in the annotation process, and corresponding annotation component objects are developed based on the differences of different image annotation types. The general operation module and multiple annotation component objects are stored in a component library. The image annotation sprite binds multiple annotation component objects by binding a mounting bridge; Each annotation component object is used to perform corresponding type of annotation on the image to be annotated and return the annotation data to the image annotation sprite; The image annotation sprite is used to store the images to be annotated and the annotation data; The annotation scene rendering layer is used to draw and display the images to be annotated and the annotation data stored in the image annotation sprite; The image annotation sprite is used to store the images to be annotated and the annotation data, including: The image annotation sprite stores the images to be annotated and their corresponding annotation data in a tree data structure; wherein, the parent node in the tree data structure stores the image information to be annotated, and the child node stores the annotation data corresponding to the image; or, the parent node in the tree data structure stores the upper layer, and the child node stores the lower layer.
2. The image annotation system according to claim 1, characterized in that, The annotation scene rendering layer includes an annotation scene rendering module, a gesture event module, a local storage module, a snapshot module, and an AI interface module; The annotation scene rendering module is used to draw and display the images to be annotated and the annotation data; The gesture event module is used for processing operation data from the mouse and keyboard; The local storage module is used to store the image annotation sprites; The snapshot module is used to store each operation as the rendering layer changes; The AI interface module is a reserved interface for expanding semi-automation, enabling data communication with external input devices and image annotation wizards.
3. The image annotation system according to claim 1, characterized in that, The information about the image to be labeled stored in the parent node of the tree data structure includes the scaling value and coordinate value of the image to be labeled.
4. The image annotation system according to claim 1, characterized in that, Each of the annotation component objects integrates an annotation algorithm library corresponding to different image annotation types. The annotation algorithm library includes a 2.5D algorithm library, a 2D algorithm library, a semantic segmentation algorithm library, a data classification algorithm library, and a key point algorithm library, which are used to perform 2.5D bounding box annotation, 2D bounding box annotation, object polygon outline annotation, data classification annotation, and key point recognition annotation on the image to be annotated.
5. The image annotation system according to claim 4, characterized in that, Each of the annotation component objects outputs corresponding annotation data based on the standard data format specification.
6. An image annotation method, characterized in that, include: A general operation module is developed based on the common parts of different image annotation types in the annotation process, and corresponding annotation component objects are developed based on the differences of different image annotation types. The general operation module and multiple annotation component objects are stored in a component library. Image annotation sprites bind multiple annotation component objects by using a binding bridge; Each annotation component object performs corresponding annotations on the image to be annotated, returns the annotation data to the image annotation sprite for storage, and draws and displays the image to be annotated and the annotation data in the annotation scene rendering layer; The image annotation sprite is used to store the images to be annotated and the annotation data, including: The image annotation sprite stores the images to be annotated and their corresponding annotation data in a tree data structure; wherein, the parent node in the tree data structure stores the image information to be annotated, and the child node stores the annotation data corresponding to the image; or, the parent node in the tree data structure stores the upper layer, and the child node stores the lower layer.
7. The image annotation method according to claim 6, characterized in that, The image annotation sprite stores the images to be annotated and the annotation data in the following manner: The image annotation sprite stores the images to be annotated and their corresponding annotation data in a tree data structure; wherein, the parent node in the tree data structure stores the image information to be annotated, and the child node stores the annotation data corresponding to the image; or, the parent node in the tree data structure stores the upper layer, and the child node stores the lower layer.
8. The image annotation method according to claim 6, characterized in that, Each of the annotation component objects integrates an annotation algorithm library corresponding to different image annotation types. The annotation algorithm library includes a 2.5D algorithm library, a 2D algorithm library, a semantic segmentation algorithm library, a data classification algorithm library, and a key point algorithm library, which are used to perform 2.5D bounding box annotation, 2D bounding box annotation, object polygon outline annotation, data classification annotation, and key point recognition annotation on the image to be annotated.
9. The image annotation method according to claim 8, characterized in that, Each of the annotation component objects outputs corresponding annotation data based on the standard data format specification.