Graph labeling method and related device

Abstract

The invention discloses a graph labeling method and a related device, and the method comprises the steps: displaying a labeling interface according to interface configuration data, and displaying a to-be-labeled image on the labeling interface; receiving annotation related operation of the user on the annotation interface; generating graphic annotation data corresponding to the annotation related operation as an annotation result of the to-be-annotated image; wherein the interface configuration data is obtained through task configuration interface configuration according to a labeling scene corresponding to the to-be-labeled image; and / or the graphic annotation data is in a first standardized data format. According to the scheme, the labeling requirements of diversified labeling scenes can be met, and the reusability of labeling results is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a graphic annotation method, graphic annotation device, electronic device, and computer-readable storage medium. Background Technology

[0002] With the development of artificial intelligence, there are increasingly more applications of image processing using machine learning models. Image processing can include image detection, image segmentation, and image recognition. Application scenarios can be OCR (Optical Character Recognition) scenarios, such as classification attribute OCR, refined OCR, mathematical OCR, intelligent driving OCR, and educational OCR, or they can be non-OCR scenarios.

[0003] Before applying a machine learning model to image processing, it needs to be trained on labeled images to ensure its image processing capabilities meet expectations. Labeled images are obtained through image annotation. Corresponding to the diverse application scenarios of machine learning models, image annotation has diverse annotation scenarios, each with different annotation requirements. Different annotation requirements necessitate different annotation information for the images.

[0004] However, the image annotation methods in related technologies are difficult to adapt to the annotation needs of diverse annotation scenarios, and the annotation results have poor reusability. Summary of the Invention

[0005] This application provides a graphic annotation method, graphic annotation device, electronic device, and computer-readable storage medium, which can solve the problems of image annotation methods in related technologies being difficult to adapt to the annotation needs of diverse annotation scenarios and having poor reusability of annotation results.

[0006] This application provides a graphic annotation method, comprising: displaying an annotation interface according to an interface configuration data, and displaying an image to be annotated on the annotation interface; receiving annotation-related operations from a user on the annotation interface; generating graphic annotation data corresponding to the annotation-related operations as the annotation result of the image to be annotated; wherein, the interface configuration data is configured through a task configuration interface according to the annotation scenario corresponding to the image to be annotated, the interface configuration data includes fields and field values ​​corresponding to several functional items, the field values ​​corresponding to the functional items indicating whether the functional items are displayed on the annotation interface, the several functional items include custom graphic annotation functional items, the custom graphic annotation functional items correspond to custom graphic annotation functions, the custom graphic annotation functions support annotation-related operations of custom graphics; and / or, the graphic annotation data is a first standardized data format.

[0007] This application provides an image annotation device, comprising: a display module, a receiving module, and a generation module. The display module is used to display an annotation interface according to the interface configuration data and to display the image to be annotated on the annotation interface; the receiving module is used to receive annotation-related operations performed by a user on the annotation interface; the generation module is used to generate graphic annotation data corresponding to the annotation-related operations, as the annotation result of the image to be annotated; wherein, the interface configuration data is obtained by configuring the annotation scenario corresponding to the image to be annotated through a task configuration interface, and the interface configuration data includes fields and field values ​​corresponding to several functional items, the field values ​​corresponding to the functional items indicating whether the functional items are displayed on the annotation interface, the several functional items including a custom graphic annotation functional item, the custom graphic annotation functional item corresponding to a custom graphic annotation function, the custom graphic annotation function supporting annotation-related operations of custom graphics; and / or, the graphic annotation data is in a first standardized data format.

[0008] This application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-described method.

[0009] This application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described method.

[0010] The above solution, on the one hand, because the interface configuration data is obtained through the task configuration interface according to the annotation scenario corresponding to the image to be annotated, the image annotation method provided in this application supports adaptive configuration of interface configuration data for the image to be annotated according to the annotation scenario, and can adapt to the annotation requirements of different annotation scenarios. Furthermore, the interface configuration data includes fields and field values ​​corresponding to custom graphic annotation function items. Custom graphic annotation function items correspond to custom graphic annotation functions, which support annotation-related operations for custom graphics. Users can annotate any graphics through these operations, thereby adapting to special image annotation requirements. Therefore, it can adapt to different annotation scenarios and special image annotation requirements without modifying the core code or redeveloping, reducing development costs. On the other hand, because the graphic annotation data is in a first standardized data format, the annotation results obtained in this application can be directly transferred across annotation scenarios and annotation platforms without format conversion, reducing processing costs and improving reusability across annotation platforms and annotation scenarios.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0013] Figure 1 This is a flowchart illustrating an embodiment of the image annotation method provided in this application; Figure 2 This is a schematic diagram of the annotation interface of this application; Figure 3 This is a schematic diagram illustrating the operations related to drawing polygons or polylines in this application; Figure 4 This is a flowchart illustrating an embodiment of the image annotation method provided in this application; Figure 5 This is a flowchart illustrating an embodiment of the image annotation method provided in this application; Figure 6 This is a schematic diagram illustrating the operations related to drawing the rectangle in this application; Figure 7 This is a flowchart illustrating an embodiment of the image annotation method provided in this application; Figure 8 This is a schematic diagram of the graphic transformation operation performed on a rectangle on the image to be annotated in this application; Figure 9 This is a flowchart illustrating an embodiment of the image annotation method provided in this application; Figure 10 This is a schematic diagram of the polygon in this application; Figure 11 This is another schematic diagram of the polygon in this application; Figure 12 This application is a schematic diagram illustrating the operation of adding points to a polygon on an image to be annotated; Figure 13 This is a flowchart illustrating an embodiment of the image annotation method provided in this application; Figure 14 This is a schematic diagram of a user performing an image cropping operation on a polygon in an image to be labeled using a clipping graphic. Figure 15 This is a flowchart illustrating an embodiment of the image annotation method provided in this application; Figure 16 This is a schematic diagram of the applicant's face. Figure 17 It is a schematic diagram of the applicant's human anatomy; Figure 18 This is a schematic diagram of the software architecture of the image annotation method of this application; Figure 19 This is a schematic diagram of the annotation process for this application; Figure 20This is another schematic diagram of the annotation interface in this application; Figure 21 This is another schematic diagram of the annotation interface in this application; Figure 22 This is a schematic diagram of the structure of an embodiment of the image annotation device of this application; Figure 23 This is a schematic diagram of the structure of an embodiment of the electronic device of this application; Figure 24 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0014] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0015] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0016] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0017] With the development of artificial intelligence, there are increasingly more applications of image processing using machine learning models. Image processing can include image detection, image segmentation, and image recognition. Application scenarios can be OCR (Optical Character Recognition) scenarios, such as classification attribute OCR, refined OCR, mathematical OCR, intelligent driving OCR, and educational OCR, or they can be non-OCR scenarios.

[0018] Before applying machine learning models to image processing, they need to be trained on labeled images to ensure their image processing capabilities meet expectations. Labeled images are obtained through image annotation. Corresponding to the diverse application scenarios of machine learning models, image annotation has diverse annotation scenarios, each with different annotation requirements. Different annotation requirements necessitate different annotation information. For example, in a classification attribute OCR scenario, the required annotation information includes characters and their classification attributes. In a mathematical OCR scenario, the required annotation information includes mathematical symbols and formulas. In an intelligent driving OCR scenario, the required annotation information includes license plate location and license plate number. In an educational OCR scenario, the required annotation information includes test questions and answers. In a biometric OCR scenario, the required annotation information includes face location and body location.

[0019] Through long-term research, the inventors of this application have discovered that image annotation methods in related technologies can be divided into two categories: general image annotation methods and customized image annotation methods. General image annotation methods provide insufficient image annotation functionality and require secondary development to adapt to specific image annotation needs in particular application scenarios. Secondary development necessitates modification of the core code, resulting in high modification costs. Customized image annotation methods are developed from scratch for specific application scenarios, meeting their annotation requirements. However, they are deeply integrated with the business processes of these specific application scenarios, exhibiting poor versatility and scalability, making them difficult to adapt to diverse annotation scenarios.

[0020] In addition, the data format of the annotation results obtained by the image annotation methods in related technologies is private, that is, the annotation results are stored in a custom data format. When data is transferred between annotation platforms and annotation scenarios, additional format conversion or data processing is often required, making it difficult to reuse the annotation results across annotation platforms and annotation scenarios, resulting in poor reusability of the annotation results.

[0021] To address at least some of the aforementioned technical problems, this application provides an image annotation method. Embodiments of the image annotation method provided in this application are described below.

[0022] Figure 1 This is a flowchart illustrating an embodiment of the image annotation method provided in this application. Figure 1 As shown, in this embodiment, the image annotation method may include the following steps: S110: Display the annotation interface according to the interface configuration, and display the image to be annotated on the annotation interface.

[0023] The execution subject of this embodiment is an image annotation device, which can be any electronic device with image annotation capabilities.

[0024] The interface configuration data includes the layout of the annotation interface, interaction rules, annotation function item pool, and whether the various function items in the annotation function item pool are displayed on the annotation interface.

[0025] One function item corresponds to several annotation functions, and these annotation functions support several annotation-related operations. Displaying function items in the annotation interface means that corresponding annotation functions are provided for the annotation of the image to be annotated.

[0026] S120: Receives annotation-related operations from the user on the annotation interface.

[0027] Annotation-related operations can include turning functions on and off, drawing annotation graphics, and semantic-related operations.

[0028] The on / off operation of function items is used to set the status of the function items displayed in the annotation interface. The status of a function item includes an on state and an off state. When the function item is on, users can perform drawing-related operations and / or semantic-related operations on the annotation interface.

[0029] Drawing-related operations can include adding points, modifying points, deleting points, transforming graphics (rotating, scaling, and translating), cropping, and drawing.

[0030] Semantic operations can include semantic addition, semantic modification, and semantic deletion operations.

[0031] S130: Generate graphic annotation data corresponding to the annotation-related operations, as the annotation result of the image to be annotated.

[0032] The interface configuration data is obtained by configuring the task configuration interface according to the annotation scenario corresponding to the image to be annotated. The interface configuration data includes fields and field values ​​corresponding to several functional items. The field value corresponding to the functional item indicates whether the functional item is displayed in the annotation interface. The several functional items include a custom graphic annotation functional item, which corresponds to a custom graphic annotation function. The custom graphic annotation function supports annotation-related operations for custom graphics. And / or, the graphic annotation data is in the first standardized data format.

[0033] Graphical annotation data includes the graphic's positional information and semantic information. Positional information may include point coordinates / sets of point coordinates, dimensions, and positional reference information (connection rules for points, positional reference points), etc. Semantic information may include the annotation text, identification information, attribute information, style information, and graphic type. Graphic types may include rectangles, polygons, polylines, circles, and custom graphics. Custom graphics can be any shape, such as regular shapes, irregular shapes, closed shapes, and open shapes. Open shapes may include images of faces, human bodies, etc.

[0034] Drawing operations allow for the addition, deletion, and modification of graphic position information. Semantic operations allow for the addition, deletion, and modification of semantic information in labeled graphics. Users can select the type of graphic to draw by enabling the corresponding function items for labeled graphics of a specific type.

[0035] The image annotation method provided in this application corresponds to an annotation function item pool, which includes a variety of annotation function items to adapt to diverse annotation scenarios.

[0036] The task configuration interface refers to the function configuration interface for the image annotation task to which the image to be annotated belongs. Users can configure whether each annotation function item is displayed on the annotation interface according to the annotation scenario, thus obtaining interface configuration data. Therefore, to adapt to different annotation scenarios, the interface can select the appropriate annotation function items from the annotation function item pool based on the annotation scenario and display them on the annotation interface, without needing to perform secondary development and extension of general image annotation methods or develop custom image annotation methods from scratch to adapt to different annotation scenarios. Furthermore, the interface configuration data includes the fields and values ​​corresponding to the custom graphic annotation function items. The custom annotation functions corresponding to the custom graphic annotation function items support annotation-related operations for custom graphics. Custom graphics can be any type of graphic; therefore, user-specific image annotation needs can be met through annotation-related operations for custom graphics, without needing to perform secondary development and extension of general image annotation methods or develop custom image annotation methods from scratch to adapt to specific image annotation needs.

[0037] The first standardized data format conforms to industry standards. This first standardized data format can be, but is not limited to, OCR-Schema data format. Using the first standardized data format for graphic annotation data means that it can be reused across annotation scenarios and platforms without format conversion.

[0038] Through the implementation of this embodiment, on the one hand, since the interface configuration data is obtained by configuring the task configuration interface according to the annotation scene corresponding to the image to be annotated, the image annotation method provided by this application supports adaptive configuration of interface configuration data for the image to be annotated according to the annotation scene, and can adapt to the annotation requirements of different annotation scenes. Furthermore, the interface configuration data includes fields and field values ​​corresponding to custom graphic annotation function items. Custom graphic annotation function items correspond to custom graphic annotation functions, and custom graphic annotation functions support annotation-related operations for custom graphics. Users can annotate any graphic according to the annotation-related operations of custom graphics, thereby adapting to special image annotation requirements. Therefore, it can adapt to different annotation scenes and special image annotation requirements without modifying the core code or redeveloping, reducing development costs. On the other hand, since the graphic annotation data is in a first standardized data format, the annotation results obtained by this application can be directly transferred across annotation scenes and annotation platforms without format conversion, reducing processing costs and improving reusability across annotation platforms and annotation scenes.

[0039] Furthermore, in some embodiments, the graphic annotation data includes several annotation-related fields of the annotated graphic and corresponding field values, wherein the several annotation-related fields include a graphic location field and a graphic type field.

[0040] The graphic location field represents the position of the labeled graphic, while the graphic type field represents the graphic type of the labeled graphic. The graphic type of the labeled graphic can be any type, such as rectangle, polygon, polyline, circle, or custom graphic. The graphic location field corresponds to the graphic location information of the labeled graphic, while the graphic type field corresponds to the graphic type in the semantic information of the labeled graphic.

[0041] In some embodiments, S130 includes: determining the field value of at least one annotation-related field based on annotation-related operations to obtain graphic annotation data.

[0042] For example, in response to enabling the "Rectangle Annotation Function" item, the field value corresponding to the graphic type field is determined to be "rectangle". In response to drawing-related operations of the annotated graphic, the field value corresponding to the graphic position field is determined.

[0043] In some embodiments, the graphic position field includes a first position field and a second position field. The first position field represents the position of a labeled graphic of type rectangle, and the field value of the first position field includes the coordinates of the target vertex of the rectangle and the size of the rectangle. The second position field represents the position of a labeled graphic of type other graphic shape, and the field value of the second position field includes the coordinates of several points on the labeled graphic of other graphic shape.

[0044] The target vertex of the rectangle can be any one of the four vertices of the rectangle. The dimensions of the rectangle can include its length and width. For example, the first position field is the position of the rectangle (x, y, w, h), where (x, y) represents the coordinates of the target vertex, x and y are the horizontal and vertical coordinates of the target vertex, respectively, and (w, h) represents the dimensions, where w and h represent the width and height, respectively. Other graphic shapes can be polygons, polylines, circles, custom shapes, etc. For example, the position of a triangle is points, where points include the coordinates of three points on the triangle.

[0045] In some embodiments, the plurality of annotation-related fields further include at least one of the following: graphic identifier field, annotation text field, annotation attribute field, circle attribute field, frame information field of annotation graphic, fill information field of annotation graphic, and custom graphic reference information field, wherein the custom graphic reference information field includes at least one of the custom graphic connection rule field and the position reference point field.

[0046] The graphic identifier field corresponds to the identifier information of the labeled graphic. The annotation text field corresponds to the annotation text of the labeled graphic. The annotation attribute field and the circle attribute field correspond to the attribute information of the labeled graphic. The box information field and the fill information field correspond to the style information of the labeled graphic. The custom graphic reference information field corresponds to the graphic position information of the labeled graphic.

[0047] The first standardized data format specifies the naming and representation of each annotation-related field in the graphic annotation data, the type of the field value corresponding to each annotation-related field, whether each annotation-related field is a required field, and the default value of each annotation-related field. Table 1 below is an example of the first standardized data format (resultData of OCR-Schema data format) for graphic annotation data.

[0048] Table 1. First Standardized Data Format for Graphical Annotation Data

[0049] Table 1 above specifies the naming of each annotation-related field in the "Field Name" column, the "Field Description" column specifies the content represented by each annotation-related field, the "Field Type" column specifies the type of the field value corresponding to each annotation-related field, and the "Required" column specifies whether each annotation-related field is required and whether the field value has a default value. Among these: Graphic Identifier Field: Named id; Represents a unique identifier for the labeled graphic to distinguish different labeled graphics from the image to be labeled; The field value is of type String; It is a required field and will be automatically generated if it is not present.

[0050] First position field: named (x, y, w, h); represents the position of the labeled graphic of type rectangle; the field value type is Number; it is a required field when the graphic type is rectangle.

[0051] The second location field, named "points," represents the location (a set of point coordinates) of a labeled graphic of other graphic types; the field value is of type Array. <number>(Array); This is a required field if the graphic type is another graphic shape.

[0052] Graphic type field: named region; represents the graphic type of the labeled graphic; the field value type is String; it is a required field.

[0053] Annotation text field: named text; represents the annotation text, category label, etc. corresponding to the annotation graphic; the field value type is String or Object; not a required field.

[0054] Label attribute field: named type; represents the label attribute of the labeled graphic, such as the object category and confidence level of the labeled graphic; the field value type is Object; not a required field.

[0055] The circular attribute field is named "radius" and represents the radius of a circular annotation graphic. The field value is of type Number and is not required.

[0056] The bounding box information field for the labeled graphic is named strokeInfo; it represents the bounding box information of the labeled graphic, such as color, thickness, and dashed line; the field value type is Object; it is a non-required field, and the field value has a default value.

[0057] The fill information field for the labeled graphic is named fillInfo; it represents the fill information of the labeled graphic, such as color and transparency; the field value is of type Object; it is a non-required field, and the field value has a default value.

[0058] The field for connecting lines in a custom graphic is named "lines"; it represents the connection rules between points in the set of point coordinates of the custom graphic; the field value is of two-dimensional data type; it is not a required field.

[0059] The custom graphic's position reference point field is named centerPoint; it represents the index of the reference vertex of the custom graphic; the field value is of type Number; it is not a required field.

[0060] Furthermore, in some embodiments, the interface configuration data is a second standardized data format, wherein the interface configuration data includes fields and field values ​​corresponding to several functional items, and the field values ​​corresponding to the functional items indicate whether the functional items are displayed on the annotation interface.

[0061] One function item corresponds to several annotation functions. Displaying a function item in the annotation interface means that the annotation function corresponding to that function item is provided for annotating the image to be annotated.

[0062] In some embodiments, the display of the annotation interface according to the interface configuration data in S110 includes: displaying a function item in the annotation interface with a field value of a target field value in the interface configuration data, wherein the target field value represents the function item displayed in the annotation interface.

[0063] In some embodiments, the interface configuration data includes scene configuration data and graphics configuration data. The scene configuration data includes fields and field values ​​corresponding to at least one scene function item, and the graphics configuration data includes fields and field values ​​corresponding to at least one graphics function item.

[0064] In some embodiments, the scene function items include at least one of the following: original icon annotation function item, image rotation function item, batch operation function item, quick annotation function item, data formula function item, attribute inheritance function item, graphic numbering function item, and graphic attribute function item.

[0065] The functions are categorized as follows: Original Image Annotation: This function includes zooming in and out of the image to be annotated. Image Rotation: This function allows for rotating the annotated image. Batch Operation: This function allows for batch deletion of annotated images. Quick Annotation: This function includes AI annotation and preset template annotation. Data Formula: This function allows for recognizing and inserting data formulas. Attribute Inheritance: This function allows the next annotated image to inherit the attributes of the previous annotated image. Image Numbering: This function allows for setting numbers for annotated images. Image Attributes: This function allows for setting attribute information for annotated images.

[0066] In some embodiments, the graphic function items include at least one of the following: rectangle annotation function item, polygon annotation function item, polyline annotation function item, circle annotation function item, custom graphic annotation function item, graphic merging function item, graphic smudge function item, add point function item, and delete point function item.

[0067] The functions are as follows: Rectangle Annotation, Polygon Annotation, Polyline Annotation, Circle Annotation, Custom Graphic Annotation, Graphic Merge (including merging annotated graphics), Graphic Smudge, Add Point, and Delete Point.

[0068] In some embodiments, the functional items in the interface configuration data can also be divided in other ways. For example, functional items corresponding to a specific graphic type can be divided into one category, and functional items that do not distinguish between graphic types can be divided into another category.

[0069] Understandably, by dividing the interface configuration data into different data blocks, the code corresponding to the interface configuration data can be decoupled into component modules corresponding to different data blocks, which facilitates management.

[0070] The second standardized data format conforms to industry standards. The second standardized data format may be the same as or different from the first standardized data format.

[0071] The second standardized data format specifies the naming, content, field value types, and whether fields are required for scene configuration data and graphics configuration data within the interface configuration data. Table 2 below shows an example of the first standardized data format (settingData in the OCR-Schema data format) for interface configuration data.

[0072] Table 2. First Standardized Data Format for Interface Configuration Data

[0073] In Table 2 above, the "Field Name" column specifies the naming of the scene configuration data and the graphics configuration data; the "Field Description" column specifies the content represented by the scene configuration data and the graphics configuration data; the "Field Type" column specifies the type of the field values ​​corresponding to the scene configuration data and the graphics configuration data; and the "Required" column specifies whether the scene configuration data and the graphics configuration data are required fields. Among these: Scene configuration data: named sceneSetting; represents scene configuration; field value type is Object; is a required field.

[0074] Graphical configuration data: named graphicalSetting; represents graphical configuration; field value type is Object; is a required field.

[0075] The second standardized data format also specifies the naming, representation, type, and default value of each scene function item and the corresponding field of the graph in the scene configuration data.

[0076] Table 3 below shows an example of the second standardized data format for scene configuration data (sceneSetting in OCR-Schema data format).

[0077] Table 3. Second standardized data format for scene configuration data

[0078] In Table 3 above, the "Field Name" column specifies the naming of the fields corresponding to each scenario's functional item, the "Field Description" column specifies the content represented by the fields corresponding to each scenario's functional item, the "Field Type" column specifies the data type of the field values ​​corresponding to each scenario's functional item, and the "Default Value" column specifies the default value of the field values ​​corresponding to each scenario's functional item. Among these: The field corresponding to the original icon annotation function item is named "zoom"; it indicates whether the original icon annotation function item is displayed. The field corresponding to the graphic rotation function item is named rotate; it indicates whether the graphic rotation function item is displayed.

[0079] The field corresponding to the batch operation function item is named "batch"; it indicates whether the batch operation function item is displayed.

[0080] The field corresponding to the quick annotation feature is named quickMark; it indicates whether the quick annotation feature is displayed.

[0081] The field corresponding to the data formula function item is named "latex"; this indicates whether the data formula function item is displayed.

[0082] The field corresponding to the attribute inheritance feature is named inherit; it indicates whether the attribute inheritance feature is displayed.

[0083] The field corresponding to the graphic numbering function item is named "serial" and indicates whether the graphic numbering function item is displayed.

[0084] The field corresponding to the graphic attribute function item is named "attribute" and indicates whether the graphic attribute function item is displayed.

[0085] The field values ​​corresponding to each scene function item are of type Boolean, and the default value of the field value is true. True means that the corresponding scene function item will be displayed in the annotation interface.

[0086] The second standardized data format also specifies the naming, content, value type, and default value of the fields corresponding to each graphic function item in the graphic configuration data. Table 4 below shows an example of the second standardized data format for graphic configuration data (graphicalSetting in OCR-Schema data format).

[0087] Table 4. Second Standardized Data Format for Graphical Configuration Data

[0088] In Table 4 above, the "Field Name" column specifies the naming of the fields corresponding to each graphical function item, the "Field Description" column specifies the content represented by the fields corresponding to each graphical function item, the "Field Type" column specifies the data type of the field values ​​corresponding to each graphical function item, and the "Default Value" column specifies the default values ​​of the field values ​​corresponding to each graphical function item. Among these: The field corresponding to the rectangle annotation function item is named rect; it indicates whether the rectangle annotation function is displayed.

[0089] The field corresponding to the polygon annotation function item is named polygon; it indicates whether the polygon annotation function item is displayed.

[0090] The field corresponding to the polyline annotation feature is named "line" and indicates whether the polyline annotation feature is displayed.

[0091] The field corresponding to the circular annotation feature is named "circle" and indicates whether the circular annotation feature is displayed.

[0092] The field corresponding to the custom graphic annotation function item is named "custom"; it indicates whether the custom graphic annotation function item is displayed.

[0093] The field corresponding to the graphic merging function item is named merge; it indicates whether the graphic merging function item is displayed.

[0094] The field corresponding to the graphic smudge function item is named "clear"; it indicates whether the graphic smudge function item is displayed.

[0095] The field corresponding to the "Add Point" function item is named "addPoint" and indicates whether the "Add Point" function item is displayed.

[0096] The field corresponding to the delete point function item is named delPoint; it indicates whether the delete point function item is displayed.

[0097] The field values ​​for each graphic function item are of type Boolean. The default value for the field value for the custom graphic annotation function item is false, which means that the custom graphic annotation function item is not displayed in the annotation interface. The field values ​​for other graphic function items are true.

[0098] Furthermore, in some embodiments, the graphic annotation method further includes: acquiring metadata, and determining information about the current annotation task and the image to be annotated based on the metadata. The metadata is in a third standardized data format and includes a task identifier field, a task status field, an image to be annotated information field, and the corresponding field values ​​for the aforementioned fields.

[0099] The task identifier field includes at least one of a first task identifier field and a second task identifier field. The first task identifier field represents the first task identifier (task ID), used to identify the annotation task to which the image to be annotated belongs. The second task identifier field represents the second task identifier (batch ID), used to identify the annotation batch to which the image to be annotated belongs. The task status represents the task status, with optional annotation statuses including in progress, completed, etc. The image information field represents the basic information of the image to be annotated, including the image's attributes, name, etc.

[0100] The third standardized data format can be the same as or different from the first and second standardized data formats.

[0101] The third standardized data format specifies the naming of each field in metadata, the content it represents, the type of the corresponding field value, and whether the field is required. Table 5 below shows an example of the third standardized data format for metadata (metadata in OCR-Schema data format).

[0102] Table 5. Third Standardized Data Format for Metadata

[0103] In Table 5 above, the "Field Name" column specifies the naming of each field in the metadata, the "Field Description" column specifies the content represented by each field in the metadata, the "Field Type" column specifies the type of the field value corresponding to each field in the metadata, and the "Required" column specifies whether each field in the metadata is required, and the default value if it is required. Among these: The first task identifier field is named taskId; it represents the annotation task ID to which the image to be annotated belongs; the field value type is String; it is not a required field and can be set according to the project requirements of the annotation task.

[0104] The second task identifier field is named batchId; it represents the batch ID to which the image to be labeled belongs; the field value is of type String; it is not a required field and can be set according to the project requirements of the labeling task.

[0105] Task Status field: named taskStatus; represents the annotation status of the image to be annotated; the field value type is String; it is not a required field and can be set according to the project requirements of the annotation task.

[0106] Image information field to be labeled: named imageInfo; represents the basic information of the image to be labeled; the field value type is Object; required field, defaults to {}.

[0107] Furthermore, each function item in the interface configuration data has two states on and off in the annotation interface. When a function item is on, users are allowed to perform drawing-related and semantic-related operations for the corresponding function. When off, drawing-related and semantic-related operations for the corresponding function item are prohibited. For example, if the "Graphic Rotation Function" is on, rotating the annotated graphic is allowed; if it is off, rotating the annotated graphic is prohibited. Similarly, if the "Rectangle Annotation Function" is on, drawing-related and semantic-related operations for rectangles are allowed; if it is off, drawing-related and semantic-related operations for rectangles are prohibited.

[0108] In some embodiments, annotation-related operations include enabling and disabling functions. When a function is enabled, it can be switched from an enabled state to an disabled state in response to a disabling operation. Conversely, when a function is disabled, it can be switched from a disabled state to an enabled state in response to an enabling operation. For example, if each function is initially disabled, and the user enables the function, the function is switched from a disabled state to an enabled state to allow the user to perform corresponding drawing-related and semantic-related operations. After the corresponding drawing-related and semantic-related operations are completed, the function is switched from an enabled state to a disabled state in response to the user disabling the function, thus preventing the user from performing corresponding drawing-related and semantic-related operations.

[0109] Furthermore, in some embodiments, the drawing-related operations target the graphic position information of the labeled graphic. These operations include graphic drawing operations, graphic rotation operations, point addition operations, point deletion operations, and point modification operations. Graphic drawing operations are used to add new labeled graphics, graphic rotation operations are used to rotate labeled graphics, point addition operations are used to add points to the set of point coordinates of the labeled graphic, point deletion operations are used to delete points from the set of point coordinates of the labeled graphic, and point modification operations are used to modify points from the set of point coordinates of the labeled graphic.

[0110] In some embodiments, semantic-related operations target the semantic information of the labeled graphic. Semantic-related operations include semantic addition operations, semantic modification operations, and semantic deletion operations. Semantic addition operations are used to add semantic information to the labeled graphic; semantic modification operations are used to modify the semantic information of the labeled graphic; and semantic deletion operations are used to delete the semantic information of the labeled graphic.

[0111] Furthermore, in some embodiments, the annotation interface includes a configuration area, a drawing area, and an annotation area, wherein the component modules corresponding to the configuration area, drawing area, and annotation area are independent, and the component modules can communicate with each other.

[0112] The configuration area displays interface configuration data, the drawing area displays the image to be annotated, and the annotation area displays graphic annotation data. Component modules can be Vue or React component modules.

[0113] On the one hand, dividing the annotation interface into configuration, drawing, and annotation areas allows users to focus more on each area during the annotation process, reducing distractions and improving annotation efficiency. On the other hand, the independent component modules corresponding to the configuration, drawing, and annotation areas mean that the code for the annotation interface can be decoupled into independent component modules according to the partitions, facilitating management. Furthermore, the ability for communication between component modules means that the partitions can synchronize data and link actions. For example, when a user adds annotation attributes to annotated graphics in the annotation area, the drawing area can simultaneously display the annotation attributes of the annotated graphics on the image to be annotated, and vice versa. Similarly, when a user enables a function in the configuration area, users can perform corresponding drawing-related and semantic-related operations in the drawing and annotation areas. In short, partitioning the annotation interface enhances the flexibility of annotation interface management and allows for the dynamic addition or customization of partitions based on annotation scenarios and needs, thereby adapting to diverse annotation requirements in diverse annotation scenarios.

[0114] In some embodiments, the configuration area includes multiple sub-configuration areas, each used to display different data blocks of interface configuration data. For example, the configuration area includes a scene sub-configuration area and a graphics sub-configuration area, where the scene sub-configuration area displays various scene function items and the graphics sub-configuration area displays various image function items. Figure 2 This is a schematic diagram of the annotation interface of this application. For example... Figure 2 As shown, the annotation interface includes a drawing area, an annotation area, a configuration operation area (i.e., the scene sub-configuration area), and a frame function selection area (i.e., the graphics sub-configuration area). The drawing area is used to display the image to be annotated, the annotation area is used to display the image annotation data, the configuration operation area is used to display the scene function items, and the frame function selection area is used to display the image function items.

[0115] In some embodiments, S110 may include S111. S111: Display at least one function item in the configuration area of ​​the annotation interface according to the interface configuration data, and display the image to be annotated in the drawing area of ​​the annotation interface.

[0116] In some embodiments, S120 may include S121 and / or S122.

[0117] S121: Obtain the annotation graphics drawn by the user on the image to be annotated displayed in the drawing area.

[0118] It can receive drawing-related operations performed by the user on the image to be annotated displayed in the drawing area, and in response to the drawing-related operations, obtain the drawn annotated graphics.

[0119] Before S121, the method may also include: obtaining the type of graphic to be drawn selected by the user in the configuration area. For example, it may receive the user's "rectangle annotation function item" activation operation in the configuration area and determine that the type of graphic to be drawn by the user is a rectangle.

[0120] S122: Obtain the annotation text of the corresponding annotation graphic entered by the user in the annotation area, or display the recognition text corresponding to the annotation graphic in the annotation area, obtain the text modified by the user after recognizing the text, and use it as the annotation text of the corresponding annotation graphic.

[0121] The identified text is obtained by performing text recognition on the image portion located within the labeled graphic in the image to be labeled.

[0122] Among them, text recognition can be achieved through OCR technology, text detection technology, and other methods.

[0123] This system can receive user-generated text-related operations on the annotation area and, in response, retrieve the annotation text from the annotated graphics. These text-related operations are semantically related operations and can include text modification, deletion, and addition.

[0124] In some embodiments, the graphic annotation data includes graphic position information. S140 may also be included after S120. S140: In response to a user performing a drawing-related operation on the annotation graphic, the current annotation graphic is drawn on the image to be annotated.

[0125] In some embodiments, S130 may include S131. S131: In response to the user performing drawing-related operations on the annotation graphic, obtain the graphic position information of the currently drawn annotation graphic, so as to serve as the graphic annotation data corresponding to the annotation graphic.

[0126] Figure 3 This is a schematic diagram illustrating the operations related to drawing polygons or polylines in this application. For example... Figure 3 As shown, for polygons or polylines, the graphic position information can be a set of point coordinates called `points`, which includes the coordinates of several points (vertices) on the polygon or polyline. Drawing operations related to polygons or polylines can include adding points, deleting points, modifying points, and rotating the graphic. Specifically, adding a point adds the coordinates of at least one point to the set of point coordinates; deleting a point deletes the coordinates of at least one point; modifying a point modifies the coordinates of at least one point; and rotating the graphic rotates the coordinates of all points in the set of point coordinates. The set of point coordinates can be updated through these drawing operations on polygons or polylines.

[0127] Figure 4 This is a flowchart illustrating an embodiment of the image annotation method provided in this application. In this embodiment, the drawing-related operations include adding points. The graphic type of the annotated graphic is a polygon or a polyline. The graphic position information corresponding to the polygon or polyline is a set of point coordinates, which includes the coordinates of several points on the polygon or polyline. The execution order of each step in this embodiment is not limited to the order shown in the diagram. Figure 4 As shown, the image annotation method may include the following steps, where S211-S213 are further extensions of S140, and S214 is a further extension of S131.

[0128] S211: In response to a user's operation of adding a point to a polygon or polyline on the image to be annotated, obtain the coordinates of the user's added point and the selected point from several points on the polygon or polyline.

[0129] S212: Copy the original set of point coordinates of the polygon or polyline to obtain the first new set of point coordinates. Add the newly added point coordinates to the first new set of point coordinates before or after the selected point to obtain the set of point coordinates of the polygon or polyline after the newly added point.

[0130] Copying is done to ensure that adding new points does not affect the original set of point coordinates.

[0131] S213: Redraw the polygon or polyline on the image to be labeled according to the set of point coordinates after the newly added point.

[0132] S214: Use the set of point coordinates of the polygon or polyline after the newly added point as the graphic position information of the polygon or polyline after the newly added point.

[0133] Unlike the previous embodiments, in this embodiment, the image position information of the polygon or polyline is a set of point coordinates. By adding a point, point coordinates can be added to the set of point coordinates to update the set of point coordinates, thereby updating the image position information of the polygon or polyline.

[0134] Figure 5 This is a flowchart illustrating an embodiment of the image annotation method provided in this application. In this embodiment, the drawing-related operations include a graphic rotation operation. The graphic type for annotation is a polygon or a polyline. The graphic position information corresponding to the polygon or polyline is a set of point coordinates, which includes the coordinates of several points on the polygon or polyline. The execution order of the steps in this embodiment is not limited to the order shown in the diagram. Figure 5 As shown, the image annotation method may include the following steps, where S311-S313 are further extensions of S140, and S314 is a further extension of S131.

[0135] S311: In response to a user performing a graphic rotation operation on a polygon or polyline on the image to be annotated, obtain the first rotation angle of the user-selected rotation point before and after the rotation.

[0136] The rotation operation point can be any point in the set of point coordinates. The first rotation angle of the rotation operation point before and after rotation refers to the angle between the line connecting the rotation operation point before rotation and the line connecting it after rotation. The line connecting the rotation operation point before rotation is the line connecting the rotation operation point to the rotation center. The rotation center can be the origin, the center of a polygon, or the center of a polyline, etc.

[0137] S312: Update the coordinates of each point in the set of point coordinates of the polygon or polyline before rotation using the first rotation angle, so as to obtain the set of point coordinates of the polygon or polyline after rotation.

[0138] In some embodiments, the coordinates of each point in the set of point coordinates before rotation can be rotated around the rotation center using a first rotation angle to update the coordinates of each point.

[0139] In some embodiments, the first rotation angle can be converted into radians, and the coordinates of each point in the set of point coordinates before rotation can be rotated around the rotation center using radians to update the coordinates of each point.

[0140] S313: Redraw the polygon or polyline on the image to be labeled according to the set of point coordinates of the polygon or polyline after rotation.

[0141] S314: Use the set of point coordinates of the polygon or polyline after rotation as the graphic position information of the rotated polygon or polyline.

[0142] For ease of understanding, examples of S311-S314 are provided below: 1. Obtain the tilt angle angle1 of the rotation operation point (x1, y1) relative to the origin (cx, cy) before rotation, and obtain the tilt angle angle2 of the rotation operation point (x2, y2) relative to the origin after rotation.

[0143] The Math.atan2() function can be used to obtain angle1 and angle2. For example, executing Math.atan2(y1-cy, x1-cx) will give angle1.

[0144] 2. Obtain the first rotation angle before and after the rotation operation point: angle = angle2 - angle1.

[0145] 3. Convert the first rotation angle angle to radians rad = (angle * Math.PI) / 180.

[0146] 4. Rotate the coordinates (xi, yi) of the i-th point in the original set of point coordinates using radians (rad) to obtain the coordinates (newX, newY) of the i-th point in the original set of point coordinates.

[0147] newX = x * Math.cos(rad) - y * Math.sin(rad); newY = x * Math.sin(rad) + y * Math.cos(rad); x = xi – cx; y = yi – cy.

[0148] 5. By rotating the coordinates of each point in the original set of point coordinates using radians (rad), we obtain the rotated set of point coordinates.

[0149] 6. Based on the set of point coordinates of the polygon or polyline after rotation, redraw the polygon or polyline on the image to be labeled.

[0150] Unlike the previous embodiments, in this embodiment, when performing a graphic rotation operation on a polygon or polyline on the image to be labeled, the first rotation angle of the rotation operation point before and after rotation is obtained. The coordinate set of the points before rotation is rotated using the first rotation angle to obtain the coordinate set of the points after rotation. The polygon or polyline is redrawn using the coordinate set of the points after rotation for the user to view, and the image position information of the polygon or polyline is updated.

[0151] Figure 6 This is a schematic diagram illustrating the operations related to drawing the rectangle in this application. For example... Figure 6 As shown, for a rectangle, the graphic position information can include the coordinates of the target vertex and the rectangle's dimensions (x, y, w, h), or it can include the coordinates of the target vertex, the rectangle's dimensions, and a second rotation angle ((x, y, w, h) + angle). In the case of (x, y, w, h), (x, y) represents the actual coordinates of the target vertex. In the case of (x, y, w, h) + angle, (x, y) represents the coordinates of the target vertex rotated to the standard direction, and angle is the second rotation angle of the target vertex rotated to the standard direction. The standard direction can be a direction without rotation. Rectangle drawing operations can include graphic transformation operations. Graphic transformation operations can include rotation, translation, scaling, etc. Rotation is used to rotate the rectangle. Translation is used to translate the rectangle. Scaling is used to scale the rectangle. Through rectangle drawing operations, (x, y, w, h) or (x, y, w, h) + angle can be updated.

[0152] Figure 7 This is a flowchart illustrating an embodiment of the image annotation method provided in this application. In this embodiment, the drawing-related operations include graphic transformation operations, the graphic type for annotation is rectangle, and the graphic position information corresponding to the rectangle includes the coordinates of the target vertex of the rectangle and the size of the rectangle. The execution order of each step in this embodiment is not limited to the order shown in the figure. Figure 7 As shown, the image annotation method may include the following steps, where S411-S414 are further extensions of S140, and S415 is a further extension of S131.

[0153] S411: In response to a user performing a graphic transformation operation on a rectangle on an image to be annotated, and the graphic transformation operation includes a graphic rotation operation, obtain the second rotation angle of the transformed rectangle relative to the standard direction.

[0154] Graphic transformation operations include graphic rotation, graphic translation, and graphic scaling. Graphic transformation operations include graphic rotation, which can be performed only, or simultaneously with graphic rotation, translation, and scaling.

[0155] The standard direction can be a direction without rotation. In the standard direction, the second rotation angle is 0.

[0156] S412: Get the new coordinates of the target vertex when the transformed rectangle is rotated to the standard orientation.

[0157] The angle at which the object is rotated to the standard direction is the second rotation angle.

[0158] S413: Use the new coordinates of the target vertex, the size of the rectangle after transformation, and the second rotation angle as reference information for drawing the rectangle.

[0159] S414: Redraw the current rectangle on the image to be annotated according to the rectangle drawing reference information.

[0160] S415: Use the drawing reference information as the position information of the graphic after the rectangle transformation.

[0161] To facilitate understanding, examples are provided for S411-S415: Figure 8 This is a schematic diagram illustrating the graphic transformation operation performed on a rectangle on the image to be annotated in this application. For example... Figure 8 As shown, the coordinates of the top left vertex of the rectangle A1-B1-C1-D1 before transformation are (xA1, yA1), the size is (w1, h1), and the second rotation angle relative to the standard direction is 0.

[0162] 1. The rectangle A1-B1-C1-D1 before transformation is rotated, translated, and scaled to obtain the transformed rectangle A2-B2-C2-D2.

[0163] 2. Obtain the dimensions of the transformed rectangle.

[0164] You can obtain the horizontal scaling ratio scaleX and the vertical scaling ratio scaleY corresponding to the scaling operation. Use the horizontal scaling ratio scaleX and the vertical scaling ratio scaleY to scale the size (w1, h1) of the rectangle A1-B1-C1-D1 before the transformation, and obtain the size (w2, h2) of the transformed rectangle, where w2 = w1 * scaleX and h2 = h1 * scaleY.

[0165] 3. Obtain the second rotation angle of the transformed rectangle.

[0166] Obtain the rotation angle k corresponding to the rotation operation, add the rotation angle k corresponding to the rotation operation to the second rotation angle 0 of the rectangle before transformation, and obtain the second rotation angle k of the rectangle after transformation.

[0167] 4. Obtain the coordinates of the top-left vertex of the transformed rectangle.

[0168] Suppose the original coordinates of the top left vertex of the transformed rectangle are (xA2, yA2), but obtaining (xA2, yA2) is complicated.

[0169] Therefore, rotate the transformed rectangle to the standard direction (i.e., rotate it in the opposite direction by the second rotation angle k, or rotate it by -K) to obtain the rectangle A3-B3-C3-D3 rotated back to the standard direction; extract the new coordinates (xA3, yA3) of the top left vertex of the rectangle A3-B3-C3-D3.

[0170] 5. Update and redraw graphic position information.

[0171] The new coordinates of the top left vertex (xA3, yA3), the dimensions of the transformed rectangle (w2, h2), and the second rotation angle k are used as the graphic position information of rectangle A2-B2-C2-D2.

[0172] Redraw rectangle A1-B1-C1-D1 as rectangle A2-B2-C2-D2 using the new coordinates of the top left vertex (xA3, yA3), the dimensions of the transformed rectangle (w2, h2), and the second rotation angle k.

[0173] Understandably, if the shape information of a rectangle includes the original coordinates (xA2, yA2) of the target vertex of the transformed rectangle and the rectangle's dimensions (w2, h2), then when performing a shape transformation operation on the rectangle, it is necessary to calculate the original coordinates (xA2, yA2) of the target vertex after the transformation using trigonometric functions and other methods, which is computationally complex.

[0174] In this embodiment, after performing a graphic transformation operation on the rectangle, the new coordinates (xA3, yA3) of the target vertex when the transformed rectangle is rotated to the standard direction, and the second rotation angle k when rotated to the standard direction are further obtained. The new coordinates (xA3, yA3), the second rotation angle k when rotated to the standard direction, and the size (w2, h2) of the rectangle are used as the graphic position information of the rectangle. The method of obtaining (xA3, yA3) is simpler than the method of obtaining (xA2, yA2), thereby simplifying the method of obtaining the graphic position information of the rectangle.

[0175] Figure 9 This is a flowchart illustrating an embodiment of the image annotation method provided in this application. In this embodiment, the drawing-related operations include adding points; the graphic type for annotation is a polygon; the graphic position information corresponding to the polygon is a set of point coordinates, which includes the coordinates of several points on the polygon. The execution order of the steps in this embodiment is not limited to the order shown in the diagram. Figure 9 As shown, the image annotation method may include the following steps, where S511-S516 are further extensions of S140, and S517 is a further extension of S131.

[0176] S511: In response to a user's operation of adding a point to a polygon on the image to be annotated, obtain the operation point selected by the user in the drawing area of ​​the annotation interface.

[0177] The operation point is the mouse point clicked by the user. This operation point may or may not be located on the edge of the polygon.

[0178] S512: Determine several sides of the polygon based on the original set of point coordinates of the polygon.

[0179] An edge can be formed by matching the coordinates of every two adjacent points in the original set of point coordinates of the polygon. The original set of point coordinates of the polygon contains N points. If the polygon is a closed shape, the N points form N edges. If the polygon is not a closed shape, the N points form N-1 edges.

[0180] S513: Select the first target edge that is closest to the operation point from a number of edges.

[0181] S514: The mapping point of the operation point on the first target edge is used as the new point.

[0182] S515: In the original set of point coordinates of the polygon, insert the coordinates of the newly added point between the coordinates of the two points corresponding to the first target edge to obtain the second set of new point coordinates.

[0183] S516: Redraw the polygon on the image to be labeled according to the second new set of point coordinates.

[0184] S517: Use the set of coordinates of the second new point as the graphic position information of the polygon after the addition of the new point.

[0185] To facilitate understanding, examples are provided for S511-S517.

[0186] The order of the coordinates of the points in the set of coordinates of a polygon represents the order in which the coordinates of those points are connected. Different orders will result in completely different polygons. Figure 10 This is a schematic diagram of the polygon in this application. Figure 11 This is another schematic diagram of the polygon in this application. Figure 10 and Figure 11 The polygons shown in the diagram all include four points: 1, 2, 3, and 4. These four points have the same coordinates, but their coordinates are ordered differently, which leads to... Figure 10 and Figure 11 The polygons in the polygon are different. Therefore, when adding the coordinates of a new point, it is necessary to determine the position of the new point's coordinates within the polygon's set of point coordinates.

[0187] Figure 12 This application illustrates the operation of adding points to a polygon on an image to be annotated. For example... Figure 12 As shown, the original set of point coordinates for polygon ABCDEF is {A(x1,y1), B(x2,y2), C(x3,y3), D(x4,y4), E(x5,y5), F(x6,y6)}. These six points A, B, C, D, E, and F can form six edges: AB, BC, CD, DE, EF, and FA. The operation point is P, which is not on any of the six edges of polygon ABCDEF. For the edge AB formed by adjacent points A(x1,y1) and B(x2,y2), the distance from the operation point P(x,y) to edge AB is calculated using the vector method. The calculation formula is as follows: 1. Calculate the projection parameters (scale values) from P(x,y) to edge AB.

[0188] t (1) Where t represents the projection parameter. Indicates the length of edge AP. This represents the length of side AB.

[0189] 2. Calculate the mapping point from P(x,y) to edge AB.

[0190] If t < 0, the projection point of P(x,y) on edge AB is located outside the starting point A(x1,y1) of edge AB, and the mapping point ab = A(x1,y1).

[0191] If t>1, the projection point of P(x,y) on edge AB is located outside the endpoint B(x2,y2) of edge AB, and the mapping point ab = B(x2,y2).

[0192] If 0 <= t <= 1, the projection point of P(x,y) on edge AB is located on edge AB, and the mapping point ab = (x1 + t*(x2-x1), y1 + t * (y2-y1)).

[0193] 3. Calculate the distance from P(x,y) to edge AB.

[0194] The three cases ab = A(x1,y1), ab = B(x2,y2), and ab = (x1 + t*(x2-x1), y1 + t * (y2-y1)) are all denoted as ab(x3,y3).

[0195] d (2) Where d represents distance.

[0196] 4. Determine the shortest distance from the operation point p to edges AB, BC, CD, DE, EF, and FA, and take the edge corresponding to the shortest distance as the first target edge.

[0197] 5. The mapping point of operation point p on the first target edge is taken as the new point.

[0198] 6. Insert the coordinates (x3, y3) of the newly added point between the coordinates of the two points corresponding to the first target edge. Assuming the first target edge is edge AB, insert the mapped point ab(x3, y3) between A(x1, y1) and B(x2, y2) to obtain the coordinate set of the second new point {A(x1, y1), ab(x3, y3), B(x2, y2), C(x3, y3), D(x4, y4), E(x5, y5), F(x6, y6)}.

[0199] 7. Redraw polygon A-ab-BCDEF on the image to be labeled according to the second set of new point coordinates.

[0200] Unlike the aforementioned embodiments, this method, on the one hand, can respond in real time to the addition of a point and redraw the polygon and update its graphic position information, allowing users to clearly observe the position of the added point within the polygon. On the other hand, after obtaining the operation point, it acquires the mapping point of the operation point on the edge of the polygon and adds the mapping point to the original point coordinate set of the polygon to obtain a second new point coordinate set. Thus, the user's operation point can be located on or off the edge of the polygon, and the addition of a point can be achieved regardless of whether it is located on the edge of the polygon, improving the flexibility and efficiency of annotation.

[0201] Figure 13 This is a flowchart illustrating an embodiment of the image annotation method provided in this application. Figure 13 As shown, in this embodiment, the drawing-related operations include a graphic clipping operation. The graphic type of the labeled graphic is a polygon, and the graphic position information corresponding to the polygon is a set of point coordinates, which includes the coordinates of several points on the polygon. The graphic clipping operation refers to using a clipping graphic to clip the polygon on the image to be labeled in order to update the graphic position information of the polygon. The execution order of each step in this embodiment is not limited to the order shown in the figure. S611-S615 are further extensions of S140, and S616 is a further extension of S131. The image annotation method may include the following steps: S611: In response to a user performing a graphic cropping operation on a polygon in an image to be labeled, based on the original set of point coordinates of the polygon, determine several first sides of the polygon and several second sides of the cut graphic.

[0202] S612: Find several intersection points between several first edges and several second edges.

[0203] S613: From a number of first edges, determine the second target edge to which each intersection point belongs.

[0204] S614: In the original set of point coordinates of the polygon, insert the coordinates of each intersection point between the coordinates of the two points corresponding to the second target edge to obtain the third new set of point coordinates.

[0205] S615: Construct a new polygon according to the third new point coordinate set, and remove the part of the new polygon covered by the clipped graphic to obtain the clipped polygon.

[0206] S616: Use the third new set of point coordinates to determine the position information of the cut polygon.

[0207] For ease of understanding, examples of S611-S616 are given below.

[0208] Figure 14 This is a schematic diagram illustrating how a user performs an image cropping operation on a polygon in an image to be labeled, using a clipping graphic. For example... Figure 14 As shown, a polygon on the image to be labeled is a1-b1-c1-d1-f1, and the cut-out shape is a2-b2-c2.

[0209] 1. Based on the original set of point coordinates {a1, b1, c1, d1, e1} of the polygon a1-b1-c1-d1-e1, the first side a1b1, b1c1, c1d1, d1e1, e1a1 can be determined. And based on the original set of point coordinates {a2, b2, c2} of the cut shape a2-b2-c2, the second side a2b2, b2c2, c2a2 can be determined.

[0210] 2. Use the intersect method of the polybooljs library to obtain the intersection points a3, b3, and c3 of each first edge and each second edge.

[0211] 3. Determine the second target edges belonging to a3, b3, and c3 from each of the first edges as a1e1, b1c1, and d1e1, respectively.

[0212] 4. In the original set of point coordinates of polygon a1-b1-c1-d1-e1, insert a3 between e1 and a1, insert b3 between b1 and c1, and insert c3 between d1 and e1 to obtain a third new set of point coordinates {a1, b1, b3, c1, d1, c3, e1, a3}.

[0213] 5. Construct a new polygon a1-b1-b3-c1-d1-c3-e1-a3 based on the coordinates of the third new point, and use the difference method provided by polybooljs to remove the part covered by the clipping graphic a2-b2-c2 in the new polygon a1-b1-b3-c1-d1-c3-e1-a3 to obtain the clipped polygon.

[0214] In some embodiments, the graphic position information of a custom graphic may include a set of point coordinates and custom graphic reference information. The set of point coordinates includes the coordinates of several points on the custom graphic, and the custom graphic reference information includes the connection rules of the custom graphic and a position reference point. The connection rules of the custom graphic include the method of dividing the coordinates of several points on the custom graphic and the order of connecting the coordinates of each point in the resulting subset. The position reference point can be any point in the set of point coordinates or the center point of the custom graphic.

[0215] For custom graphics, the drawing operations include drawing the custom graphic (also known as adding a new custom graphic) and moving at least one template point (also known as moving a point in the custom graphic). The drawing operation adds a custom graphic to the image to be labeled. The moving operation moves at least one template point of the custom graphic.

[0216] The custom graphic can be either a closed shape or a non-closed shape. Non-closed shapes can be, for example, human faces or human figures.

[0217] Figure 15 This is a flowchart illustrating an embodiment of the image annotation method provided in this application. In this embodiment, the drawing-related operations include drawing a custom graphic. The graphic type of the annotation graphic is a custom graphic. The graphic position information of the custom graphic includes a set of point coordinates and custom graphic reference information. The set of point coordinates includes the coordinates of several points on the custom graphic, and the custom graphic reference information includes the connection rules of the custom graphic. This embodiment is a further extension of S140. Figure 15 As shown, the image annotation method may include the following steps: S711: In response to a drawing operation of a custom graphic, obtain the template graphic position information of the custom graphic.

[0218] The template graphic location information includes a set of template point coordinates and template reference information, and the template reference information includes template connection rules.

[0219] The template graphic position information can be regarded as the initial graphic position information of the custom graphic, the template point coordinate set can be regarded as the initial point coordinate set of the custom graphic, the template reference information can be regarded as the initial reference information of the custom graphic, and the template connection rules can be regarded as the initial connection rules of the custom graphic.

[0220] S712: Connect the template points corresponding to the template point coordinate set using the template connection rules to obtain the custom graphic drawn this time.

[0221] In some embodiments, the template point coordinate set includes the absolute coordinates of several template points. The absolute coordinates of each template point can be directly connected using template connection rules to obtain the custom graphic being drawn.

[0222] In some embodiments, the template point coordinate set includes the relative coordinates of several template points, and the template graphic position information also includes a template position reference point for the custom graphic, where the template position reference point is a template point in the template point coordinate set. In this case, S712 includes: obtaining the absolute coordinates of the template reference point; obtaining the conversion relationship between the absolute and relative coordinates of the template reference point; converting the relative coordinates of each template point in the template point coordinate set using the conversion relationship to obtain the absolute coordinates of each template point; and connecting the absolute coordinates of each template point using template connection rules to obtain the custom graphic drawn this time.

[0223] S713: In response to a move operation on at least one template point on the custom graphic being drawn, update the set of template point coordinates using the coordinates of the moved at least one template point to obtain a fourth new set of point coordinates.

[0224] In some embodiments, the coordinates of the template points in the template point set are not related, and the movement of some template points will not cause the movement of the coordinates of other template points. In this case, the coordinates of the template points before the movement can be directly updated using the coordinates of the moved template points to obtain the fourth new set of point coordinates. For example, the set of template point coordinates for a human figure includes the coordinates of template points 1-17. If template point 17 is moved, the coordinates of the moved template point 17 can be used to update the coordinates of the original template point 17 in the set of template point coordinates for the human figure, thus obtaining the fourth new set of point coordinates for the human figure.

[0225] In some embodiments, the coordinates of the template points in the template point coordinate set are related, and the movement of the coordinates of some template points will cause the coordinates of other template points to move. In this case, the coordinates of the moved template points and their related relationships can be used to obtain the coordinates of the other template points after the movement; then, the coordinates of each template point in the template point coordinate set are updated to obtain a fourth new set of point coordinates. For example, the template point coordinate set of the human figure includes the coordinates of template points 1-17. Template point 17 is moved. The coordinates of the moved template point 17 and their related relationships can be used to obtain the coordinates of the moved template points 1-16. The coordinates of the moved template points 1-17 are then used to update the coordinates of the original template points 1-17 in the template point coordinate set to obtain a fourth new set of point coordinates for the human figure.

[0226] S714: Use template connection rules to connect the template points corresponding to the coordinate set of the fourth new point to obtain a redrawn custom graphic.

[0227] Understandably, in related technologies, when users draw custom graphics on an image to be labeled, they need to draw points one by one and then manually connect the drawn points. For example, Figure 16 This is a schematic diagram of the applicant's face. For example... Figure 16 As shown, the face graphic includes 68 facial key points. During the face graphic drawing operation of the image to be labeled, the user needs to draw 68 facial key points one by one on the image to be labeled, and then manually select polylines to connect these 68 facial key points to form a face graphic. Figure 17 This is a schematic diagram of the applicant's anatomy. For example... Figure 17 As shown, during the process of drawing a human figure on an image to be labeled, the user needs to draw these 17 key human body points one by one on the image to be labeled, and then manually select polylines to connect these 17 key human body points on the image to be labeled to form a human figure. Therefore, the workload of drawing custom figures is large and the drawing efficiency is extremely low.

[0228] In this embodiment, since the custom graphic has template connection rules and a set of template point coordinates, when the user needs to draw the custom graphic, the template connection rules can be used directly to connect each template point in the set of template point coordinates in response to the drawing operation of the custom graphic, thus obtaining the custom graphic being drawn. Therefore, the user does not need to draw each point one by one and then manually connect the drawn points, thereby reducing the workload of drawing custom graphics and improving drawing efficiency.

[0229] For ease of understanding, examples of S711-S714 are given below.

[0230] The labeled scenario is a biometric OCR scenario, and the custom graphic is a face.

[0231] 1. The template image location information of the face image includes the set of template point coordinates "points", the template connection rules "lines", and the template position reference point "centerPoint".

[0232] The template point coordinate set "points" includes the relative coordinates of 68 facial keypoints. The template connection rules "lines" include the connection rules for the 68 facial keypoints. The template position reference point "centerPoint" is a facial keypoint from the template point coordinate set "points".

[0233] The template point coordinate set "points" includes the relative coordinates of facial key points 1-68, where facial key points 1-17 correspond to the facial contour, facial key points 18-22 correspond to the left eyebrow, facial key points 23-27 correspond to the right eyebrow, facial key points 28-31 correspond to the bridge of the nose, facial key points 32-36 correspond to the nostrils, facial key points 37-42 correspond to the left eye, facial key points 43-48 correspond to the right eye, facial key points 49-60 correspond to the outer lips, and facial key points 61-68 correspond to the inner lips.

[0234] The template connection rule "lines" is {1-17, 18-22, 23-27, 28-31, 32-36, (37, 38, 39, 40, 41, 42, 37), (43, 44, 45, 46, 47, 48, 43), (49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 49), (61, 62, 63, 64, 65, 66, 67, 68, 61)}. This divides the facial key points 1-68 into 9 subsets, with each subset representing a connection unit. The order of the facial key points within each subset determines the connection order.

[0235] The template location reference point "centerPoint" is face key point 2.

[0236] 1. When a user needs to draw a custom graphic, the "Custom Annotation Function" will be enabled in response to the user's activation of the "Custom Annotation Function" on the annotation interface.

[0237] 2. Receive the coordinates of the selected point on the image to be labeled by the user (coordinates of mouse click or hover).

[0238] 3. Use the coordinates of the selected point as the absolute coordinates of the template position reference point "centerPoint".

[0239] 4. Based on the conversion relationship between the absolute and relative coordinates of the template position reference point "centerPoint", the relative coordinates of each facial key point in the template point coordinate set "points" are converted to obtain the absolute coordinates of each facial key point.

[0240] 5. Use the template connection rule "lines" to connect the absolute coordinates of each facial key point to obtain the face graphic drawn in this drawing.

[0241] Specifically, the absolute coordinates of facial key points 1-17, 18-22, 23-27, 28-31, and 32-36 are connected sequentially, and the connection results are non-closed sub-figures of facial contour, left eyebrow, right eyebrow, bridge of nose, and alar of nose, respectively. Furthermore, the absolute coordinates of the facial key points (37, 38, 39, 40, 41, 42, 37) are connected sequentially to form the closed sub-shape of the left eye; the absolute coordinates of the facial key points (43, 44, 45, 46, 47, 48, 43) are connected sequentially to form the closed sub-shape of the right eye; the absolute coordinates of the facial key points (49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 49) are connected sequentially to form the closed sub-shape of the outer lip; and the absolute coordinates of the facial key points (61, 62, 63, 64, 65, 66, 67, 68, 61) are connected sequentially to form the closed sub-shape of the inner lip. All these sub-shapes together constitute the facial image drawn in this experiment.

[0242] The image annotation method provided in this application will be illustrated below with a specific example.

[0243] I. Software Architecture Figure 18 This is a schematic diagram of the software architecture of the image annotation method in this application. For example... Figure 18 As shown, the software architecture of the image annotation method includes a service layer, a data layer, a core layer, a business layer, and a scene layer.

[0244] The service layer includes a preprocessing service module and an engine service module. The preprocessing service module performs format conversion, size normalization, and quality optimization on the original image to obtain the image to be labeled. The engine service module performs AI annotation on the image to be labeled, obtaining AI-annotated image annotation data, which serves as the AI ​​annotation result for the image to be labeled.

[0245] The data layer includes a metadata module, a graphic annotation data module, and a user interface configuration data module. The metadata module stores metadata, the graphic annotation data module stores graphic annotation data, and the user interface configuration data module stores user interface configuration data.

[0246] The core layer includes a rendering engine module (based on an SVG.js-encapsulated drawing library). The rendering engine module is used to render metadata, interface configuration data, images to be annotated, and graphic annotation data to the annotation interface.

[0247] The business layer includes a drawing layer module, an annotation layer module, and a configuration layer module. The drawing layer module manages the drawing area. The annotation layer module manages the annotation area, and the configuration layer module manages the configuration area.

[0248] The scenario layer includes a math OCR module, an education OCR module, an intelligent driving OCR module, and other OCR scenario modules. The math OCR module is used to adapt to math OCR annotation scenarios, the education OCR module is used to adapt to education OCR annotation scenarios, the intelligent driving OCR module is used to adapt to intelligent driving OCR annotation scenarios, and the other OCR scenario modules are used to adapt to other OCR annotation scenarios.

[0249] II. Labeling Process Figure 19 This is a schematic diagram of the annotation process for this application. For example... Figure 19 As shown, the annotation process includes: 1. Task Creation The administrator can create image annotation tasks for images to be annotated through the task creation interface of the management terminal of the annotation platform (CV platform), and configure the parameters of the graphic annotation task through the task configuration interface to obtain interface configuration data and metadata.

[0250] Administrators can configure the field values ​​for each function item based on the annotation scenario through the task configuration interface to obtain the interface configuration data. Setting the field value corresponding to a function item as the target field value means providing the corresponding annotation function for the image annotation task. For example, in the scene function items, the field values ​​corresponding to the original image annotation, batch operation, quick annotation, and attribute inheritance functions are target field values. In the graphic function items, the field values ​​corresponding to the rectangle annotation, polygon annotation, polyline annotation, circle annotation, custom annotation, graphic merging, graphic smearing, add point, and delete point functions are target field values.

[0251] 2. Task Acceptance Annotators can view the image annotation task list through the annotation platform's annotation terminal, and select and receive image annotation tasks from it.

[0252] 3. Display annotation interface In response to the annotator's start of the image annotation task, the annotation interface is displayed on the annotation terminal according to the interface configuration data, and the image to be annotated is displayed on the annotation interface.

[0253] Figure 20 This is another schematic diagram of the annotation interface in this application. For example... Figure 20 As shown, the annotation interface includes a drawing area, a configuration area, and an annotation area. The configuration area includes a scene sub-configuration area and a graphics sub-configuration area. The drawing area displays the image to be annotated. The scene sub-configuration area displays scene function items including original image annotation, batch operation, quick annotation, and attribute inheritance. The graphics sub-configuration area displays graphics function items including rectangle annotation, polygon annotation, polyline annotation, circle annotation, custom annotation, graphics merging, graphics smudge, add point, and delete point.

[0254] The AI-annotated graphic data is used to display the AI-annotated graphic on the image to be annotated, as well as the corresponding attribute information and text information of the AI-annotated graphic in the annotation area.

[0255] Therefore, the annotation interface presents the annotator with a reasonable interface layout (configuration area, drawing area and annotation area), annotation tools (function items) and AI image annotation data, making it easier for the annotator to annotate efficiently.

[0256] 4. Image annotation Annotators can interact with the annotation interface to perform annotation-related operations, update the graphic annotation data, and use it as the annotation result for the image to be annotated.

[0257] 5. Saving and subsequent processing of annotation results After annotation is completed, the annotators save the annotation results to the annotation platform's storage via the annotation terminal, awaiting subsequent processing. Subsequent processing includes exporting, cleaning, and training.

[0258] III. Image Annotation Methods Taking license plate annotation in a vehicle image as an example, the annotation requires marking the rectangular box containing the license plate location (graphical location information), the license plate number (text information), and the license plate attributes (attribute information, such as whether it's a gasoline vehicle or an electric vehicle). After displaying the annotation interface for the vehicle image, users can perform annotation-related operations on the interface.

[0259] In response to the user's action of enabling the rectangle annotation function, the rectangle annotation function is enabled. Figure 21 This is another schematic diagram of the annotation interface in this application. Compared to Figure 20 , Figure 21 The rectangle annotation function has turned gray, which means that the rectangle annotation function has been turned on.

[0260] In response to the user's operation of drawing a rectangle on the vehicle image, the system obtains the graphic position information of the rectangle and draws the rectangle on the vehicle image according to the graphic position information.

[0261] In response to the user's semantic-related operations in the annotation area, obtain the license plate annotation text "A123456" and the annotation attribute "oil vehicle".

[0262] Figure 22 This is a schematic diagram of the structure of an embodiment of the image annotation device of this application. For example... Figure 22 As shown, the image annotation device 80 includes a display module 81, a receiving module 82, and a generating module 83. Wherein: Display module 81 is used to display the annotation interface according to the interface configuration and display the image to be annotated on the annotation interface.

[0263] The receiving module 82 is used to receive annotation-related operations performed by the user on the annotation interface.

[0264] The generation module 83 is used to generate graphic annotation data corresponding to annotation-related operations, so as to serve as the annotation result of the image to be annotated.

[0265] The interface configuration data is obtained by configuring the task configuration interface according to the annotation scenario corresponding to the image to be annotated. The interface configuration data includes fields and field values ​​corresponding to several functional items. The field value corresponding to the functional item indicates whether the functional item is displayed in the annotation interface. The several functional items include a custom graphic annotation functional item, which corresponds to a custom graphic annotation function. The custom graphic annotation function supports annotation-related operations for custom graphics. And / or, the graphic annotation data is in the first standardized data format.

[0266] For further detailed descriptions of the image annotation device 80, please refer to the preceding embodiments, which will not be repeated here.

[0267] Figure 23 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. Figure 23 As shown, the electronic device 90 includes a memory 91 and a processor 92. The processor 92 is used to execute program instructions stored in the memory 91 to implement the steps in any of the above method embodiments. In a specific implementation scenario, the electronic device 90 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 90 may also include a laptop computer, a tablet computer, or other carrier device, which is not limited here.

[0268] Specifically, processor 92 controls itself and memory 91 to implement the steps in any of the above method embodiments. Processor 92 can also be referred to as a CPU (Central Processing Unit). Processor 92 may be an integrated circuit chip with signal processing capabilities. Processor 92 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 92 can be implemented using integrated circuit chips.

[0269] Please see Figure 24 , Figure 24 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 100 stores program instructions 101 thereon, which, when executed by a processor, implement the steps in any of the above method embodiments.

[0270] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0271] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0272] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. In another image location, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0273] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.< / number>

Claims

1. An image labeling method, characterized by, The method comprises: displaying a labeling interface according to interface configuration data, and displaying an image to be labeled on the labeling interface; receiving a labeling-related operation of a user on the labeling interface; generating graphical labeling data corresponding to the labeling-related operation as a labeling result of the image to be labeled; wherein the interface configuration data is configured through a task configuration interface according to a labeling scenario corresponding to the image to be labeled, the interface configuration data comprises fields and field values corresponding to a plurality of function items, the field values corresponding to the function items represent whether to display the function items on the labeling interface, the plurality of function items comprise a custom graphical labeling function item, the custom graphical labeling function item corresponds to a custom graphical labeling function, and the custom graphical labeling function supports labeling-related operations of a custom graph; and / or the graphical labeling data is in a first standardized data format.

2. The method of claim 1, wherein, The graphical labeling data comprises a plurality of labeling-related fields and corresponding field values of a labeling graph, and the plurality of labeling-related fields comprise a graph position field and a graph type field.

3. The method of claim 2, wherein, The graph position field comprises a first position field and a second position field, the first position field represents the position of a labeling graph with a rectangular graph type, the field value of the first position field comprises the coordinates of a target vertex of the rectangle and the size of the rectangle, and the second position field represents the position of a labeling graph with other graph shapes, and the field value corresponding to the second position field comprises the coordinates of a plurality of points of the labeling graph with the other graph shapes. And / or, the plurality of labeling-related fields further comprise at least one of the following: a graph identification field, a labeling text field, a labeling attribute field, a circular attribute field, a frame information field of a labeling graph, a filling information field of a labeling graph, and a custom graph reference information field, the custom graph reference information field comprises at least one of a connection rule field and a position reference point field of a custom graph. And / or, the generating of the graphical labeling data corresponding to the labeling-related operation comprises: determining the field values of at least one of the labeling-related fields based on the labeling-related operation to obtain the graphical labeling data.

4. The method of claim 1, wherein, The interface configuration data is in a second standardized data format; and / or The displaying of the labeling interface according to the interface configuration data comprises: displaying, on the labeling interface, a function item with a target field value in the interface configuration data, wherein the target field value represents that the function item is displayed on the labeling interface. And / or, the interface configuration data includes scene configuration data and graphic configuration data, the scene configuration data includes at least one corresponding field and field value of the scene function item, and the graphic configuration data includes at least one corresponding field and field value of the graphic function item, the scene function item includes at least one of the following: original label function item, image rotation function item, batch operation function item, quick labeling function item, data formula function item, attribute inheritance function item, graphic numbering function item, graphic attribute function item; The graphic function item includes at least one of the following: rectangle label function item, polygon label function item, polyline label function item, circle label function item, the custom graphic label function item, graphic merging function item, graphic smearing function item, new point function item, delete point function item.

5. The method of claim 1, wherein, The method further comprises: Obtaining metadata, determining information of the current labeling task and information of the image to be labeled based on the metadata, wherein the metadata is a third standardized data format, and the metadata includes a task identification field, a task state field and an image to be labeled information field, and the field values corresponding to the aforementioned fields.

6. The method of claim 1, wherein, The labeling interface includes a configuration area, a drawing area and a labeling area, wherein the corresponding component modules of the configuration area, the drawing area and the labeling area are independent, and the component modules can communicate with each other. The method further comprises: According to the interface configuration data, at least one function item is displayed in the configuration area of the labeling interface, and the image to be labeled is displayed in the drawing area of the labeling interface. The method further comprises: Obtaining the labeling text corresponding to the labeling graphic input by the user in the labeling area, or displaying the recognized text corresponding to the labeling graphic in the labeling area, obtaining the text modified by the user as the labeling text corresponding to the labeling graphic, wherein the recognized text is obtained by text recognition on the image part in the image to be labeled within the labeling graphic. The graphic labeling data includes graphic position information.

7. The method of claim 1, wherein, The method further comprises: In response to the user's drawing-related operation on the labeling graphic, the current labeling graphic is drawn on the image to be labeled. The method further comprises: In response to the user's drawing-related operation on the labeling graphic, the graphic position information of the current labeling graphic is obtained as the graphic labeling data corresponding to the labeling graphic. The drawing-related operation includes at least one of the new point operation and the graphic rotation operation, the graphic type of the labeling graphic is a polygon or a polyline, the graphic position information corresponding to the polygon or the polyline is a point coordinate set, and the point coordinate set includes the coordinates of a plurality of points on the polygon or the polyline.

8. The method of claim 7, wherein, ​ The response to the user drawing related operation of the annotation figure, drawing the current annotation figure on the image to be annotated, comprises: In response to a user adding a point operation on a polygon or a polyline on the image to be annotated, obtaining the added point coordinates of the user and a selected point from the plurality of points on the polygon or the polyline; copying a point coordinate set of the polygon or the polyline, obtaining a first new point coordinate set, adding the added point coordinates to the first new point coordinate set before or after the selected point, and obtaining a point coordinate set of the polygon or the polyline after adding the point; and redrawing the polygon or the polyline on the image to be annotated according to the point coordinate set of the polygon or the polyline after adding the point. In response to a user rotating a figure operation on a polygon or a polyline on the image to be annotated, obtaining a first rotation angle of a selected rotating operation point before and after rotation; updating the coordinates of each point in the point coordinate set of the polygon or the polyline before rotation by using the first rotation angle, to obtain a point coordinate set of the polygon or the polyline after rotation; and redrawing the polygon or the polyline on the image to be annotated according to the point coordinate set of the polygon or the polyline after rotation. The response to the user drawing related operation of the annotation figure, obtaining the figure position information of the currently drawn annotation figure, comprises: In response to a user adding a point operation or a figure rotating operation on a polygon or a polyline on the image to be annotated, the point coordinate set of the polygon or the polyline after adding the point or rotating is taken as the figure position information of the polygon or the polyline after adding the point or rotating.

9. The method of claim 7, wherein, The drawing related operation comprises a figure transformation operation, the figure type of the annotation figure is a rectangle, and the figure position information corresponding to the rectangle comprises the coordinates of a target vertex of the rectangle and the size of the rectangle. The response to the user drawing related operation of the annotation figure, drawing the current annotation figure on the image to be annotated, comprises: In response to a user performing a figure transformation operation on a rectangle on the image to be annotated, and the figure transformation operation comprising a figure rotating operation, obtaining a second rotation angle of the transformed rectangle relative to a standard direction, obtaining new coordinates of the target vertex when the transformed rectangle is rotated to the standard direction, and taking the new coordinates of the target vertex, the size of the rectangle after transformation, and the second rotation angle as the drawing reference information of the rectangle; and redrawing the current rectangle on the image to be annotated according to the drawing reference information of the rectangle. The response to the user drawing related operation of the annotation figure, obtaining the figure position information of the currently drawn annotation figure, comprises: In response to a user performing a figure transformation operation on a rectangle on the image to be annotated, and the figure transformation operation comprising a figure rotating operation, taking the drawing reference information as the figure position information of the transformed rectangle.

10. The method of claim 7, wherein, The drawing-related operation includes a new point operation, the graphic type of the annotation graphic is a polygon, and the graphic position information corresponding to the polygon is a point coordinate set, the point coordinate set including coordinates of points on the polygon; The operation of drawing the current annotation graphic on the image to be annotated in response to the user performing the drawing-related operation on the annotation graphic includes: In response to the user performing a new point operation on a polygon on the image to be annotated, an operation point selected by the user in a drawing area of the annotation interface is obtained; Based on the original point coordinate set of the polygon, a plurality of edges of the polygon are determined; A first target edge closest to the operation point is selected from the plurality of edges; A mapping point of the operation point on the first target edge is taken as a new point; The coordinates of the new point are inserted between the coordinates of two points corresponding to the first target edge in the original point coordinate set of the polygon to obtain a second new point coordinate set; The polygon is redrawn on the image to be annotated according to the second new point coordinate set; The operation of obtaining the graphic position information of the annotation graphic currently drawn in response to the user performing the drawing-related operation on the annotation graphic includes: In response to the user performing a new point operation on a polygon on the image to be annotated, the second new point coordinate set is taken as the graphic position information of the polygon after the new point.

11. The method of claim 7, wherein, The drawing-related operation includes a graphic clipping operation, the graphic type of the annotation graphic is a polygon, and the graphic position information corresponding to the polygon is a point coordinate set, the point coordinate set including coordinates of points on the polygon; The operation of drawing the current annotation graphic on the image to be annotated in response to the user performing the drawing-related operation on the annotation graphic includes: In response to the user performing a graphic clipping operation on a polygon on the image to be annotated, based on the original point coordinate set of the polygon, a plurality of first edges of the polygon are determined, and a plurality of second edges of a clipping graphic are determined; A plurality of intersection points between the plurality of first edges and the plurality of second edges are obtained; From the plurality of first edges, a second target edge to which each intersection point belongs is determined; The coordinates of each intersection point are inserted between the coordinates of two points corresponding to the second target edge in the original point coordinate set of the polygon to obtain a third new point coordinate set; A new polygon is constructed according to the third new point coordinate set, and a portion of the new polygon covered by the clipping graphic is removed to obtain the polygon after clipping; The operation of obtaining the graphic position information of the annotation graphic currently drawn in response to the user performing the drawing-related operation on the annotation graphic includes: In response to the user performing a graphic clipping operation on a polygon on the image to be annotated, the third new point coordinate set is used to determine the graphic position information of the polygon after clipping.

12. The method of claim 7, wherein, The drawing-related operation includes an adding operation of a custom graph, a graph type of the labeled graph is the custom graph, graph position information of the custom graph includes a point coordinate set, custom graph reference information, the point coordinate set includes coordinates of points on the custom graph, and the custom graph reference information includes a connection rule of the custom graph; The drawing-related operation includes an adding operation of a custom graph, a graph type of the labeled graph is the custom graph, graph position information of the custom graph includes a point coordinate set, custom graph reference information, the point coordinate set includes coordinates of points on the custom graph, and the custom graph reference information includes a connection rule of the custom graph; In response to the drawing operation of the custom graph, template graph position information of the custom graph is acquired, wherein the template graph position information includes a template point coordinate set and template reference information, and the template reference information includes a template connection rule; The template points corresponding to the template point coordinate set are connected according to the template connection rule, to obtain the custom graph drawn this time. In response to a moving operation on at least one template point of the custom graph drawn this time, coordinates of the at least one template point after moving are used to update the template point coordinate set, to obtain a fourth new point coordinate set; and the template points corresponding to the fourth new point coordinate set are connected according to the template connection rule, to obtain a custom graph drawn again.

13. The method of claim 12, wherein, The template point coordinate set includes relative coordinates of a plurality of template points, and the template graph position information further includes a template position reference point of the custom graph, the template position reference point being one of the template points in the template point coordinate set; The template points corresponding to the template point coordinate set are connected according to the template connection rule, to obtain the custom graph drawn this time. An absolute coordinate of the template reference point is acquired; A conversion relationship between the absolute coordinate and the relative coordinate of the template reference point is acquired; The relative coordinates of the template points in the template point coordinate set are converted according to the conversion relationship, to obtain absolute coordinates of the template points; The absolute coordinates of the template points are connected according to the template connection rule, to obtain the custom graph drawn this time.

14. An image labeling apparatus characterized by comprising: The display module is configured to display a labeling interface according to interface configuration data and display a to-be-labeled image on the labeling interface; The receiving module is configured to receive a labeling-related operation of a user on the labeling interface; The generating module is configured to generate graph labeling data corresponding to the labeling-related operation as a labeling result of the to-be-labeled image; The interface configuration data is configured according to a labeling scenario corresponding to the to-be-labeled image through a task configuration interface, and includes fields and field values corresponding to a plurality of function items, the field values of the function items representing whether to display the function items on the labeling interface, the plurality of function items including a custom graph labeling function item, the custom graph labeling function item corresponding to a custom graph labeling function, and the custom graph labeling function supporting labeling-related operations of a custom graph; and / or the graph labeling data is in a first standardized data format. ​ 15. An electronic device, comprising: comprising a memory and a processor for executing program instructions stored in the memory to implement the method of any one of claims 1 to 13.

16. A computer readable storage medium having stored thereon program instructions, wherein, The program instructions, when executed by a processor, implement the method of any one of claims 1 to 13.

Images