Labeling method and device, storage medium and computer equipment

Abstract

The invention discloses a labeling method and device, a storage medium and computer equipment, and the method comprises the steps: responding to a first interaction operation in an idle state of a labeling tool, and querying whether the first interaction operation belongs to a preset idle state transition operation or not; if the first interaction operation belongs to the idle state transfer operation, determining a first state corresponding to the first interaction operation in the idle state, and transferring the state of the labeling tool to the first state; in response to a second interaction operation in the first state, based on the operation type of the second interaction operation, data labeling is conducted on a labeled sample or the labeling tool is triggered to conduct state transition, and a second transferable state corresponding to the first state is not limited to the idle state. According to the method, the operation amount of labeling personnel is reduced, the operation efficiency is improved, and the intelligent degree of a labeling tool is improved.

Description

Technical Field

[0001] This application relates to the technical field of data annotation, and particularly to an annotation method, device, storage medium, and computer device. Background Art

[0002] Data annotation is an important task, which involves marking the original data set with the required categories or attributes. Specifically, data annotators need to read, watch, or listen to the original data (such as images, videos, audio, etc.), and add labels or annotations according to specific annotation rules. Taking the field of autonomous driving as an example, with the rapid development of artificial intelligence and the automotive industry, autonomous driving technology has become the focus of widespread attention inside and outside the industry. Currently, autonomous driving technology has been actually applied in many cities. For autonomous driving technology, well-annotated data is a very important foundation. By annotating a large amount of image, video, and sensor data, more accurate and reliable autonomous driving algorithms can be trained.

[0003] For data annotation work, generally, annotators need to perform multiple operations under different functions to complete the annotation of a sample. The working process of the existing annotation tools is that the annotator selects a function in the idle state of the tool, completes a series of operations under this function, and then exits this function and returns to the idle state. Then, the annotator selects the next function and performs operations. The annotation process is not smooth, the operation is cumbersome, and the annotation efficiency is low. Summary of the Invention

[0004] In view of this, this application provides an annotation method, device, storage medium, and computer device, which helps to reduce the operation amount of annotators, improve the operation efficiency, and enhance the intelligence level of the annotation tool.

[0005] According to one aspect of this application, an annotation method is provided, which is applied to an annotation tool based on a state machine. The method includes:

[0006] In response to a first interaction operation in the idle state of the annotation tool, query whether the first interaction operation belongs to a preset idle state transition operation, where the idle state transition operation includes an operation for triggering the annotation tool to transition from the idle state to a first transferable state corresponding to the idle state;

[0007] If the first interaction operation belongs to the idle state transition operation, determine the first state corresponding to the first interaction operation in the idle state, and transfer the state of the annotation tool to the first state, where the first transferable state includes the first state;

[0008] In response to a second interaction operation in the first state, based on the operation type of the second interaction operation, perform data annotation on the annotated sample or trigger the annotation tool to perform a state transition.

[0009] Optionally, after transferring the state of the annotation tool to the first state, the method further includes:

[0010] In the first state, perform data annotation on the annotated sample based on the first interaction operation.

[0011] Optionally, the performing data annotation on the annotated sample or triggering the annotation tool to perform a state transition based on the operation type of the second interaction operation includes:

[0012] Query whether the second interaction operation belongs to a preset first state transition operation, where the first state transition operation includes an operation for triggering the annotation tool to transfer from the first state to a second transferable state corresponding to the first state;

[0013] If the second interaction operation belongs to the first state transition operation, determine the second state corresponding to the second interaction operation in the first state, transfer the state of the annotation tool to the second state, and execute the data annotation process in the second state based on the annotated data in the first state and the second interaction operation, where the second transferable state includes the second state;

[0014] If the second interaction operation does not belong to the first state transition operation, perform data annotation on the annotated sample based on the second interaction operation.

[0015] Optionally, the transferable states corresponding to different states in the annotation tool do not overlap, partially overlap, or completely overlap; different states include an idle state and multiple functional states;

[0016] The annotation tool is configured with response operation information for different states, and the response operation information includes an operation behavior and a trigger constraint condition corresponding to at least one response operation, where the trigger constraint condition is used to determine whether an interaction operation generated according to the operation behavior can be responded to;

[0017] The response operation includes an annotation function operation and / or a state transition operation, the annotation function operation is used to trigger the execution of a data annotation process, and the state transition operation is used to trigger the execution of a state transition process.

[0018] Optionally, the functional state includes a polygon drawing state; the method further includes:

[0019] In response to a third interaction operation in the polygon drawing state, query whether the third interaction operation belongs to the actionable operation information in the polygon drawing state;

[0020] If it belongs, based on the current drawing data and the actionable operation information in the polygon drawing state, determine the target actionable operation corresponding to the third interaction operation, and based on the target actionable operation, perform vertex drawing on the annotated sample or trigger the annotation tool to perform a state transition;

[0021] If it does not belong, maintain the polygon drawing state; output an operation error prompt message, or do not respond to the third interaction operation.

[0022] Optionally, the annotation function operations in the polygon drawing state include vertex drawing operations, and also include a first drawing point deletion operation and / or a second drawing point deletion operation;

[0023] The operation behavior of the vertex drawing operation includes a first click behavior, and the trigger constraint condition of the vertex drawing operation includes that there is no drawn vertex at the click position; the vertex drawing operation is used to create a vertex at the click position;

[0024] The operation behavior of the first drawing point deletion operation includes a second click behavior, and the trigger constraint condition of the first drawing point deletion operation includes that the click position contains a drawn vertex; the first drawing point deletion operation is used to delete the drawn vertex at the click position;

[0025] The operation behavior of the second drawing point deletion operation includes a third click behavior, and the trigger constraint condition of the second drawing point deletion operation includes that there is at least one drawn vertex; the second drawing point deletion operation is used to delete the latest drawn vertex.

[0026] Optionally, the functional state further includes a polygon drawing adsorption search state;

[0027] The state transition operations in the polygon drawing state include a first end drawing operation and / or a second end drawing operation, and also include a polygon drawing adsorption search operation;

[0028] The operation behavior of the first end drawing operation includes a fourth click behavior, and the trigger constraint condition of the first end drawing operation includes that there is only one drawn vertex; the first end drawing operation is used to trigger the annotation tool to transfer to the idle state;

[0029] The operation behavior of the second end drawing operation includes a first input behavior, and the trigger constraint condition of the second end drawing operation is empty; the second end drawing operation is used to trigger the annotation tool to transfer to the idle state;

[0030] The operation behavior of the polygon drawing adsorption search operation includes a second input behavior. The triggering constraint condition for the polygon drawing adsorption search operation is that there is a drawn polygon. The polygon drawing adsorption search operation is used to trigger the annotation tool to transfer to the polygon drawing adsorption search state.

[0031] Optionally, based on the target responsive operation, triggering the annotation tool to perform a state transition includes at least one of the following:

[0032] When the target responsive operation is the first end drawing operation, trigger the annotation tool to transfer to the idle state and delete the vertices drawn in the polygon drawing state.

[0033] When the target responsive operation is the second end drawing operation, trigger the annotation tool to transfer to the idle state, generate a target polygon based on the vertices drawn in the polygon drawing state, and annotate the part circled by the target polygon on the annotated sample.

[0034] When the target responsive operation is the polygon drawing adsorption search operation, trigger the annotation tool to transfer to the polygon drawing adsorption search state, and query for the adsorbable vertex corresponding to the position of the mouse cursor among the vertices of the generated polygon.

[0035] According to another aspect of the present application, there is provided an annotation device applied to an annotation tool based on a state machine. The device includes:

[0036] An operation response module for, in response to a first interaction operation in the idle state of the annotation tool, querying whether the first interaction operation belongs to a preset idle state transition operation, where the idle state transition operation includes an operation for triggering the annotation tool to transfer from the idle state to a first transferable state corresponding to the idle state.

[0037] An operation execution module for, if the first interaction operation belongs to the idle state transition operation, determining a first state corresponding to the first interaction operation in the idle state and transferring the state of the annotation tool to the first state, where the first transferable state includes the first state; and,

[0038] In response to a second interaction operation in the first state, perform data annotation on the annotated sample or trigger the annotation tool to perform a state transition based on the operation type of the second interaction operation.

[0039] Optionally, the operation execution module is further configured to: after transferring the state of the annotation tool to the first state, in the first state, perform data annotation on the annotated sample based on the first interaction operation.

[0040] Optionally, the operation execution module is further configured to:

[0041] Query whether the second interaction operation belongs to a preset first state transition operation, where the first state transition operation includes an operation for triggering the annotation tool to transition from the first state to a second transferable state corresponding to the first state;

[0042] If the second interaction operation belongs to the first state transition operation, determine the second state corresponding to the second interaction operation in the first state, transfer the state of the annotation tool to the second state, and execute the data annotation process in the second state based on the already annotated data in the first state and the second interaction operation, where the second transferable state includes the second state;

[0043] If the second interaction operation does not belong to the first state transition operation, perform data annotation on the annotated sample based on the second interaction operation.

[0044] Optionally, the transferable states corresponding to different states in the annotation tool do not overlap, partially overlap, or completely overlap; different states include an idle state and multiple functional states;

[0045] The annotation tool is configured with response operation information for different states, and the response operation information includes an operation behavior and a trigger constraint condition corresponding to at least one response operation, and the trigger constraint condition is used to determine whether an interaction operation generated according to the operation behavior can be responded to;

[0046] The response operation includes an annotation function operation and / or a state transition operation, the annotation function operation is used to trigger the execution of a data annotation process, and the state transition operation is used to trigger the execution of a state transition process.

[0047] Optionally, the functional state includes a polygon drawing state; the operation execution module is further configured to:

[0048] In response to a third interaction operation in the polygon drawing state, query whether the third interaction operation belongs to the response operation information in the polygon drawing state;

[0049] If it belongs, based on the current drawing data and the actionable information in the polygon drawing state, determine the target actionable operation corresponding to the third interaction operation, and based on the target actionable operation, perform vertex drawing on the labeled sample or trigger the annotation tool to perform a state transition;

[0050] If it does not belong, maintain the polygon drawing state; output an operation error prompt message, or do not respond to the third interaction operation.

[0051] Optionally, the annotation function operations in the polygon drawing state include vertex drawing operations, and also include a first drawing point deletion operation and / or a second drawing point deletion operation;

[0052] The operation behavior of the vertex drawing operation includes a first click behavior, and the trigger constraint condition of the vertex drawing operation includes that there is no drawn vertex at the click position; the vertex drawing operation is used to create a vertex at the click position;

[0053] The operation behavior of the first drawing point deletion operation includes a second click behavior, and the trigger constraint condition of the first drawing point deletion operation includes that the click position contains a drawn vertex; the first drawing point deletion operation is used to delete the drawn vertex at the click position;

[0054] The operation behavior of the second drawing point deletion operation includes a third click behavior, and the trigger constraint condition of the second drawing point deletion operation includes that there is at least one drawn vertex; the second drawing point deletion operation is used to delete the latest drawn vertex.

[0055] Optionally, the function state further includes a polygon drawing adsorption search state;

[0056] The state transition operations in the polygon drawing state include a first end drawing operation and / or a second end drawing operation, and also include a polygon drawing adsorption search operation;

[0057] The operation behavior of the first end drawing operation includes a fourth click behavior, and the trigger constraint condition of the first end drawing operation includes that there is only one drawn vertex; the first end drawing operation is used to trigger the annotation tool to transfer to the idle state;

[0058] The operation behavior of the second end drawing operation includes a first input behavior, and the trigger constraint condition of the second end drawing operation is empty; the second end drawing operation is used to trigger the annotation tool to transfer to the idle state;

[0059] The operation behavior of the polygon drawing adsorption search operation includes a second input behavior, and the trigger constraint condition of the polygon drawing adsorption search operation is that there is a drawn polygon; the polygon drawing adsorption search operation is used to trigger the annotation tool to transfer to the polygon drawing adsorption search state.

[0060] Optionally, the operation execution module is further configured to:

[0061] When the target responsive operation is the first end drawing operation, trigger the annotation tool to transfer to the idle state, and delete the vertices drawn in the polygon drawing state;

[0062] When the target responsive operation is the second end drawing operation, trigger the annotation tool to transfer to the idle state, and generate a target polygon based on the vertices drawn in the polygon drawing state, and mark the part circled by the target polygon on the annotated sample;

[0063] When the target responsive operation is the polygon drawing adsorption search operation, trigger the annotation tool to transfer to the polygon drawing adsorption search state, and query the adsorbable vertex corresponding to the position of the mouse cursor among the vertices of the generated polygon.

[0064] According to another aspect of the present application, there is provided a storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned annotation method is implemented.

[0065] According to still another aspect of the present application, there is provided a computer device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, and when the processor executes the program, the above-mentioned annotation method is implemented.

[0066] By means of the above technical solution, an annotation method, device, storage medium, and computer device provided by the present application, during the process of an annotator using a state machine-based annotation tool for sample annotation, in the idle state of the annotation tool, when receiving a first interaction operation belonging to the idle state transition operation, automatically trigger the annotation tool to transfer from the idle state to the corresponding first state, and in the first state of the annotation tool, the annotator can perform data annotation on the annotated sample within the annotation function range provided by the first state, or directly transfer from the first state to other states not limited to the idle state. The embodiment of the present application configures a state machine for the annotation tool, enabling the annotation tool to directly transfer from a certain functional state to other functional states without going through the transition of the idle state, which helps to reduce the operation amount of the annotator, improve the operation efficiency, and enhance the intelligence level of the annotation tool.

[0067] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specifically illustrates the specific implementation manners of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0069] Figure 1 A flowchart showing a labeling method provided by an embodiment of the present application is shown;

[0070] Figure 2 A schematic diagram showing the transfer relationship between different states in a state machine provided by an embodiment of the present application is shown;

[0071] Figure 3 A schematic diagram showing the transfer relationship between different states in another state machine provided by an embodiment of the present application is shown;

[0072] Figure 4 A schematic structural diagram of a labeling device provided by an embodiment of the present application is shown;

[0073] Figure 5 A schematic diagram of the device structure of a computer device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0074] The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0075] In this embodiment, a labeling method is provided, as Figure 1 shown, which is applied to a labeling tool based on a state machine. The method includes:

[0076] Step 101, in response to a first interaction operation in the idle state of the labeling tool, query whether the first interaction operation belongs to a preset idle state transition operation, where the idle state transition operation includes an operation for triggering the labeling tool to transition from the idle state to a first transferable state corresponding to the idle state.

[0077] Step 102, if the first interaction operation belongs to the idle state transition operation, determine a first state corresponding to the first interaction operation in the idle state, and transfer the state of the labeling tool to the first state, where the first transferable state includes the first state.

[0078] The annotation method provided by the embodiment of the present application is applied in a state machine-based annotation tool. By modeling various behaviors and states during the annotation process, different functional states are abstracted, the conversion relationships between various states are established, a complete state machine is constructed, and the state machine is configured in the annotation tool, so as to manage the annotation interaction operation process, improve the annotation efficiency, and facilitate the use of annotators.

[0079] In this embodiment, when an annotator uses the annotation tool for annotation, the annotation tool is opened to enter the idle state. In the idle state, the annotator can select the original data to be annotated. In this application, the original data to be annotated is taken as a picture sample for explanation. The annotator can select one or more samples to be annotated in the idle state. In the idle state of the annotation tool, the annotator can trigger the annotation tool to enter the functional state by performing a specific type of interaction operation. Among them, the state that can be directly transferred from the idle state is called the first transferable state corresponding to the idle state. The state machine of the annotation tool is pre-configured with operations for triggering the annotation tool to transfer from the idle state to each first transferable state, that is, the idle state transfer operation. In response to the annotator performing the first interaction operation in the idle state, it is queried whether the first interaction operation belongs to the pre-set idle state transfer operation. If it belongs, the state transfer of the annotation tool is performed based on the first interaction operation. If it does not belong, the corresponding process in the idle state is executed based on the first interaction operation. For example, in the idle state of the annotation tool, an idle state operation interface is displayed. The annotator can click the "Select Annotation Sample" function on this interface. The click operation of the annotator on "Select Annotation Sample" does not belong to the idle state transfer operation and only triggers the annotation tool to execute the function in the idle state. However, if the annotator selects the "Polygon Drawing" function and clicks the left mouse button in the idle state, it can trigger the annotation tool to transfer from the idle state to the polygon drawing state, display the polygon drawing state operation interface, and at the same time the annotation tool starts to execute the operation command in the polygon drawing state.

[0080] Optionally, after transferring the state of the annotation tool to the first state, the method further includes: in the first state, performing data annotation on the sample to be annotated based on the first interaction operation.

[0081] In this embodiment, for example, after selecting the polygon drawing tool in the idle state and clicking the left mouse button, this operation behavior can trigger the annotation tool to enter the polygon drawing state from the idle state. In the polygon drawing state, the annotation tool can draw the first vertex at the mouse click position. Thus, the operation amount is further reduced, the smooth transition between different states is realized, and the degree of intelligence is improved.

[0082] Optionally, the transferable states corresponding to different states in the annotation tool do not overlap, partially overlap, or completely overlap; the different states include an idle state and multiple functional states; the annotation tool is configured with response operation information for different states, and the response operation information includes an operation behavior and a trigger constraint condition corresponding to at least one response operation, and the trigger constraint condition is used to determine whether an interaction operation generated according to the operation behavior can be responded to; the response operations include an annotation function operation and / or a state transfer operation, the annotation function operation is used to trigger the execution of a data annotation process, and the state transfer operation is used to trigger the execution of a state transfer process.

[0083] In this embodiment, the annotation tool can provide an idle state and multiple functional states. Different functional states can complete annotation work of different functions. For example, the functional states include a polygon drawing state, a polygon drawing adsorption search state, a search segmentation state, a search movement state, a movement state, a movement adsorption search state, a distance measurement state, a polygon segmentation search state, an audit arrow drawing state, and so on. Any state can be transferred to one or more other states. For example, in the polygon drawing state, it can be directly transferred to the polygon drawing adsorption search state without having to first exit the polygon drawing state and return to the idle state and then enter the polygon drawing adsorption search state from the idle state. The other states that can be directly transferred to in a certain state are called the transferable states corresponding to this state, and the transferable states corresponding to different states can be the same, partially the same, or completely different. For example, the transferable states corresponding to functional state 1 are the idle state, functional state 2, and functional state 3; the transferable states corresponding to functional state 2 are the idle state and functional state 1; the transferable states corresponding to functional state 3 are the idle state and functional state 1; the transferable states corresponding to functional state 4 are the idle state.

[0084] Such as Figure 2As shown in the figure, a schematic diagram of the transition relationship between different states in a state machine is shown. Each state corresponds to its own transferable states, and specific operation behaviors can trigger the annotation tool to transfer from one state to another. For example, in the idle state (i.e., the initial state), operation behavior 1 will trigger the annotation tool to transfer to state 4, operation behavior 3 will trigger the annotation tool to transfer to state 2, operation behavior 6 will trigger the annotation tool to transfer to state 1, and operation behavior 11 will trigger the annotation tool to transfer to state 3. In addition, the transfer of the annotation tool from one state to another is not only related to whether an operation behavior occurs, but can also be restricted by specific conditions, that is, a specific operation behavior occurs in one state and meets specific trigger constraint conditions to transfer to another state. For example, when transferring from the idle state to the distance measurement state, in addition to generating the operation behaviors of "clicking the ruler tool" and "clicking the left mouse button" in the idle state, it is also necessary to meet the condition that the annotated sample has been opened. As Figure 3 As shown in the figure, a schematic diagram of the transition relationship between different states in a state machine is shown.

[0085] In addition, the responsive operations corresponding to any state can include annotation function operations and state transfer operations in that state. For example, the annotation function operations in the polygon drawing state include the operation of drawing the vertices of the polygon, and the state transfer operations include the operations that trigger the annotation tool to transfer to other states.

[0086] Step 103, in response to a second interaction operation in the first state, based on the operation type of the second interaction operation, perform data annotation on the annotated sample or trigger the annotation tool to perform a state transfer, where the second transferable state corresponding to the first state is not limited to the idle state.

[0087] In this embodiment, after the annotation tool transfers from the idle state to the first state, it will respond to the generated interaction operations in the manner in the first state. Specifically, sample annotation can be performed in the first state or a state transfer can occur again. For example, in an autonomous driving sample picture, the vehicles, pedestrians, other static objects, etc. to be recognized are annotated. The transferable states corresponding to the first state are not limited to the idle state, and can also include other functional states. For example, in the polygon drawing state, in response to the operation of the annotator clicking the left mouse button, a vertex is drawn on the annotated sample, and in response to the operation of pressing the "Alt" key on the keyboard, it transfers from the current polygon drawing state to the polygon drawing adsorption search state. Thus, it is possible to directly transfer from a certain functional state to other functional states without going through the transition of the idle state, reducing the operation amount of the annotator and improving the operation efficiency.

[0088] Optionally, step 103 includes:

[0089] Step 103-1: Query whether the second interaction operation belongs to a preset first state transition operation, where the first state transition operation includes an operation for triggering the annotation tool to transition from the first state to a second transferable state corresponding to the first state;

[0090] Step 103-2: If the second interaction operation belongs to the first state transition operation, determine the second state corresponding to the second interaction operation in the first state, transfer the state of the annotation tool to the second state, and execute the data annotation process in the second state based on the already annotated data in the first state and the second interaction operation, where the second transferable state includes the second state;

[0091] Step 103-3: If the second interaction operation does not belong to the first state transition operation, perform data annotation on the sample to be annotated based on the second interaction operation.

[0092] In this embodiment, in the first state of the annotation tool, in response to a second interaction operation, first identify the operation type of the second interaction operation. If the second interaction operation is an annotation operation, then perform data annotation on the sample to be annotated based on the second interaction operation. If the second interaction operation is a state transition operation, then transfer the annotation tool to another state. Specifically, when receiving the second interaction operation, it can first be queried whether the second interaction operation belongs to the first state transition operation corresponding to the first state. If it belongs, it means that the second interaction operation is a state transition operation, and then further query the state that the second interaction operation should trigger for transfer in the first state, that is, determine the second state corresponding to the second interaction operation in the first state, so as to perform the state transition. In addition, in addition to transferring to the second state, the annotation tool can also determine the annotation intention of the annotator based on the already annotated data in the first state and the second interaction operation, so as to further perform data annotation after transferring to the second state. For example, in the polygon drawing state, when the annotator presses the "Alt" key to trigger the annotation tool to enter the polygon drawing adsorption search state, the annotation tool can automatically query the adsorbable points of the last vertex drawn in the polygon drawing state and display them. Further reducing the operation amount of the annotator, improving the intelligence level of the annotation tool, and improving the annotation efficiency.

[0093] By applying the technical solution of this embodiment, during the process of a annotator using a state machine-based annotation tool for sample annotation, in the idle state of the annotation tool, when a first interaction operation belonging to the idle state transition operation is received, the annotation tool is automatically triggered to transfer from the idle state to the corresponding first state. And in the first state of the annotation tool, the annotator can perform data annotation on the sample to be annotated within the range of annotation functions provided by the first state, or directly transfer from the first state to other states not limited to the idle state. By configuring a state machine for the annotation tool in the embodiment of the present application, the annotation tool can directly transfer from a certain functional state to other functional states without going through the transition of the idle state, which helps to reduce the operation amount of the annotator, improve the operation efficiency, and enhance the intelligence level of the annotation tool.

[0094] Further, as a refinement and extension of the specific implementation manner of the above embodiment, taking the annotation in the polygon drawing state as an example, the embodiment of the present application also provides an annotation method, which includes:

[0095] Step 201, in response to a third interaction operation in the polygon drawing state, query whether the third interaction operation belongs to the responsive operation information in the polygon drawing state.

[0096] Step 202, if it belongs, based on the current drawing data and the responsive operation information in the polygon drawing state, determine the target responsive operation corresponding to the third interaction operation, and based on the target responsive operation, perform vertex drawing on the sample to be annotated or trigger the annotation tool to perform a state transition.

[0097] Step 203, if it does not belong, maintain the polygon drawing state; output an operation error prompt message, or do not respond to the third interaction operation.

[0098] In the embodiment of the present application, in any functional state, it can be judged according to the responsive operation information of the functional state whether the received interaction operation belongs to the operation behavior of a certain responsive operation and whether the current annotated data meets the trigger constraint conditions of this responsive operation. If both are satisfied, then the interaction operation can be determined as a responsive operation. Further, if the responsive operation is an annotation function operation, then directly perform data annotation on the sample in the current state; and if the responsive operation is a state transition operation, then first perform a state transition and then execute the annotation process in the transferred state. For example, after selecting the polygon drawing tool in the idle state and clicking the left mouse button, this operation behavior can trigger the annotation tool to enter the polygon drawing state from the idle state and draw the first vertex at the click position of the mouse cursor.

[0099] Taking the polygon drawing state as an example, in this state, in response to the third interaction operation, query whether the operation behavior of the third interaction operation belongs to the operation behavior of a certain responsive operation and whether the currently drawn data meets the trigger constraint conditions of this responsive operation. If the third interaction operation belongs to a certain responsive operation, then when the responsive operation belongs to the annotation operation, sample data annotation is performed, and when the responsive operation belongs to the state transition operation, a state transition is performed. If the third interaction operation does not belong to any responsive operation, then the current polygon drawing state remains unchanged, and at the same time, an operation error prompt message can be output, or no response to the third interaction operation is given.

[0100] Taking several functions provided in the polygon drawing state as an example, the annotation method provided in this application is further described. In an embodiment of this application, optionally, the annotation function operations in the polygon drawing state include vertex drawing operations, and also include a first drawing point deletion operation and / or a second drawing point deletion operation; the operation behavior of the vertex drawing operation includes a first click behavior, and the trigger constraint condition of the vertex drawing operation includes that there is no drawn vertex at the click position; the vertex drawing operation is used to create a vertex at the click position; the operation behavior of the first drawing point deletion operation includes a second click behavior, and the trigger constraint condition of the first drawing point deletion operation includes that the click position contains a drawn vertex; the first drawing point deletion operation is used to delete the drawn vertex at the click position; the operation behavior of the second drawing point deletion operation includes a third click behavior, and the trigger constraint condition of the second drawing point deletion operation includes that there is at least one drawn vertex; the second drawing point deletion operation is used to delete the latest drawn vertex.

[0101] In this embodiment, in the polygon drawing state, the annotator can perform vertex drawing operations on the polygon and delete the drawn vertices. The responsive annotation function operations in this state include vertex drawing operations and at least one way of drawing point deletion operations, that is, a first drawing point deletion operation and / or a second drawing point deletion operation. Among them, when the annotator performs vertex drawing, the position of the vertex can be selected by clicking the left mouse button (first click behavior) on the sample to be annotated. If there is no already drawn vertex at the clicked position (the trigger constraint condition of the vertex drawing operation), then a drawing point can be created at the selected position as a vertex of the polygon. When the annotator wants to delete a drawn vertex, the right mouse button can be clicked (second click behavior, third click behavior). Further, if the annotator clicks at the drawn vertex, then the drawn vertex at the click position can be directly deleted. If the annotator clicks at a blank position and there are currently drawn vertices, then the latest drawn vertex is deleted.

[0102] Taking several state transition methods in the polygon drawing state as examples, the annotation method provided by this application will be further described. In an embodiment of this application, optionally, the functional state further includes a polygon drawing adsorption search state; the state transition operations in the polygon drawing state include a first end drawing operation and / or a second end drawing operation, and also include a polygon drawing adsorption search operation; the operation behavior of the first end drawing operation includes a fourth click behavior, and the trigger constraint condition of the first end drawing operation is that there is only one drawn vertex; the first end drawing operation is used to trigger the annotation tool to transfer to the idle state; the operation behavior of the second end drawing operation includes a first input behavior, and the trigger constraint condition of the second end drawing operation is empty; the second end drawing operation is used to trigger the annotation tool to transfer to the idle state; the operation behavior of the polygon drawing adsorption search operation includes a second input behavior, and the trigger constraint condition of the polygon drawing adsorption search operation is that there is a drawn polygon; the polygon drawing adsorption search operation is used to trigger the annotation tool to transfer to the polygon drawing adsorption search state.

[0103] In this embodiment, in the polygon drawing state, it can be transferred to the idle state or the polygon drawing point adsorption search state. Among them, when only one vertex is drawn, the annotator can trigger the annotation tool to transfer to the idle state by clicking the right mouse button (the fourth click behavior) (clicking the right mouse button when multiple vertices are drawn can delete the most recently drawn point). When multiple vertices are drawn, the annotator can press the "Q" key on the keyboard (the first input behavior) to end the drawing and return to the idle state. The annotator can also enter the polygon drawing adsorption search state by holding down the "Alt" key (the second input behavior) in the polygon drawing state.

[0104] Optionally, triggering the annotation tool to perform a state transition based on the target responsive operation includes at least one of the following:

[0105] 1. When the target responsive operation is the first end drawing operation, trigger the annotation tool to transfer to the idle state and delete the vertices drawn in the polygon drawing state;

[0106] 2. When the target responsive operation is the second end drawing operation, trigger the annotation tool to transfer to the idle state, generate a target polygon based on the vertices drawn in the polygon drawing state, and mark the part circled by the target polygon on the annotated sample;

[0107] 3. When the target responsive operation is the polygon drawing adsorption search operation, trigger the annotation tool to transfer to the polygon drawing adsorption search state, and query for the adsorbable vertex corresponding to the position of the mouse cursor among the vertices of the generated polygon.

[0108] In this embodiment, after the state transfer occurs in the polygon drawing state, corresponding processes can further be executed in the transferred state. For example, when exiting the polygon drawing state by clicking the right mouse button and returning to the idle state in the case of only one drawing point, the vertices drawn in the polygon drawing state are deleted; when returning to the idle state by pressing the "Q" key in the case of multiple vertices being drawn, the multiple vertices drawn in the polygon drawing state are used to generate a target polygon, and the area within the target polygon is intercepted from the annotated sample for annotation; after entering the polygon drawing adsorption search state, among the vertices of the already drawn polygon, query for the adsorbable vertices whose distance from the position of the mouse cursor is less than the threshold and highlight them. The annotator can select one of the highlighted vertices, so that the latest drawn vertex in the polygon drawing state is adsorbed to this selected vertex.

[0109] Further, as Figure 1 a specific implementation of the method, an embodiment of the present application provides an annotation device, which is applied to an annotation tool based on a state machine, as Figure 4 shown, the device includes:

[0110] An operation response module, configured to respond to a first interaction operation in the idle state of the annotation tool, and query whether the first interaction operation belongs to a preset idle state transfer operation, where the idle state transfer operation includes an operation for triggering the annotation tool to transfer from the idle state to a first transferable state corresponding to the idle state;

[0111] An operation execution module, configured to, if the first interaction operation belongs to the idle state transfer operation, determine a first state corresponding to the first interaction operation in the idle state, and transfer the state of the annotation tool to the first state, where the first transferable state includes the first state; and,

[0112] In response to a second interaction operation in the first state, perform data annotation on the annotated sample based on the operation type of the second interaction operation or trigger the annotation tool to perform a state transfer.

[0113] Optionally, the operation execution module is further configured to: after transferring the state of the annotation tool to the first state, perform data annotation on the annotated sample based on the first interaction operation in the first state.

[0114] Optionally, the operation execution module is further configured to:

[0115] Query whether the second interaction operation belongs to a preset first state transition operation, where the first state transition operation includes an operation for triggering the annotation tool to transition from the first state to a second transferable state corresponding to the first state;

[0116] If the second interaction operation belongs to the first state transition operation, determine the second state corresponding to the second interaction operation in the first state, transfer the state of the annotation tool to the second state, and execute the data annotation process in the second state based on the annotated data in the first state and the second interaction operation, where the second transferable state includes the second state;

[0117] If the second interaction operation does not belong to the first state transition operation, perform data annotation on the sample to be annotated based on the second interaction operation.

[0118] Optionally, the transferable states corresponding to different states in the annotation tool do not overlap, partially overlap, or completely overlap; different states include an idle state and multiple functional states;

[0119] The annotation tool is configured with response operation information for different states, and the response operation information includes an operation behavior and a trigger constraint condition corresponding to at least one response operation, and the trigger constraint condition is used to determine whether an interaction operation generated according to the operation behavior can be responded to;

[0120] The response operations include annotation function operations and / or state transition operations, the annotation function operations are used to trigger the execution of the data annotation process, and the state transition operations are used to trigger the execution of the state transition process.

[0121] Optionally, the functional state includes a polygon drawing state; the operation execution module is further configured to:

[0122] In response to a third interaction operation in the polygon drawing state, query whether the third interaction operation belongs to the response operation information in the polygon drawing state;

[0123] If it belongs, determine the target response operation corresponding to the third interaction operation based on the current drawing data and the response operation information in the polygon drawing state, and perform vertex drawing on the sample to be annotated or trigger the annotation tool to perform a state transition based on the target response operation;

[0124] If it does not belong, maintain the polygon drawing state; output an operation error prompt message, or do not respond to the third interaction operation.

[0125] Optionally, the annotation function operations in the polygon drawing state include vertex drawing operations, and also include a first drawing point deletion operation and / or a second drawing point deletion operation;

[0126] The operation behavior of the vertex drawing operation includes a first click behavior, and the trigger constraint condition of the vertex drawing operation includes that there is no drawn vertex at the click position; the vertex drawing operation is used to create a vertex at the click position;

[0127] The operation behavior of the first drawing point deletion operation includes a second click behavior, and the trigger constraint condition of the first drawing point deletion operation includes that the click position contains a drawn vertex; the first drawing point deletion operation is used to delete the drawn vertex at the click position;

[0128] The operation behavior of the second drawing point deletion operation includes a third click behavior, and the trigger constraint condition of the second drawing point deletion operation includes that there is at least one drawn vertex; the second drawing point deletion operation is used to delete the latest drawn vertex.

[0129] Optionally, the function state further includes a polygon drawing adsorption search state;

[0130] The state transition operations in the polygon drawing state include a first end drawing operation and / or a second end drawing operation, and also include a polygon drawing adsorption search operation;

[0131] The operation behavior of the first end drawing operation includes a fourth click behavior, and the trigger constraint condition of the first end drawing operation includes that there is only one drawn vertex; the first end drawing operation is used to trigger the annotation tool to transfer to the idle state;

[0132] The operation behavior of the second end drawing operation includes a first input behavior, and the trigger constraint condition of the second end drawing operation is empty; the second end drawing operation is used to trigger the annotation tool to transfer to the idle state;

[0133] The operation behavior of the polygon drawing adsorption search operation includes a second input behavior, and the trigger constraint condition of the polygon drawing adsorption search operation is that there is a drawn polygon; the polygon drawing adsorption search operation is used to trigger the annotation tool to transfer to the polygon drawing adsorption search state.

[0134] Optionally, the operation execution module is further configured to:

[0135] When the target responsive operation is the first end drawing operation, trigger the annotation tool to transfer to the idle state, and delete the vertices drawn in the polygon drawing state;

[0136] When the target responsive operation is the second end drawing operation, trigger the annotation tool to transfer to the idle state, generate a target polygon based on the vertices drawn in the polygon drawing state, and annotate the part of the annotated sample circled by the target polygon;

[0137] When the target responsive operation is the polygon drawing adsorption search operation, trigger the annotation tool to transfer to the polygon drawing adsorption search state, and query the adsorbable vertex corresponding to the position of the mouse cursor among the vertices of the generated polygon.

[0138] It should be noted that for other corresponding descriptions of each functional unit involved in the annotation device provided in the embodiments of the present application, reference can be made to the corresponding descriptions in the above method embodiments, and details are not described herein again.

[0139] The embodiments of the present application further provide a computer device, which may specifically be a personal computer, a server, a network device, etc. As Figure 4 shown, the computer device includes a bus, a processor, a memory, and a communication interface, and may further include an input / output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0140] Those skilled in the art can understand that Figure 4 the structure shown in

[0141] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0144] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the various embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the various embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the various embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0145] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0146] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A labeling method, characterized in that Applied to a state machine-based annotation tool, the method includes: In response to a first interaction operation in the idle state of the annotation tool, query whether the first interaction operation belongs to a preset idle state transition operation, where the idle state transition operation includes an operation for triggering the annotation tool to transition from the idle state to a first transferable state corresponding to the idle state; If the first interaction operation belongs to the idle state transition operation, determine a first state corresponding to the first interaction operation in the idle state, and transfer the state of the annotation tool to the first state, where the first transferable state includes the first state; In response to a second interaction operation in the first state, based on the operation type of the second interaction operation, perform data annotation on the sample to be annotated or trigger the annotation tool to perform a state transition, where the second transferable state corresponding to the first state is not limited to the idle state.

2. The method according to claim 1, characterized in that, After transferring the state of the annotation tool to the first state, the method further includes: In the first state, perform data annotation on the sample to be annotated based on the first interaction operation.

3. The method according to claim 1, characterized in that, The performing data annotation on the sample to be annotated or triggering the annotation tool to perform a state transition based on the operation type of the second interaction operation includes: Query whether the second interaction operation belongs to a preset first state transition operation, where the first state transition operation includes an operation for triggering the annotation tool to transition from the first state to a second transferable state corresponding to the first state; If the second interaction operation belongs to the first state transition operation, determine a second state corresponding to the second interaction operation in the first state, transfer the state of the annotation tool to the second state, and perform the data annotation process in the second state based on the annotated data in the first state and the second interaction operation, where the second transferable state includes the second state; If the second interaction operation does not belong to the first state transition operation, perform data annotation on the sample to be annotated based on the second interaction operation.

4. The method according to any one of claims 1 to 3, wherein The transferable states corresponding to different states in the annotation tool do not overlap, partially overlap, or completely overlap; different states include an idle state and multiple functional states; The annotation tool is configured with response operation information for different states, and the response operation information includes an operation behavior and a trigger constraint condition corresponding to at least one response operation, and the trigger constraint condition is used to determine whether an interaction operation generated according to the operation behavior can be responded to; The response operations include annotation function operations and / or state transition operations, the annotation function operations are used to trigger the execution of a data annotation process, and the state transition operations are used to trigger the execution of a state transition process.

5. The method according to claim 4, characterized in that, The functional state includes a polygon drawing state; the method further includes: In response to a third interaction operation in the polygon drawing state, query whether the third interaction operation belongs to the responsive operation information in the polygon drawing state; If it belongs, based on the current drawing data and the responsive operation information in the polygon drawing state, determine the target responsive operation corresponding to the third interaction operation, and based on the target responsive operation, perform vertex drawing on the annotated sample or trigger the annotation tool to perform a state transition; If it does not belong, maintain the polygon drawing state; output an operation error prompt message, or do not respond to the third interaction operation.

6. The method according to claim 5, wherein The annotation function operations in the polygon drawing state include vertex drawing operations, and also include a first drawing point deletion operation and / or a second drawing point deletion operation; The operation behavior of the vertex drawing operation includes a first click behavior, and the trigger constraint condition of the vertex drawing operation includes that there is no drawn vertex at the click position; The vertex drawing operation is used to create a vertex at the click position; The operation behavior of the first drawing point deletion operation includes a second click behavior, and the trigger constraint condition of the first drawing point deletion operation includes that the click position contains a drawn vertex; The first drawing point deletion operation is used to delete the drawn vertex at the click position; The operation behavior of the second drawing point deletion operation includes a third click behavior, and the trigger constraint condition of the second drawing point deletion operation includes that there is at least one drawn vertex; The second drawing point deletion operation is used to delete the latest drawn vertex.

7. The method according to claim 6, characterized in that, The functional state further includes a polygon drawing adsorption search state; The state transition operations in the polygon drawing state include a first end drawing operation and / or a second end drawing operation, and also include a polygon drawing adsorption search operation; The operation behavior of the first end drawing operation includes a fourth click behavior, and the trigger constraint condition of the first end drawing operation includes that there is only one drawn vertex; the first end drawing operation is used to trigger the annotation tool to transfer to the idle state; The operation behavior of the second end drawing operation includes a first input behavior, and the trigger constraint condition of the second end drawing operation is empty; the second end drawing operation is used to trigger the annotation tool to transfer to the idle state; The operation behavior of the polygon drawing adsorption search operation includes a second input behavior, and the trigger constraint condition of the polygon drawing adsorption search operation is that there is a drawn polygon; the polygon drawing adsorption search operation is used to trigger the annotation tool to transfer to the polygon drawing adsorption search state.

8. A marking device, characterized in that, Applied to an annotation tool based on a state machine, the device includes: An operation response module, configured to respond to a first interaction operation in the idle state of the annotation tool, and query whether the first interaction operation belongs to a preset idle state transition operation, where the idle state transition operation includes an operation for triggering the annotation tool to transfer from the idle state to a first transferable state corresponding to the idle state; An operation execution module, configured to, if the first interaction operation belongs to the idle state transition operation, determine a first state corresponding to the first interaction operation in the idle state, and transfer the state of the annotation tool to the first state, where the first transferable state includes the first state; and, In response to a second interaction operation in the first state, perform data annotation on the sample to be annotated or trigger the annotation tool to perform a state transition based on the operation type of the second interaction operation.

9. A storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device, comprising at least one processor, the processor being coupled to a memory, and a computer program running on the processor being stored on the memory, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images