Image labeling method and device, equipment and storage medium

Abstract

The invention discloses an image annotation method and device, equipment and a storage medium. The method comprises the steps of determining a to-be-processed image; in response to detection of a first labeling operation and a second labeling operation on a to-be-labeled object in the to-be-processed image, determining a first labeling point and a second labeling point respectively corresponding to the first labeling operation and the second labeling operation; detecting the movement position of the cursor at each moment; and under the condition that a third labeling operation on the to-be-labeled object is detected, displaying a labeling box for labeling the to-be-labeled object based on the first labeling point, the second labeling point, the moving position at each moment and the third labeling operation. The labeling frame determined by the labeling method is the rotating frame, and the size, the position and the rotating angle of the rotating frame are determined according to the to-be-labeled object, so that the labeling frame can better fit the to-be-labeled object, and the labeling frame does not need to be adjusted after labeling is completed; therefore, image annotation can be accurately and quickly performed, and the annotation efficiency and the annotation accuracy of the image are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image annotation method and apparatus, device, and storage medium. Background Art

[0002] Model training is an essential part of realizing intelligent driving, and the model training process usually requires a large amount of labeled data, so data labeling is a very important link.

[0003] As we all know, the more accurate the data annotation, the better the algorithm model training effect. To improve the training effect of the model, a large amount of labeled data is required. However, the existing data annotation methods are complex and inefficient when annotating rotating objects, resulting in increasingly high annotation costs. Summary of the invention

[0004] At present, the commonly used data annotation method is to first mark a rectangle according to the position of the object, then rotate the rectangle, and finally adjust the position and size of the rectangle to obtain the object's annotation box. This method has a complex annotation process and low efficiency.

[0005] In order to solve the above technical problems, the present disclosure provides an image annotation method and apparatus, a device, and a storage medium, which can solve the problems of complex process and low efficiency of existing data annotation methods.

[0006] According to a first aspect of the present disclosure, there is provided an image annotation method, comprising: determining an image to be processed; in response to detecting a first annotation operation and a second annotation operation on an object to be annotated in the image to be processed, determining a first annotation point corresponding to the first annotation operation and a second annotation point corresponding to the second annotation operation; detecting a moving position of a cursor on the image to be processed at each moment; and in the case of detecting a third annotation operation on the object to be annotated, displaying an annotation box for annotating the object to be annotated based on the first annotation point, the second annotation point, the moving position at each moment and the third annotation operation.

[0007] According to a second aspect of the present disclosure, an image annotation device is provided, comprising: an image determination module, configured to determine an image to be processed; an annotation point determination module, configured to determine, in response to detecting a first annotation operation and a second annotation operation on an object to be annotated in the image to be processed, a first annotation point corresponding to the first annotation operation and a second annotation point corresponding to the second annotation operation; a cursor position determination module, configured to detect a moving position of a cursor on the image to be processed at each moment; and an annotation display module, configured to display an annotation box for annotating the object to be annotated based on the first annotation point, the second annotation point, the moving position at each moment and the third annotation operation when a third annotation operation on the object to be annotated is detected.

[0008] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the image annotation method provided in the first aspect.

[0009] A fourth aspect of the present disclosure provides an electronic device, comprising: a processor; a memory for storing processor executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement the image annotation method provided in the first aspect above.

[0010] A fifth aspect of the present disclosure provides a computer program product. When instructions in the computer program product are executed by a processor, the image annotation method provided in the first aspect is executed.

[0011] Based on the image annotation method provided by the present disclosure, by first detecting the annotation points corresponding to the first annotation operation and the second annotation operation on the object to be annotated, then detecting the real-time moving position of the cursor on the image to be processed, and finally, when the third annotation operation is detected, the annotation frame of the object to be annotated is determined and displayed based on the annotation points, the real-time moving position of the cursor and the third annotation operation; since the annotation frame determined by the steps in the image annotation method provided by the present disclosure is a rotating frame, and the size, position and rotation angle of the rotating frame are determined according to the object to be annotated, the annotation frame can be more closely fitted to the object to be annotated, and there is no need to adjust the annotation frame after the annotation is completed; in this way, the image annotation can be performed accurately and quickly, thereby improving the image annotation efficiency and annotation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of a scene to which the image annotation method provided by an exemplary embodiment of the present disclosure is applicable.

[0013] Figure 2A It is a flowchart of an image annotation method provided by an exemplary embodiment of the present disclosure.

[0014] Figure 2B It is a schematic diagram of a drawing interface of an image annotation method provided by an exemplary embodiment of the present disclosure.

[0015] Figure 2C It is a schematic diagram of a drawing interface of an image annotation method provided by another exemplary embodiment of the present disclosure.

[0016] Figure 3A It is a flowchart of an image annotation method provided by another exemplary embodiment of the present disclosure.

[0017] Figure 3B It is a schematic diagram of a drawing interface of an image annotation method provided by yet another exemplary embodiment of the present disclosure.

[0018] Figure 3C It is a schematic diagram of a drawing interface of an image annotation method provided by yet another exemplary embodiment of the present disclosure.

[0019] Figure 3D It is a schematic diagram of a drawing interface of an image annotation method provided by yet another exemplary embodiment of the present disclosure.

[0020] Figure 4 It is a flowchart of an image annotation method provided by yet another exemplary embodiment of the present disclosure.

[0021] Figure 5A It is a flowchart of an image annotation method provided by yet another exemplary embodiment of the present disclosure.

[0022] Figure 5B It is a schematic diagram of a drawing interface of an image annotation method provided by yet another exemplary embodiment of the present disclosure.

[0023] Figure 6 It is a flowchart of an image annotation method provided by yet another exemplary embodiment of the present disclosure.

[0024] Figure 7 It is a flowchart of an image annotation method provided by yet another exemplary embodiment of the present disclosure.

[0025] Figure 8 A schematic diagram of the structure of an image annotation device provided by an exemplary embodiment of the present disclosure.

[0026] Fig. 9 A schematic diagram of the structure of an image annotation device provided by another exemplary embodiment of the present disclosure.

[0027] Fig.10 The present invention is a schematic structural diagram of an image annotation device provided by yet another exemplary embodiment of the present invention.

[0028] Fig.11 It is a structural schematic diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0030] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0031] Application Overview

[0032] The image annotation method provided by the embodiments of the present disclosure can be applied to, for example, the front end of a vehicle's environmental perception scenario, and any other feasible scenario.

[0033] like Figure 1 As shown, the vehicle 10 is provided with at least a plurality of image sensors 11 ( Figure 1 Only part of the image sensor 11 is shown in the figure. The image sensor 11 is responsible for collecting environmental information around the vehicle 10 and outputting the collected multiple image data. Furthermore, the processing unit in the vehicle 10 can perform environmental perception based on the collected multiple image data, such as identifying target objects such as lane lines, traffic signs, pedestrians, other vehicles, etc., so as to realize path planning and decision execution. This process usually needs to be implemented using an algorithm model such as a neural network model or a deep learning model, and in order to obtain accurate model output results, the model usually needs to be trained. Model training requires the use of label information as the true value, so image annotation is an indispensable part of model training. The more accurate the image annotation, the better the model training effect. Furthermore, the image annotation method provided by the embodiment of the present disclosure can process the sample image output by the image sensor 11 to obtain the annotation data of the sample image, and then obtain the label information of the sample image based on the annotation data of the sample image. It should be noted that, under normal circumstances, 7 image sensors 11 are provided on the vehicle 10, for example, an image sensor corresponding to a front-view wide-angle camera, an image sensor corresponding to a front-view narrow-angle camera, an image sensor corresponding to a side-view camera, etc. The images captured by the seven image sensors 11 can all be used as input data for the image annotation method provided in the embodiment of the present disclosure.

[0034] Since objects in images may have various rotation angles, existing image annotation methods need to first mark a rectangle according to the object position, then rotate the rectangle, and then adjust the position and size of the rectangle to finally obtain the object's annotation box. The entire annotation process is complicated and inefficient.

[0035] In order to solve the above problems, an embodiment of the present disclosure provides an image annotation method, which first detects the annotation points corresponding to the first annotation operation and the second annotation operation on the object to be annotated, then detects the real-time moving position of the cursor on the image to be processed, and finally, when the third annotation operation is detected, determines and displays the annotation box of the object to be annotated based on the annotation points, the real-time moving position of the cursor and the third annotation operation; since the annotation box determined by the steps in the image annotation method provided by the present disclosure is a rotating box, and the size, position and rotation angle of the rotating box are determined according to the object to be annotated, the annotation box can be more closely fitted to the object to be annotated, and there is no need to adjust the annotation box after the annotation is completed; in this way, the image can be annotated accurately and quickly, thereby improving the image annotation efficiency and annotation accuracy.

[0036] Exemplary Methods

[0037] Figure 2A The following is a flow chart of an image annotation method provided by an embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as Figure 2A As shown, the method includes the following steps S201-S204.

[0038] Step S201: Determine an image to be processed.

[0039] For example, Figure 1 As shown, the image captured by the image sensor 11 can be first obtained and used as the image to be processed. It should be noted that the format of the image to be processed is not limited in the embodiment of the present disclosure. For example, the image to be processed can be in RGB format or RAW format.

[0040] In some examples, the image to be processed includes at least one object to be annotated, and the object to be annotated may be any object in the image to be processed. For example, the object to be annotated may be various ground elements, such as lane lines, pedestrian crossings, lane markings, speed limit signs, arrows, etc. For another example, the object to be annotated may also be pedestrians, vehicles, traffic lights, buildings, etc.

[0041] Step S202: in response to detecting a first labeling operation and a second labeling operation on the object to be labeled in the image to be processed, determining a first labeling point corresponding to the first labeling operation and a second labeling point corresponding to the second labeling operation.

[0042] Exemplarily, the image to be processed may be displayed in a canvas, so that the operator can perform a labeling operation on the object to be labeled in the canvas. The operation types of the first labeling operation and the second labeling operation on the object to be labeled in the image to be processed include, but are not limited to, a click operation, a drag operation, a pointing operation, etc. For example, the first labeling operation and the second labeling operation on the object to be labeled may be a click operation on the object to be labeled, or a click operation on the edge area of ​​the object to be labeled. For another example, the first labeling operation on the object to be labeled may be a click operation, and the second labeling operation may be a drag operation after the click operation.

[0043] Correspondingly, when the processor in the electronic device detects the first annotation operation and the second annotation operation on the object to be annotated in the processed image, it determines the first annotation point corresponding to the first annotation operation and the second annotation point corresponding to the second annotation operation. For example, when the processor detects the first click operation and the second click operation on the edge area of ​​the object to be annotated, it determines the first click position corresponding to the first click operation and the second click position corresponding to the second click operation; wherein the first click position is the first annotation point, and the second click position is the second annotation point. For another example, when the processor detects the click operation on the object to be annotated, it determines the click position corresponding to the click operation. When a drag operation is detected after the click operation, the end point position of the drag operation is determined. wherein the click position is the first annotation point, and the end position is the second annotation point.

[0044] It should be noted that there is a sequence between the first annotation operation and the second annotation operation, and the first annotation operation is placed before the second annotation operation. In addition, the first annotation operation and the second annotation operation in the embodiment of the present disclosure can be operations on the object to be annotated itself, or operations on the relevant area of ​​the object to be annotated; wherein the relevant area of ​​the object to be annotated includes but is not limited to the edge area of ​​the object to be annotated, etc.

[0045] In some examples, such as Figure 2B As shown, the first annotation point corresponding to the first annotation operation on the object to be annotated 22 in the processed image 21 is point a where the cross cursor is located. Figure 2B On the basis of Figure 2C As shown, the second annotation point corresponding to the second annotation operation on the object to be annotated 22 in the processed image 21 is point b where the cross cursor is located.

[0046] Step S203: Detect the moving position of the cursor on the image to be processed at each moment.

[0047] For example, after the operator performs the first and second annotation operations on the object to be annotated, the operator may move the cursor on the image to be processed. Further, after detecting the first and second annotation operations, the processor of the electronic device may monitor the cursor in real time, thereby detecting the moving position of the cursor on the image to be processed at each moment. The moving position of the cursor on the image to be processed at each moment refers to the position information of the cursor at each moment during the movement process.

[0048] It should be noted that the various moments in the embodiment of the present disclosure refer to the moments between the moment when the second marking operation ends and the moment when the third marking operation in the following step S204 begins.

[0049] Step S204: when a third annotation operation on the object to be annotated is detected, a annotation box for annotating the object to be annotated is displayed based on the first annotation point, the second annotation point, the moving positions at each moment, and the third annotation operation.

[0050] Exemplarily, after the operator performs the first and second annotation operations on the object to be annotated, the operator may move the cursor on the image to be processed. Further, after the cursor is moved to a suitable position, the operator may perform a third annotation operation on the object to be annotated. Thus, when the third annotation operation on the object to be annotated is detected, the processor may determine and display an annotation box for annotating the object to be annotated based on the first annotation point, the second annotation point, the movement position of the cursor at each time, and the third annotation operation.

[0051] The image annotation method provided by the embodiment of the present disclosure first detects the annotation points corresponding to the first annotation operation and the second annotation operation on the object to be annotated, then detects the real-time moving position of the cursor on the image to be processed, and finally, when the third annotation operation is detected, determines and displays the annotation frame of the object to be annotated based on the annotation points, the real-time moving position of the cursor and the third annotation operation; since the annotation frame determined by the steps in the image annotation method provided by the present disclosure is a rotating frame, and the size, position and rotation angle of the rotating frame are determined according to the object to be annotated, the annotation frame can be more closely fitted to the object to be annotated, and there is no need to adjust the annotation frame after the annotation is completed; in this way, the image annotation can be performed accurately and quickly, thereby improving the image annotation efficiency and annotation accuracy.

[0052] like Figure 3A As shown in the above Figure 2A Based on the illustrated embodiment, step S204 may include the following steps S2041 to S2043.

[0053] Step S2041: Based on the first marking point, the second marking point and the moving position at each moment, a marking box for marking the object to be marked at each moment is displayed.

[0054] Exemplarily, based on the first marking point, the second marking point and the moving position at each moment, a marking box for marking the object to be marked at each moment may be determined and displayed in real time.

[0055] In some examples, such as Figure 3B As shown, in the image to be processed 21, the cross cursor moves to position 23 at the first moment, and at this time, based on the first marking point a, the second marking point b and the position 23 after the cross cursor moves at the first moment, a marking frame 24 for marking the object to be marked 22 at the first moment is displayed. Figure 3B On the basis of Figure 3CAs shown, in the image to be processed 21, the cross cursor moves to position 25 at the second moment, and based on the first marking point a, the second marking point b and the position 25 after the cross cursor moves at the second moment, a marking frame 26 for marking the object to be marked 22 at the second moment is displayed. Figure 3C On the basis of Figure 3D As shown, in the image to be processed 21, the cross cursor moves to position 27 at the third moment. At this time, based on the first marking point a, the second marking point b and the position 27 after the cross cursor moves at the third moment, a marking box 28 for marking the object to be marked 22 at the third moment is displayed.

[0056] Step S2042: in response to detecting a third labeling operation on the object to be labeled, determining a target labeling frame from labeling frames used to label the object to be labeled at each moment based on the third labeling operation.

[0057] Exemplarily, the processor of the electronic device will display the annotation box used to annotate the object to be annotated at each moment based on the first annotation point, the second annotation point and the moving position of the cursor at each moment. Furthermore, the operator can refer to the annotation box used to annotate the object to be annotated at each moment, and perform a third annotation operation on the object to be annotated. For example, the third annotation operation can be a click operation, and of course it can also be other types of operations. The embodiment of the present disclosure does not limit this. Furthermore, when the processor detects the third annotation operation on the object to be annotated, it can determine the target annotation box from the annotation boxes used to annotate the object to be annotated at each moment based on the third annotation operation. For example, the operator judges the annotation boxes used to annotate the object to be annotated at each moment, and the operator can click when there is a suitable annotation box. At this time, in response to detecting the click operation, the processor can determine the annotation box displayed at the current moment as the target annotation box according to the click operation.

[0058] Step S2043: display the target marking frame.

[0059] For example, after the target annotation frame is determined, the target annotation frame can be fixedly displayed on the image to be processed. The target annotation frame is fixedly displayed on the image to be processed, which means that the target annotation frame is always displayed on the image to be processed and does not change with the movement of the cursor. Figure 3D As shown, when the annotation box 28 used to annotate the object 22 to be annotated at the third moment is displayed on the image to be processed, if a click operation is detected, the annotation box corresponding to the triggering moment of the click operation (i.e., the annotation box 28) is displayed as the target annotation box on the image to be processed.

[0060] In some examples, the above steps may be performed on each object to be annotated in the image to be processed, and when the target annotation boxes of each object to be annotated are displayed complete, the annotation of the image to be processed is completed.

[0061] The image annotation method provided by the embodiment of the present disclosure first displays the annotation box used to annotate the object to be annotated at each moment based on the first annotation point, the second annotation point and the moving position at each moment; and then determines the target annotation box from the annotation box used to annotate the object to be annotated at each moment based on the third annotation operation; in this way, the image annotation can be performed quickly at one time, and because the target annotation box is determined from the annotation box used to annotate the object to be annotated at each moment, the annotation accuracy can also be guaranteed.

[0062] like Figure 4 As shown in the above Figure 3A Based on the illustrated embodiment, step S2041 may include the following steps S21-S22.

[0063] Step S21: determining a first target point corresponding to each moment in the image to be processed based on the first marked point, the second marked point and the moving position at each moment.

[0064] Exemplarily, the moving position at each moment includes the position of the cursor at the first moment, the moving position of the cursor at the second moment...the moving position of the cursor at the Nth moment. The processor can determine the first target point corresponding to the first moment in the image to be processed based on the first annotation point, the second annotation point, and the position of the cursor at the first moment. The processor can also determine the first target point corresponding to the second moment in the image to be processed based on the first annotation point, the second annotation point, and the position of the cursor at the second moment. Similarly, the processor can determine the first target point corresponding to the Nth moment in the image to be processed based on the first annotation point, the second annotation point, and the position of the cursor at the Nth moment.

[0065] Step S22: Based on the first marking point, the second marking point and the first target point corresponding to each moment, a marking box for marking the object to be marked at each moment is displayed.

[0066] Exemplarily, the first target point corresponding to each moment includes the first target point corresponding to the first moment, the first target point corresponding to the second moment...the first target point corresponding to the Nth moment. The processor can determine and instantly display the annotation box used to annotate the object to be annotated at the first moment based on the first annotation point, the second annotation point, and the first target point corresponding to the first moment. The processor can also determine and instantly display the annotation box used to annotate the object to be annotated at the second moment based on the first annotation point, the second annotation point, and the first target point corresponding to the second moment. By analogy, the processor can determine and instantly display the annotation box used to annotate the object to be annotated at the Nth moment based on the first annotation point, the second annotation point, and the first target point corresponding to the Nth moment. In this way, the annotation box used to annotate the object to be annotated at each moment can be instantly displayed during the movement of the cursor for reference by the operator.

[0067] In some examples, since the annotation box used to annotate the object to be annotated at each moment is determined based on three points, the annotation box can be a rectangular box with a certain rotation angle. In the data annotation process, since the positions of most objects in the image have a certain rotation angle, using a rectangular box with a rotation angle to annotate the object can better fit the object, thereby improving the annotation accuracy.

[0068] The image annotation method provided by the disclosed embodiment first determines the first target point corresponding to each moment in the image to be processed based on the first annotation point, the second annotation point and the moving position at each moment; then displays the annotation box for annotating the object to be annotated at each moment based on the first annotation point, the second annotation point and the first target point corresponding to each moment; thus, the annotation box with a rotation angle can be directly determined based on the three points, and the method is simple to operate and highly efficient. Moreover, since the annotation box determined by the method has a rotation angle, it can fit the object to be annotated more closely, thereby achieving accurate annotation of the object to be annotated.

[0069] like Figure 5A As shown in the above Figure 4 Based on the illustrated embodiment, step S21 may include the following steps S21a to S21c.

[0070] Step S21a: determine a first line segment based on the first marked point and the second marked point.

[0071] For example, using the theory that two points form a line, the processor may determine the first line segment based on the first marked point and the second marked point.

[0072] In some examples, such as Figure 5B As shown, the first marked point is point a, and the second marked point is point b, and then the first line segment ab can be determined based on the first marked point a and the second marked point b.

[0073] Step S21b: determining a second straight line based on the first line segment, the second marked point and a preset positional relationship.

[0074] For example, the annotation box in the embodiment of the present disclosure adopts a rectangular box, so the preset position relationship is that the product of the slope of the first line segment and the slope of the second line is -1, and then the processor can determine the slope of the second line based on the slope of the first line segment, and then determine the second line according to the position of the second annotation point and the slope of the second line. Figure 5B As shown, the second straight line 51 can be determined based on the first line segment ab, the second marked point b and the preset position relationship.

[0075] Step S21c: based on the first line segment, the second straight line and the moving position at each moment, determine the first target point corresponding to each moment in the image to be processed.

[0076] Exemplarily, the first target point corresponding to the first moment can be determined in the image to be processed based on the first line segment, the second straight line and the moving position at the first moment. The first target point corresponding to the first moment can also be determined in the image to be processed based on the first line segment, the second straight line and the moving position at the second moment. Similarly, the first target point corresponding to the Nth moment can be determined in the image to be processed based on the first line segment, the second straight line and the moving position at the Nth moment. For example, assuming that the line segment formed by the moving position of the cursor at the Nth moment and the first target point corresponding to the Nth moment is the third line segment, the first line segment and the third line segment are parallel and have the same slope. Then, the straight line where the third line segment is located can be determined based on the slope of the first line segment and the moving position of the cursor at the Nth moment. In this way, the intersection of the second straight line and the straight line where the third line segment is located is the first target point corresponding to the Nth moment.

[0077] In some examples, such as Figure 5B As shown, the moving position of the cursor at the Nth moment is point c, and the first target point corresponding to the Nth moment is point d. Then the processor can determine the first target point d corresponding to the Nth moment in the image to be processed based on the first line segment ab, the second straight line 51 and the moving position of the cursor at the Nth moment. Further, the processor can determine the rectangular annotation frame of the object to be annotated based on points a, b and d, so as to implement the annotation of the object using the three-point annotation method.

[0078] The image annotation method provided by the embodiment of the present disclosure first determines the first line segment based on the first annotation point and the second annotation point; then determines the second straight line based on the first line segment, the second annotation point and a preset positional relationship; finally, determines the first target point corresponding to each moment in the image to be processed based on the first line segment, the second straight line and the moving position at each moment; in this way, the first target point for determining the annotation box can be calculated according to the geometric correlation method, and the calculation method is simple and efficient.

[0079] In some embodiments, based on the first line segment, the second straight line and the moving position at each moment, the first target point corresponding to each moment is determined in the image to be processed, including: determining the slope of the first line segment; based on the slope of the first line segment and the moving position at each moment, determining the third straight line corresponding to each moment; determining the intersection of the second straight line and the third straight line corresponding to each moment as the first target point corresponding to each moment.

[0080] For example, the slope is a quantity indicating the degree of inclination of a straight line with respect to the horizontal axis. Therefore, the slope of the first line segment refers to the degree of inclination of the straight line where the first line segment is located with respect to the horizontal axis. Furthermore, based on the slope of the first line segment and the moving position at each moment, the third straight line corresponding to each moment can be determined; after the third straight line corresponding to each moment is determined, the intersection of the second straight line and the third straight line corresponding to each moment can be determined as the first target point corresponding to each moment.

[0081] In some examples, such as Figure 5B As shown, the processor can determine the third straight line 52 corresponding to the Nth moment based on the slope of the first line segment ab and the moving position c of the cursor at the Nth moment; and determine the intersection point d between the second straight line 51 and the third straight line 52 corresponding to the Nth moment as the first target point corresponding to the Nth moment.

[0082] like Figure 6 As shown in the above Figure 4 Based on the illustrated embodiment, step S22 may include the following steps S22a to S22c.

[0083] Step S22a: determining the second target point corresponding to each moment based on the first marked point, the second marked point and the first target point corresponding to each moment.

[0084] Exemplarily, the annotation box in the embodiment of the present disclosure is a rectangular box, and the second target point corresponding to each moment can be determined based on the first annotation point, the second annotation point and the first target point corresponding to each moment.

[0085] Step S22b: determining a labeling box for labeling the object to be labeled at each moment based on the first labeling point, the second labeling point, the first target point corresponding to each moment, and the second target point corresponding to each moment.

[0086] Exemplarily, the specific position of the rectangular annotation box used to annotate the object to be annotated at each moment can be determined based on the first annotation point, the second annotation point, the first target point corresponding to each moment, and the second target point corresponding to each moment, wherein the first annotation point, the second annotation point, the first target point corresponding to each moment, and the second target point corresponding to each moment are the four vertices of the rectangular annotation box.

[0087] Step S22c, display the marking box at each moment.

[0088] Exemplarily, a labeling box used to label the object to be labeled at each moment may be displayed in real time while the cursor moves.

[0089] The image annotation method provided by the embodiment of the present disclosure first determines the second target point corresponding to each moment based on the first annotation point, the second annotation point and the first target point corresponding to each moment; then determines the annotation box used to annotate the object to be annotated at each moment based on the first annotation point, the second annotation point, the first target point corresponding to each moment and the second target point corresponding to each moment; in this way, the rectangular annotation box used to annotate the object to be annotated can be calculated according to the geometric correlation method, and the calculation method is simple and efficient.

[0090] like Figure 7 As shown in the above Figure 2A Based on the illustrated embodiment, steps S205 to S207 are also included.

[0091] Step S205: determine a plurality of vertices of the annotation box and an image coordinate system corresponding to the image to be processed.

[0092] Exemplarily, in the embodiment of the present disclosure, the annotation box is a rectangular annotation box, so the multiple vertices of the annotation box are the four vertices of the rectangular annotation box. The image coordinate system corresponding to the image to be processed refers to a physical image coordinate system with the center of the image to be processed as the origin of the coordinate system.

[0093] Step S206: Determine the position information of the plurality of vertices in the image coordinate system and the rotation angle of the annotation box in the image coordinate system.

[0094] Exemplarily, the position information of the four vertices of the rectangular annotation box in the image coordinate system and the rotation angle of the rectangular annotation box in the image coordinate system may be determined. For example, the position information of the four vertices in the image coordinate system may be expressed in the form of coordinates (X, Y). If the rectangular annotation box is determined clockwise, the rotation angle of the rectangular annotation box in the image coordinate system may be positive; if the rectangular annotation box is determined counterclockwise, the rotation angle of the rectangular annotation box in the image coordinate system may be negative.

[0095] Step S207: Determine label information of the image to be processed based on the rotation angle and multiple pieces of position information.

[0096] Exemplarily, the label information of the image to be processed is the data that the model algorithm needs to process, so it is necessary to calculate the specific data of the annotation box used to annotate the object to be labeled (i.e., the rotation angle and multiple position information) at the code level, and then determine the label information of the image to be processed based on the specific data of the annotation box.

[0097] The image annotation method provided by the embodiment of the present disclosure first determines the position information of multiple vertices in the image coordinate system and the rotation angle of the annotation box in the image coordinate system; then determines the position information of multiple vertices in the image coordinate system and the rotation angle of the annotation box in the image coordinate system; finally, based on the rotation angle and multiple position information, determines the label information of the image to be processed; in this way, the label information of the image to be processed can be quickly and accurately determined based on the displayed annotation box, thereby improving the training effect of the algorithm model.

[0098] Exemplary Devices

[0099] Figure 8 An image annotation device provided by an embodiment of the present disclosure is as follows: Figure 8 As shown, the image annotation device 800 includes an image determination module 801 , an annotation point determination module 802 , a cursor position determination module 803 and an annotation display module 804 .

[0100] The image determination module 801 is used to determine the image to be processed.

[0101] The annotation point determination module 802 is used to determine a first annotation point corresponding to the first annotation operation and a second annotation point corresponding to the second annotation operation in response to detecting a first annotation operation and a second annotation operation on the object to be annotated in the processed image.

[0102] The cursor position determination module 803 is used to detect the moving position of the cursor on the image to be processed at each moment.

[0103] The annotation display module 804 is used to display an annotation frame for annotating the object to be annotated based on the first annotation point, the second annotation point, the moving position at each moment and the third annotation operation when a third annotation operation on the object to be annotated is detected.

[0104] In some embodiments, Fig. 9 As shown, the annotation display module 804 includes a real-time annotation display unit 8041 , an annotation frame determination unit 8042 and an annotation display unit 8043 .

[0105] The real-time annotation display unit 8041 is used to display the annotation box used to annotate the object to be annotated at each moment based on the first annotation point, the second annotation point and the moving position at each moment;

[0106] A labeling box determining unit 8042 is configured to, in response to detecting a third labeling operation on the object to be labeled, determine a target labeling box from labeling boxes used to label the object to be labeled at each moment based on the third labeling operation;

[0107] The annotation display unit 8043 is used to display the target annotation frame.

[0108] In some embodiments, the real-time annotation display unit 8041 is used to determine the first target point corresponding to each moment in the image to be processed based on the first annotation point, the second annotation point and the moving position at each moment; based on the first annotation point, the second annotation point and the first target point corresponding to each moment, display the annotation box for annotating the object to be annotated at each moment.

[0109] In some embodiments, the real-time annotation display unit 8041 is specifically used to determine a first line segment based on a first annotation point and a second annotation point; determine a second straight line based on the first line segment, the second annotation point and a preset positional relationship; determine a first target point corresponding to each moment in the image to be processed based on the first line segment, the second straight line and the moving position at each moment; and display an annotation box for annotating the object to be annotated at each moment based on the first annotation point, the second annotation point and the first target point corresponding to each moment.

[0110] In some embodiments, the real-time annotation display unit 8041 is specifically used to determine a first line segment based on a first annotation point and a second annotation point; determine a second straight line based on the first line segment, the second annotation point and a preset positional relationship; determine the slope of the first line segment; determine a third straight line corresponding to each moment based on the slope of the first line segment and the moving position at each moment; determine the intersection of the second straight line and the third straight line corresponding to each moment as the first target point corresponding to each moment; and display a annotation box for annotating the object to be annotated at each moment based on the first annotation point, the second annotation point and the first target point corresponding to each moment.

[0111] In some embodiments, the real-time annotation display unit 8041 is specifically used to determine the first target point corresponding to each moment in the image to be processed based on the first annotation point, the second annotation point and the moving position at each moment; determine the second target point corresponding to each moment based on the first annotation point, the second annotation point and the first target point corresponding to each moment; determine the annotation box used to annotate the object to be annotated at each moment based on the first annotation point, the second annotation point, the first target point corresponding to each moment and the second target point corresponding to each moment; and display the annotation box at each moment.

[0112] In some embodiments, Fig.10 As shown, the image tagging apparatus 800 further includes a data determination module 805 and a label determination module 806 .

[0113] The data determination module 805 is used to determine a plurality of vertices of the annotation box and an image coordinate system corresponding to the image to be processed;

[0114] The data determination module 805 is further used to determine the position information of the plurality of vertices in the image coordinate system and the rotation angle of the annotation box in the image coordinate system;

[0115] The label determination module 806 is used to determine the label information of the image to be processed based on the rotation angle and multiple position information.

[0116] The beneficial technical effects corresponding to the exemplary embodiment of the above-mentioned image annotation device 800 can refer to the corresponding beneficial technical effects of the above-mentioned exemplary method part, which will not be repeated here.

[0117] Exemplary Electronic Devices

[0118] Fig.11 A structural diagram of an electronic device 110 provided in an embodiment of the present disclosure includes at least one processor 111 and a memory 112 .

[0119] The processor 111 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 110 to perform desired functions.

[0120] The memory 112 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 111 may execute one or more computer program instructions to implement the image annotation method and / or other desired functions of the various embodiments of the present disclosure described above.

[0121] In one example, the electronic device 110 may further include: an input device 113 and an output device 114 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0122] The input device 113 may also include, for example, a keyboard, a mouse, etc.

[0123] The output device 114 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and its connected remote output devices, etc.

[0124] Of course, to simplify, Fig.11 Only some of the components related to the present disclosure in the electronic device 110 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 110 may also include any other appropriate components.

[0125] Exemplary computer program products and computer-readable storage media

[0126] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the image annotation method of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0127] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0128] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps in the image annotation method of various embodiments of the present disclosure described in the above “Exemplary Method” section.

[0129] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or device including electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0130] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and cannot be considered as necessary for each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to being implemented by adopting the above specific details.

[0131] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. An image annotation method, comprising: Determine the image to be processed; In response to detecting a first labeling operation and a second labeling operation on the object to be labeled in the image to be processed, determining a first labeling point corresponding to the first labeling operation and a second labeling point corresponding to the second labeling operation; Detecting the moving position of the cursor on the image to be processed at each moment; When a third labeling operation on the object to be labeled is detected, a labeling frame for labeling the object to be labeled is displayed based on the first labeling point, the second labeling point, the moving positions at each time, and the third labeling operation.

2. The method according to claim 1, wherein: The method of, when a third annotation operation on the object to be annotated is detected, displaying a annotation frame for annotating the object to be annotated based on the first annotation point, the second annotation point, the moving position at each moment, and the third annotation operation, comprises: Based on the first marking point, the second marking point and the moving position at each moment, displaying a marking frame for marking the object to be marked at each moment; In response to detecting a third labeling operation on the object to be labeled, determining a target labeling frame from the labeling frames used to label the object to be labeled at each moment based on the third labeling operation; The target annotation box is displayed.

3. The method according to claim 2, wherein: The step of displaying a marking frame for marking the object to be marked at each moment based on the first marking point, the second marking point and the moving position at each moment includes: Determining, in the image to be processed, a first target point corresponding to each moment based on the first marked point, the second marked point, and the moving position at each moment; Based on the first marking point, the second marking point and the first target point corresponding to each moment, a marking box for marking the object to be marked at each moment is displayed.

4. The method according to claim 3, wherein: The determining, based on the first marked point, the second marked point and the moving position at each moment, a first target point corresponding to each moment in the image to be processed includes: Determine a first line segment based on the first marked point and the second marked point; Determine a second straight line based on the first line segment, the second marked point and a preset positional relationship; Based on the first line segment, the second straight line and the moving position at each moment, a first target point corresponding to each moment is determined in the image to be processed.

5. The method according to claim 4, wherein: The determining, based on the first line segment, the second straight line and the moving position at each moment, a first target point corresponding to each moment in the image to be processed includes: determining a slope of the first line segment; Determine a third straight line corresponding to each moment based on the slope of the first line segment and the moving position at each moment; The intersection point of the second straight line and the third straight line corresponding to each moment is determined as the first target point corresponding to each moment.

6. The method according to claim 3, wherein: The step of displaying a labeling box for labeling the object to be labeled at each moment based on the first labeling point, the second labeling point, and the first target point corresponding to each moment includes: Determine a second target point corresponding to each moment based on the first marked point, the second marked point, and the first target point corresponding to each moment; Determine, based on the first marking point, the second marking point, the first target point corresponding to each moment, and the second target point corresponding to each moment, a marking box for marking the object to be marked at each moment; The annotation frame at each time is displayed.

7. The method according to any one of claims 1 to 6, further comprising: Determine a plurality of vertices of the annotation box and an image coordinate system corresponding to the image to be processed; Determine position information of the plurality of vertices in the image coordinate system and a rotation angle of the annotation frame in the image coordinate system; Based on the rotation angle and the plurality of position information, label information of the image to be processed is determined.

8. An image annotation device, comprising: An image determination module, used for determining an image to be processed; a marking point determination module, configured to, in response to detecting a first marking operation and a second marking operation on the object to be marked in the image to be processed, determine a first marking point corresponding to the first marking operation and a second marking point corresponding to the second marking operation; A cursor position determination module, used for detecting the moving position of the cursor on the image to be processed at each moment; The annotation display module is used to display an annotation frame for annotating the object to be annotated based on the first annotation point, the second annotation point, the moving position at each moment and the third annotation operation when a third annotation operation on the object to be annotated is detected.

9. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the image annotation method according to any one of claims 1 to 7.

10. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the image annotation method described in any one of claims 1 to 7.

Images