A marking verification method, device and electronic equipment

Abstract

A method, device and electronic device for annotation verification, the method comprising: obtaining all annotation points in an initial annotation image sent by a user terminal, and screening out annotation points in a preset annotation box from all annotation points, taking annotation points not in the preset annotation box as initial error annotation points, screening out annotation points in the preset annotation box that do not conform to a first preset rule, and taking annotation points that do not conform to the first preset rule as first error annotation points, connecting the annotation points extracted to conform to the first preset rule in pairs, screening out lines that do not conform to a second preset rule from all lines, and taking annotation points corresponding to lines that do not conform to the second preset rule as second error annotation points, merging the initial error annotation points, the first error annotation points and the second error annotation points, and obtaining target error annotation points corresponding to the initial annotation image. Through the above method, the annotation points are verified according to different dimensions, and the error annotation points are accurately screened out.

Description

Technical Field

[0001] The present application relates to the technical field of image annotation processing, and in particular to an annotation verification method, device and electronic equipment. Background Art

[0002] With the development of machine learning, in order to obtain more accurate data using machine learning, it is usually necessary to label the data used for machine learning. The accuracy of data labeling will greatly affect the efficiency of machine learning and the accuracy of the training model obtained using machine learning.

[0003] In order to confirm whether the data is labeled accurately, a key point labeling review method is currently used. This method is based on multiple labelers labeling the same labeled object separately. The labelers label the labeled object in the following way: Figure 1 As shown, Figure 1 A coordinate system is established with the intersection of the left edge and the bottom edge of the image of the labeled object as the origin. The solid circle indicates the position where the labeler labels the labeled object. The position is used as the labeling point. An labeled object has multiple labeling points. After each labeler labels the labeled object, the coordinate parameter sets of each labeler after labeling the labeled object are obtained, and then multiple coordinate parameter sets corresponding to the labeled object are obtained.

[0004] After obtaining multiple coordinate parameter sets of the annotated object, the annotation point corresponding to the same position of the annotated object and the coordinate parameter corresponding to the annotation point are extracted from each coordinate parameter set, multiple coordinate parameters corresponding to the same annotation point are obtained, and the annotation distance between the multiple coordinate parameters is calculated. It is detected whether the annotation distance is lower than a threshold value. When the annotation distance is higher than the threshold value, it is determined that the coordinate parameter corresponding to the annotation distance is incorrect.

[0005] The above-described review method for key point marking requires obtaining at least two coordinate parameter sets after marking the marked object by two marking personnel, extracting two coordinate parameters corresponding to the same position of the marked object in the two coordinate parameter sets, and obtaining the coordinate distance between the coordinate parameters before reviewing the marked points. This method will consume a lot of manpower, and in the process of different marking personnel marking the marked objects, different marking personnel have different understandings of the positions of the marked points and have random deviations when marking the marked points, which will cause the same marked point to correspond to multiple different marked positions, resulting in the inability to confirm the wrong marked points when reviewing the marked points. Summary of the invention

[0006] The present application provides a method, device and electronic device for labeling verification, which screen out initial erroneous labeling points outside a preset labeling box and screen the labeling points in the preset labeling box using a first preset rule and a second preset rule, thereby screening out erroneous labeling points from different dimensions. The first preset rule is to detect according to the position of the labeling point to determine the first erroneous labeling point with a positional labeling error, and the second preset rule is to further determine the second erroneous labeling point according to the distance relationship between the labeling points and the vector product formed by the labeling points, thereby achieving accurate screening out of erroneous labeling points from all labeling points corresponding to the initial labeling image.

[0007] In a first aspect, the present application provides a marking verification method, the method comprising:

[0008] Acquire all the annotation points in the initial annotation image sent by the user terminal, and select the annotation points in a preset annotation frame from all the annotation points, and use the annotation points not in the preset annotation frame as initial erroneous annotation points;

[0009] Screening out the marking points in the preset marking frame that do not meet the first preset rule, and taking the marking points that do not meet the first preset rule as first erroneous marking points;

[0010] The extracted annotation points that meet the first preset rule are connected in pairs, the lines that do not meet the second preset rule are screened out from all the lines, and the annotation points corresponding to the lines that do not meet the second preset rule are used as second erroneous annotation points;

[0011] The initial erroneous marking point, the first erroneous marking point, and the second erroneous marking point are merged to obtain a target erroneous marking point corresponding to the initial marking image.

[0012] Through the method described above, all the annotation points in the initial annotation image are screened multiple times. First, the annotation points within the preset annotation box are screened out, and the initial erroneous annotation points with large deviations are preliminarily screened out. Then, the annotation points within the preset annotation box are screened using the first preset rule and the second preset rule, ensuring that the connecting lines with the erroneous annotation points can be obtained and the erroneous annotation points can be determined, thereby achieving the screening of erroneous annotation points from all annotations in the initial annotation image and improving the accuracy of screening out erroneous annotation points.

[0013] In one possible design, the annotation points in the preset annotation box are selected from all the annotation points, including:

[0014] Obtaining first coordinate parameters corresponding to each annotation point and coordinate parameters corresponding to vertices of the preset annotation frame;

[0015] Determining an area corresponding to the preset annotation box based on coordinate parameters corresponding to the vertices of the preset annotation box;

[0016] The marking points whose first coordinate parameters are within the area corresponding to the preset marking frame are selected from all the marking points.

[0017] In a possible design, screening out the annotation points in the preset annotation frame that do not conform to the first preset rule, and taking the annotation points that do not conform to the first preset rule as the first erroneous annotation points, includes:

[0018] Obtaining a first category attribute identifier corresponding to the annotation point in the preset annotation frame and a preset category attribute identifier corresponding to the preset annotation frame, wherein the category attribute identifier is a designated area category name of the annotated object;

[0019] Screening out the annotation points whose first category attribute identifier is inconsistent with the preset category attribute identifier;

[0020] The marking point that is inconsistent with the preset category attribute identifier is taken as the first erroneous marking point.

[0021] In a possible design, the extracted annotation points that meet the first preset rule are connected in pairs, including:

[0022] Extracting annotation points that meet a first preset rule from the initial annotation image, and using the annotation points as target annotation points;

[0023] Dividing the initial annotated image equally into a first area and a second area;

[0024] Connect all target marked points in the first area in pairs;

[0025] Connect all target annotation points in the second area in pairs.

[0026] In a possible design, the links that do not meet the second preset rule are screened out from all the links, and the marking points that do not meet the second preset rule are used as the second erroneous marking points, including:

[0027] Obtaining coordinate parameters of the marked points corresponding to the endpoints of the connecting line, and calculating the distance corresponding to the connecting line based on the coordinate parameters;

[0028] Filter out the connection lines corresponding to the distance being higher than the preset distance, and determine that the connection lines corresponding to the distance being higher than the preset distance do not meet the second preset rule;

[0029] The marking point corresponding to the connecting line that does not meet the second preset rule is used as the second erroneous marking point.

[0030] In a possible design, the links that do not meet the second preset rule are screened out from all the links, and the marking points that do not meet the second preset rule are used as second erroneous marking points, and further include:

[0031] Obtaining coordinate parameters of the marked points corresponding to the endpoints of the connecting line, and obtaining the vector corresponding to the connecting line based on the coordinate parameters;

[0032] Calculate the product between the two vectors, filter out the lines corresponding to the products being less than zero, and determine that the lines corresponding to the products being less than zero do not meet the second preset rule;

[0033] The marking point corresponding to the connecting line that does not meet the second preset rule is used as the second erroneous marking point.

[0034] In a second aspect, the present application provides a marking verification device, the device comprising:

[0035] An acquisition module is used to acquire all the annotation points in the initial annotation image sent by the user terminal, and filter out the annotation points in a preset annotation frame from all the annotation points, and use the annotation points not in the preset annotation frame as initial erroneous annotation points;

[0036] A screening module, used to screen out the marking points in the preset marking frame that do not meet the first preset rule, and use the marking points that do not meet the first preset rule as first erroneous marking points;

[0037] An extraction module, used to connect the extracted annotation points that meet the first preset rule in pairs, filter out the lines that do not meet the second preset rule among all the lines, and use the annotation points corresponding to the lines that do not meet the second preset rule as second erroneous annotation points;

[0038] The merging module is used to merge the initial erroneous marking point, the first erroneous marking point and the second erroneous marking point to obtain a target erroneous marking point corresponding to the initial marking image.

[0039] In a possible design, the acquisition module is specifically used to obtain the first coordinate parameters corresponding to each annotation point and the coordinate parameters corresponding to the vertices of the preset annotation box, determine the area corresponding to the preset annotation box based on the coordinate parameters corresponding to the vertices of the preset annotation box, and filter out the annotation points whose first coordinate parameters are within the area corresponding to the preset annotation box from all the annotation points.

[0040] In a possible design, the screening module is specifically used to obtain the first category attribute identifier corresponding to the annotation point in the preset annotation box and the preset category attribute identifier corresponding to the preset annotation box, screen out the annotation points whose first category attribute identifier is inconsistent with the preset category attribute identifier, and use the annotation points inconsistent with the preset category attribute identifier as the first erroneous annotation points.

[0041] In a possible design, the extraction module is specifically used to extract annotation points that meet the first preset rule in the initial annotation image, use the annotation points as target annotation points, equally divide the initial annotation image into a first area and a second area, connect all target annotation points in the first area in pairs, and connect all target annotation points in the second area in pairs.

[0042] In a possible design, the extraction module is also used to obtain coordinate parameters of the annotation points corresponding to the endpoints of the connecting line, calculate the distance corresponding to the connecting line based on the coordinate parameters, screen out the connecting lines corresponding to the distances higher than the preset distance, determine that the connecting lines corresponding to the distances higher than the preset distance do not comply with the second preset rules, and use the annotation points corresponding to the connecting lines that do not comply with the second preset rules as second erroneous annotation points.

[0043] In a possible design, the extraction module is also used to obtain coordinate parameters of the annotation points corresponding to the endpoints of the connecting line, obtain the vectors corresponding to the connecting line based on the coordinate parameters, calculate the product between the vectors, screen out the connecting lines corresponding to the products less than zero, determine that the connecting lines corresponding to the products less than zero do not meet the second preset rules, and use the annotation points corresponding to the connecting lines that do not meet the second preset rules as second erroneous annotation points.

[0044] In a third aspect, the present application provides an electronic device, including:

[0045] Memory, used to store computer programs;

[0046] The processor is used to implement the above-mentioned steps of a marking verification method when executing the computer program stored in the memory.

[0047] In a fourth aspect, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned annotation verification method are implemented.

[0048] For each aspect from the first to the fourth aspect and the technical effects that may be achieved by each aspect, please refer to the above description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of the annotator annotating the object provided for this application;

[0050] Figure 2 A flowchart of the steps of a marking verification method provided for this application;

[0051] Figure 3A schematic diagram of annotating an initial annotated image provided by this application;

[0052] Figure 4 A schematic diagram showing that the initial annotated image provided in the present application is equally divided into a first area and a second area;

[0053] Figure 5 A schematic diagram of the structure of a marking verification device provided in this application;

[0054] Figure 6 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The specific operating methods in the method embodiments can also be applied to device embodiments or system embodiments. It should be noted that in the description of the present application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A is connected to B, which can represent: A is directly connected to B and A is connected to B through C. In addition, in the description of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0056] In the previous technology, when reviewing whether there are erroneous marking points in the image of the marked object, at least two labelers are required to label the image of the marked object, which will consume a lot of manpower. When different labelers label the image of the marked object, they have different understandings of the positions of the marking points and there are random deviations when marking the marking points. This will cause the same marking point to correspond to multiple different marking positions, resulting in the inability to confirm erroneous marking points when reviewing the marking points.

[0057] In order to solve the above-described problems, the embodiments of the present application provide a method for labeling verification, which is used to detect erroneous labeling points from all labeling points corresponding to the labeled image. The method and device described in the embodiments of the present application are based on the same technical concept. Since the principles of the problems solved by the method and the device are similar, the embodiments of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0058] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0059] Reference Figure 2The present application provides a method for annotation verification, which can filter out erroneous annotation points from all annotation points corresponding to the annotated image. The implementation process of the method is as follows:

[0060] Step S21: Acquire all the annotation points in the initial annotation image sent by the user terminal, and select the annotation points in a preset annotation frame from all the annotation points, and use the annotation points not in the preset annotation frame as initial erroneous annotation points.

[0061] The embodiment of the present application is to filter out erroneous annotation points from all annotation points corresponding to the initial annotation image. Therefore, it is first necessary to obtain the initial annotation image sent by the user end. After obtaining the initial annotation image sent by the user end, all annotation points are extracted from the initial annotation image.

[0062] It should be further explained that the initial annotated image may be one initial annotated image or multiple initial annotated images. The embodiment of the present application takes one initial annotated image as an example.

[0063] After the above description of the initial annotation image, in order to obtain the erroneous annotation points with large deviations, it is necessary to determine the annotation points within the preset annotation box. In the embodiment of the present application, the position of the annotation points is determined based on the coordinate parameters. In order to more clearly illustrate the process of determining that the annotation points are within the preset annotation box based on the coordinate parameters, an example will be used for explanation. The specific process of determining that the annotation points are within the preset annotation box is as follows:

[0064] In order to ensure that the annotation point is within the preset annotation box, Figure 3 The diagram shows the schematic diagram of labeling the initial labeled image. Figure 3 In the figure, the intersection of the left edge and the lower edge of the initial annotation image is used as the origin to establish a rectangular coordinate system. Each annotation point in the initial annotation image can determine the corresponding first coordinate parameter. The first coordinate parameter can be used to accurately locate the annotation point. The black solid dot represents the annotation point. In order to obtain the area corresponding to the preset annotation box, and then to determine whether the annotation point is in the preset annotation box, it is necessary to obtain the coordinate parameters corresponding to the vertices of the preset annotation box. Figure 3 The vertices of the preset annotation box are points A and B. The area of ​​the preset annotation box can be determined according to the coordinate parameters of the two vertices A and B.

[0065] After obtaining the first coordinate parameter of each annotation point and the coordinate parameter corresponding to the vertex of the preset annotation box, in order to detect whether the annotation point is within the area of ​​the preset annotation box, it is necessary to determine whether the horizontal coordinate of the annotation point is within the horizontal coordinate range corresponding to the vertex of the preset annotation box and whether the vertical coordinate of the annotation point is within the vertical coordinate range corresponding to the vertex of the preset annotation box. When the horizontal coordinate of the annotation point is within the horizontal coordinate range corresponding to the vertex of the preset annotation box and the vertical coordinate is also within the vertical coordinate range corresponding to the vertex of the preset annotation box, it can be determined that the annotation point corresponding to the coordinate parameter is within the preset annotation box.

[0066] For example: the coordinates corresponding to the vertices of the preset annotation box are (1, 1) and (4, 4), and the coordinate of annotation point 1 is (2, 3). Based on (1, 1) and (4, 4), it can be determined that the horizontal coordinate range of the preset annotation box is [1, 4], and the vertical coordinate range of the preset annotation box is [1, 4]. The horizontal coordinate of annotation point 1 is: 1<2<4, and the vertical coordinate of annotation point 1 is: 1<3<4. Therefore, it can be known that annotation point 1 is within the area of ​​the preset annotation box.

[0067] By adopting the above method of detecting whether the annotation point is within the preset annotation box, each annotation point is detected, and the annotation points corresponding to the first coordinate parameters within the area of ​​the preset annotation box are screened out.

[0068] After the marking points within the area of ​​the preset marking box are screened out, the marking points within the area of ​​the preset marking box are removed from all the marking points as erroneous marking points, and the erroneous marking points are used as initial erroneous marking points.

[0069] Through the above description, all the annotation points corresponding to the initial annotation image are preliminarily screened, and the annotation points with large deviations from other annotation points are screened out, thereby ensuring the accuracy of screening out erroneous annotation points.

[0070] Step S22: filter out the marking points in the preset marking frame that do not conform to the first preset rule, and use the marking points that do not conform to the first preset rule as first erroneous marking points.

[0071] After filtering out the initial erroneous annotation points, in order to filter out the erroneous annotation points at the annotation position, it is necessary to obtain the first category attribute identifier of the annotation point and the preset category attribute identifier of the preset annotation box. Each annotation point is associated with the first category attribute identifier corresponding to the annotation point, and each preset annotation box is associated with the preset category attribute identifier corresponding to the preset annotation box. The category attribute identifier is the designated area category name of the annotated object.

[0072] It should be further explained that the association relationship between the annotation points and the first category attribute identifiers is shown in Table 1:

[0073]

[0074] Table 1

[0075] It should be noted in Table 1 that the number of annotation points corresponding to the first category attribute identifier can be set according to actual needs. In Table 1, the number of annotation points corresponding to the eyes is set to 2, and the number of annotation points corresponding to the eyebrows is set to 6. Through the three annotation points, one of the eyebrows of the annotated object can be determined. The association between other first category attribute identifiers and annotation points can be referred to Table 1, which will not be elaborated here.

[0076] After explaining the association between the annotation point and the first category attribute identifier, it is also necessary to explain the association between the preset annotation box and the preset category attribute identifier. The association between the preset annotation box and the preset category attribute identifier is shown in Table 2:

[0077] Preset callout box Preset category attribute identifier Preset label box 1 Eye Preset label box 2 Eyebrow ...... ......

[0078] Table 2

[0079] In Table 2, preset annotation box 1 is associated with eyes, preset annotation box 2 is associated with eyebrows, and the association relationship between other preset annotation boxes and preset category attribute identifiers can be referred to Table 2. It should be noted that the number of annotation points in the preset marking box can be set according to actual conditions. The number of annotation points in the preset marking box is at least one. The preset category attribute identifier is unique and is used to distinguish the preset annotation boxes to avoid incorrectly determined erroneous annotation points due to duplication of preset category attribute identifiers.

[0080] After obtaining the first category attribute identifier of the annotation point and the preset category attribute identifier of the preset annotation box, in order to determine the annotation point with incorrect annotation position, it is necessary to detect whether the annotation point in each preset annotation box meets the first preset rule. The first preset rule is that the first category attribute identifier corresponding to the annotation point is consistent with the preset category attribute identifier corresponding to the preset annotation box. If the first category attribute identifier is consistent with the preset category attribute identifier, then the annotation point corresponding to the first category attribute meets the first preset rule, indicating that the annotation position of the annotation point is correct. If the first category attribute identifier is inconsistent with the preset category attribute identifier, then the annotation point corresponding to the first category attribute does not meet the first preset rule, indicating that the annotation position of the annotation point is wrong. All the annotation points with incorrect annotation positions are obtained, and the annotation point is used as the first erroneous annotation point.

[0081] Through the method described above, the first erroneous marking point is determined by detecting that the first category attribute identifier of the marking point is inconsistent with the preset category attribute identifier, and the erroneous marking point caused by the marking position error is screened out.

[0082] Step S23: connecting the extracted annotation points that meet the first preset rule in pairs, filtering out the links that do not meet the second preset rule among all the links, and taking the annotation points that do not meet the second preset rule as second erroneous annotation points.

[0083] After determining the first erroneous annotation point, extract the annotation point that meets the first preset rule, and use the annotation point as the target annotation point. After obtaining the target annotation point, divide the initial annotation image equally into the first area and the second area, so as to realize the connection between the target annotation points. In order to more clearly explain the process of connecting the target annotation points, an example will be given. The specific process of connecting the target annotation points is as follows:

[0084] In the embodiment of the present application, the schematic diagram of equally dividing the initial annotated image into the first area and the second area is as follows: Figure 4 As shown, Figure 4 In the method, the midline of the labeled object in the initial labeled image is used to equally divide the initial labeled image. Since there is more than one way to equally divide, other ways of equally dividing are not described in detail here.

[0085] exist Figure 4 In the figure, E, F, and G are the target marking points of the first area, and C, D, and Z are the target marking points of the second area. The target marking points of the first area are connected in pairs. Figure 4 To connect E, F, and G in pairs, connect the target annotation points in the second area in pairs. Figure 4 In the figure, C, D, and Z are connected in pairs.

[0086] After connecting the target annotation points in the first area and the second area in pairs, in order to obtain the connection line with the wrong annotation point, the distance and vector corresponding to the connection line are detected. The specific process of obtaining the distance and vector corresponding to the connection line based on the coordinate parameters of the connection line is as follows:

[0087]

[0088] Table 3

[0089] Table 3 takes the connection EF as an example, d EF Represents the distance corresponding to the line EF. For other lines, the distance and vector corresponding to the line can be obtained based on the coordinate parameters, which can be referred to in Table 3. Since the process of obtaining the distance and vector corresponding to the line is the same, it will not be elaborated here to avoid repetition.

[0090] After the distance and vector corresponding to the connecting line are described above, it is necessary to determine the wrong marking points in the preset marking box. The specific process of determining the wrong marking points in the preset marking box using different methods is as follows:

[0091] Method 1: Determine the error mark point based on the distance corresponding to the connection line.

[0092] Through the above description of the preset marking box, the preset number value of the marking points in the preset marking box can be obtained. In order to determine whether there are erroneous marking points in the preset marking box, it is necessary to count the actual number value of the marking points in each preset marking box, and detect whether the actual number value is consistent with the preset number value of the marking points in the preset marking box. When the actual number value is inconsistent with the preset number value, it can be determined that there are erroneous marking points in the preset marking box.

[0093] After determining that there are erroneous annotation points in the preset annotation box, in order to determine the specific location of the erroneous annotation points, it is necessary to determine the connection line with the erroneous annotation points according to the distance corresponding to the connection line being higher than the preset distance. In order to more clearly illustrate the process of determining the erroneous annotation points based on the distance of the connection line, an example will be given to illustrate the specific process of determining the connection line with the erroneous annotation points.

[0094] exist Figure 3 In the figure, points E and Z represent the marking points corresponding to the ears of the marked object, points F and C represent the marking points corresponding to the eyes of the marked object, and points G and D represent the marking points corresponding to the shoulders of the marked object. After connecting points F, E, and G in the first area, under normal circumstances, the distance from the ear to the eye on one side of the marked object must be smaller than the distance from the ear to the shoulder, that is: d EF Less than d EG .

[0095] In the above example, the preset distance is d EG In the embodiment of the present application, the preset distance is not a fixed value. The preset distance is the distance between any two annotation points after the annotated object is correctly annotated. When determining the preset distance, the preset distance can be selected according to actual conditions on the basis that the preset distance is higher than the distance compared with the preset distance.

[0096] After explaining the preset distance, when d EF Higher than d EG When d EF The corresponding connection line does not meet the second preset rule, and it is determined that there is an incorrectly labeled point between points E and F. In order to further determine the incorrectly labeled point from points E and F, it is necessary to compare the distances corresponding to the connection lines of points E and F with other points. If the labeling is correct, the d corresponding to the labeled object EF Should be less than d EC point, if d EF Greater than d EC , then point F is determined to be an incorrectly marked point.

[0097] It should be further explained that, in order to determine the erroneous marking point based on the distance of the connecting line, at least two distance comparisons are required. When the marking points corresponding to the two endpoints in the connecting line are both erroneous marking points, the process of determining that the marking point is an erroneous marking point is consistent with the above description, but the number of comparisons is far more than the number described above. In the process of determining the erroneous marking point, the marking points of the first area and the second area can be connected by a line.

[0098] Method 2: Determine the incorrectly marked points based on the vectors corresponding to the connecting lines.

[0099] In method one, the erroneous marking points existing in the preset marking box are described. In order to determine the erroneous marking points, the preset marking box where the erroneous marking points exist can be locked first, and the marking points in the preset marking box can be connected to obtain the vectors corresponding to the connecting lines, and then the erroneous marking points can be determined based on the product between the vectors being less than zero. Alternatively, the marking points in the preset marking box can be unlocked, and connecting lines can be made for all target marking points to obtain the vectors corresponding to the connecting lines, and then the erroneous marking points can be determined by the product between the two vectors being less than zero.

[0100] The above-described situations are all based on line training. Since the process of determining the wrong labeling point based on the product between vectors being less than zero is the same, in order to avoid a lot of repetition, they are not described separately here. The specific process of determining the wrong labeling point based on the vector obtained by the line is as follows:

[0101] In the above description, the coordinate vector of each target annotation point has been obtained. The vector is a line segment with a direction. When the product between the two obtained vectors is less than zero, it means that the lines corresponding to the two vectors cross. Since the target annotation points are divided into the first area and the second area, the lines corresponding to the target annotation points are also divided into the lines of the first area and the lines of the second area. The lines of the first area and the lines of the second area will not cross when the lines are normal. When the lines of the first area and the second area cross, it means that there are erroneous annotation points in the target annotation points corresponding to the crossed lines.

[0102] Furthermore, in order to detect whether there are cross-connections, it is necessary to randomly multiply the vectors corresponding to the connections in the first area with the vectors corresponding to the connections in the second area. It should be noted that, for example: Figure 4 In the vector With vector There is no intersection between them. When there is an intersection, it means that the vector With vector There must be erroneous annotation points among the corresponding annotation points.

[0103] To determine the mislabeled points, it is necessary to form vectors by connecting line segment EF to either point C or point Z, and connect point E to point C to form a vector. Then multiply the vector by the vector to obtain the first product. Connect point E to point Z to form a vector Then multiply the vector by the vector to obtain the second product. Since the multiplication calculation of vectors is common knowledge, no specific description is provided here. When the product of the first vector and the second vector is less than zero, it means that point C and point Z are on both sides of line segment EF. When the product of the first vector and the second vector is greater than zero, it means that point C and point Z are on the same side of line segment EF.

[0104] When the product of the first vector and the second vector is less than zero, it can be determined that there are mislabeled points among point C and point Z. To determine whether one of point C and point Z is the mislabeled point, or both point C and point Z are mislabeled points, it is necessary to determine the coordinates of point C and point Z. The coordinates of point C are (c, d), and the coordinates of point Z are (a, b). Given that the coordinates of point E are (g, h) and the coordinates of point F are (j, k), and h < k, check whether c and a are less than k, and take the target labeled point corresponding to the abscissa value less than k as the mislabeled point.

[0105] Furthermore, when there are multiple intersecting connections, in the embodiments of the present application, it is also possible to perform multiple calculations in the manner described above where the product of two vectors is less than zero. Since the calculation method and the method of determining the incorrect connection have been described above, no further description is provided here.

[0106] It should be noted that in the embodiments of the present application, either method one or method two can be used to screen out the mislabeled points, or a combination of method one and method two can be used to screen out the mislabeled points. Since the process of screening out the mislabeled points is the same, the process of screening out the mislabeled points using a combination of method one and method two refers to the above description of method one and method two, and no further elaboration is provided here.

[0107] Take the mislabeled points screened out by the method described above as the second mislabeled points.

[0108] Based on the above description of the distance corresponding to the connection and the vector, by comparing the distance with the preset distance, the incorrect marked points that do not conform to logic are determined, and the orientation of the target labeled points is determined through the product of two vectors, thereby realizing the screening of mislabeled points from the target labeled points and improving the accuracy of screening out mislabeled points.

[0109] Step S24: merging the initial erroneous marking point, the first erroneous marking point, and the second erroneous marking point to obtain a target erroneous marking point corresponding to the initial marking image.

[0110] In the above steps, the initial error marking point, the first error marking point and the second error marking point have been obtained. In order to finally present all the error marking points to the user, the initial error marking point, the first error marking point and the second error marking point need to be merged, and the merged error marking point is used as the target error marking point. Finally, the error marking point corresponding to the initial annotated image is obtained.

[0111] It should be noted that, for filtering out erroneous annotation points from all annotation points of multiple initial annotated images, reference can be made to the process of filtering out erroneous annotation points from all annotation points of an initial annotated image in the embodiment of the present application. Since the process of filtering out erroneous annotation points is the same, it will not be elaborated here to avoid repetition.

[0112] Based on the method described above, target erroneous annotation points are screened out from all annotation points corresponding to the initial annotation image in different ways. First, all annotation points are preliminarily screened using a preset annotation box to obtain target annotation points. First erroneous annotation points are screened out from the target annotation points using a first preset rule. Second erroneous annotation points are screened out from the target annotation points using a second preset rule. Multiple screening of all annotation points is achieved, ensuring that erroneous annotation points can be screened out from all annotation points and improving the accuracy of the screened erroneous annotation points.

[0113] Based on the same inventive concept, the embodiment of the present application also provides a marking verification device, the thread-bound device is used to implement the function of a marking verification method, referring to Figure 5 , the device comprises:

[0114] The obtaining module 501 is used to obtain all the annotation points in the initial annotation image sent by the user terminal, and select the annotation points in the preset annotation box from all the annotation points, and use the annotation points not in the preset annotation box as initial erroneous annotation points;

[0115] A screening module 502 is used to screen out the marking points in the preset marking frame that do not meet the first preset rule, and use the marking points that do not meet the first preset rule as first erroneous marking points;

[0116] An extraction module 503 is used to connect the extracted annotation points that meet the first preset rule in pairs, filter out the lines that do not meet the second preset rule among all the lines, and use the annotation points corresponding to the lines that do not meet the second preset rule as second erroneous annotation points;

[0117] The merging module 504 is used to merge the initial erroneous marking point, the first erroneous marking point, and the second erroneous marking point to obtain a target erroneous marking point corresponding to the initial marked image.

[0118] In a possible design, the acquisition module 501 is specifically used to obtain the first coordinate parameters corresponding to each annotation point and the coordinate parameters corresponding to the vertices of the preset annotation box, determine the area corresponding to the preset annotation box based on the coordinate parameters corresponding to the vertices of the preset annotation box, and filter out the annotation points whose first coordinate parameters are within the area corresponding to the preset annotation box from all the annotation points.

[0119] In a possible design, the screening module 502 is specifically used to obtain the first category attribute identifier corresponding to the annotation point in the preset annotation box and the preset category attribute identifier corresponding to the preset annotation box, screen out the annotation points whose first category attribute identifier is inconsistent with the preset category attribute identifier, and use the annotation points inconsistent with the preset category attribute identifier as the first erroneous annotation points.

[0120] In a possible design, the extraction module 503 is specifically used to extract annotation points that meet the first preset rule in the initial annotation image, use the annotation points as target annotation points, equally divide the initial annotation image into a first area and a second area, connect all target annotation points in the first area in pairs, and connect all target annotation points in the second area in pairs.

[0121] In a possible design, the extraction module 503 is also used to obtain coordinate parameters of the annotation points corresponding to the endpoints of the connecting line, calculate the distance corresponding to the connecting line based on the coordinate parameters, filter out the connecting lines corresponding to the distances higher than the preset distance, determine that the connecting lines corresponding to the distances higher than the preset distance do not comply with the second preset rules, and use the annotation points corresponding to the connecting lines that do not comply with the second preset rules as second erroneous annotation points.

[0122] In a possible design, the extraction module 503 is also used to obtain coordinate parameters of the annotation points corresponding to the endpoints of the connecting line, obtain the vectors corresponding to the connecting line based on the coordinate parameters, calculate the product between the vectors, screen out the connecting lines corresponding to the products less than zero, determine that the connecting lines corresponding to the products less than zero do not meet the second preset rules, and use the annotation points corresponding to the connecting lines that do not meet the second preset rules as second erroneous annotation points.

[0123] Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application, and the electronic device can realize the functions of the aforementioned marking verification device, referring to Figure 6 , the electronic device comprises:

[0124] At least one processor 601, and a memory 602 connected to the at least one processor 601. The specific connection medium between the processor 601 and the memory 602 is not limited in the embodiment of the present application. Figure 6 In the example, the processor 601 and the memory 602 are connected via a bus 600. The bus 600 is Figure 6 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 600 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 601 can also be called a controller, and there is no limitation on the name.

[0125] In the embodiment of the present application, the memory 602 stores instructions that can be executed by at least one processor 601. The at least one processor 601 can execute a method of marking verification discussed above by executing the instructions stored in the memory 602. The processor 601 can implement Figure 5 The functions of each module in the device shown.

[0126] Among them, the processor 601 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 602 and calling the data stored in the memory 602, the various functions of the device and processing data, the device can be monitored as a whole.

[0127] In one possible design, the processor 601 may include one or more processing units, and the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0128] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of a labeling verification method disclosed in the embodiments of the present application can be directly embodied as a hardware processor execution, or a combination of hardware and software modules in the processor.

[0129] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 602 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 602 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0130] By programming the processor 601, the code corresponding to the annotation verification method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 2 The illustrated embodiment is a step of marking verification. How to design and program the processor 601 is a technique known to those skilled in the art and will not be described in detail here.

[0131] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes a marking verification method discussed above.

[0132] In some possible implementations, various aspects of a method for labeling verification provided by the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of a labeling verification method according to various exemplary implementations of the present application described above in this specification.

[0133] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0134] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0137] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A marking verification method, characterized in that: include: Acquire all the annotation points in the initial annotation image sent by the user terminal, and select the annotation points in a preset annotation frame from all the annotation points, and use the annotation points that are not in the preset annotation frame as initial erroneous annotation points; Screening out the marking points in the preset marking frame that do not meet the first preset rule, and taking the marking points that do not meet the first preset rule as first erroneous marking points; The extracted annotation points that meet the first preset rule are connected in pairs, the lines that do not meet the second preset rule are screened out from all the lines, and the annotation points corresponding to the lines that do not meet the second preset rule are used as second erroneous annotation points; Merging the initial erroneous marking point, the first erroneous marking point, and the second erroneous marking point to obtain a target erroneous marking point corresponding to the initial marking image; The step of filtering out the annotation points in the preset annotation frame that do not conform to the first preset rule and taking the annotation points that do not conform to the first preset rule as the first erroneous annotation points includes: Obtaining a first category attribute identifier corresponding to the annotation point in the preset annotation frame and a preset category attribute identifier corresponding to the preset annotation frame, wherein the category attribute identifier is a designated area category name of the annotated object; Screening out the annotation points whose first category attribute identifier is inconsistent with the preset category attribute identifier; The marking point that is inconsistent with the preset category attribute identifier is taken as the first erroneous marking point.

2. The method according to claim 1, characterized in that And filter out the annotation points in the preset annotation box from all the annotation points, including: Obtaining first coordinate parameters corresponding to each annotation point and coordinate parameters corresponding to vertices of the preset annotation frame; Determining an area corresponding to the preset annotation box based on coordinate parameters corresponding to the vertices of the preset annotation box; The marking points whose first coordinate parameters are within the area corresponding to the preset marking frame are selected from all the marking points.

3. The method according to claim 1, characterized in that The extracted annotation points that meet the first preset rule are connected in pairs, including: Extracting annotation points that meet a first preset rule from the initial annotation image, and using the annotation points as target annotation points; Dividing the initial annotated image equally into a first area and a second area; Connect all target marked points in the first area in pairs; Connect all target annotation points in the second area in pairs.

4. The method according to claim 1, characterized in that Filter out the links that do not meet the second preset rule from all the links, and use the annotation points that do not meet the second preset rule as the second erroneous annotation points, including: Obtaining coordinate parameters of the marked points corresponding to the endpoints of the connecting line, and calculating the distance corresponding to the connecting line based on the coordinate parameters; Filter out the connection lines corresponding to the distance being higher than the preset distance, and determine that the connection lines corresponding to the distance being higher than the preset distance do not meet the second preset rule; The marking point corresponding to the connecting line that does not meet the second preset rule is used as the second erroneous marking point.

5. The method according to claim 1, characterized in that Screening out the links that do not meet the second preset rule from all the links, and taking the marking points that do not meet the second preset rule as second erroneous marking points, further comprising: Obtaining coordinate parameters of the marked points corresponding to the endpoints of the connecting line, and obtaining the vector corresponding to the connecting line based on the coordinate parameters; Calculate the product between the two vectors, filter out the lines corresponding to the products being less than zero, and determine that the lines corresponding to the products being less than zero do not meet the second preset rule; The marking point corresponding to the connecting line that does not meet the second preset rule is used as the second erroneous marking point.

6. A marking verification device, characterized in that: The device comprises: An acquisition module is used to acquire all the annotation points in the initial annotation image sent by the user terminal, and filter out the annotation points in a preset annotation frame from all the annotation points, and use the annotation points not in the preset annotation frame as initial erroneous annotation points; A screening module, used to screen out the marking points in the preset marking frame that do not meet the first preset rule, and use the marking points that do not meet the first preset rule as first erroneous marking points; An extraction module, used to connect the extracted annotation points that meet the first preset rule in pairs, filter out the lines that do not meet the second preset rule among all the lines, and use the annotation points corresponding to the lines that do not meet the second preset rule as second erroneous annotation points; A merging module, used to merge the initial erroneous marking point, the first erroneous marking point, and the second erroneous marking point to obtain a target erroneous marking point corresponding to the initial marking image; Wherein, the screening module is specifically used for: Obtaining a first category attribute identifier corresponding to the annotation point in the preset annotation frame and a preset category attribute identifier corresponding to the preset annotation frame, wherein the category attribute identifier is a designated area category name of the annotated object; Screening out the annotation points whose first category attribute identifier is inconsistent with the preset category attribute identifier; The marking point that is inconsistent with the preset category attribute identifier is taken as the first erroneous marking point.

7. The device according to claim 6, characterized in that The obtaining module is specifically used to obtain the first coordinate parameters corresponding to each annotation point and the coordinate parameters corresponding to the vertices of the preset annotation box, determine the area corresponding to the preset annotation box based on the coordinate parameters corresponding to the vertices of the preset annotation box, and filter out the annotation points whose first coordinate parameters are within the area corresponding to the preset annotation box from all the annotation points.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 5 when executing the computer program stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.

Images