Image processing method, device and storage medium

Abstract

The application belongs to the technical field of computers, and particularly relates to an image processing method, device and storage medium. The method comprises the following steps: obtaining at least two labeled images obtained by labeling a same target image, each labeled image having a first labeling box obtained by labeling one or more target objects in the target image; determining a group of first labeling boxes corresponding to the target objects in the labeled image; comparing first labeling box information corresponding to the group of first labeling boxes to obtain a comparison result; and generating a labeling result of the target object based on the first labeling box information in the case that the comparison result is consistent. The method can solve the problem that different labeling personnel may have inconsistent understanding of the same sample image, which may cause the image labeling result to be inconsistent. The labeling result of the target object is generated based on the obtained labeled image, and manual participation in the labeling box processing process is not required, so that the image labeling result can be ensured to be consistent, and the error rate of image labeling can be reduced.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer, and particularly relates to an image processing method, device and storage medium. BACKGROUND

[0002] At present, an image recognition model is usually used to identify a target object in an image. Before being put into use, the image recognition model needs to be trained using a sample image after labeling.

[0003] A traditional labeling process of a sample image includes that a labeling personnel labels a target object in a sample image to obtain a labeled image. One sample image can be labeled by multiple labeling personnel.

[0004] However, different labeling personnel can have different understandings of the same sample image, which can cause the problem of inconsistent image labeling results. In addition, when facing a large amount of data, the labeling personnel can have the problems of missing labeling and incorrect labeling. SUMMARY

[0005] The present application provides an image processing method, device and storage medium, which can solve the problem that different labeling personnel can have different understandings of the same sample image, which can cause the problem of inconsistent image labeling results. In addition, when facing a large amount of data, the labeling personnel can have the problems of missing labeling and incorrect labeling.

[0006] The present application provides the following technical solutions:

[0007] In a first aspect, an image processing method is provided, and the method includes:

[0008] Obtaining at least two labeled images obtained by labeling a same target image, each labeled image having a first labeling box obtained by labeling one or more target objects in the target image;

[0009] Determining a group of first labeling boxes corresponding to the target objects in the at least two labeled images;

[0010] Comparing first labeling box information corresponding to the group of first labeling boxes to obtain a comparison result;

[0011] In a case where the comparison result indicates that the group of first labeling boxes is consistent, generating a labeling result of the target object based on the first labeling box information.

[0012] Optionally, the determining a group of first labeling boxes corresponding to the same target object in the at least two labeled images includes:

[0013] Based on the matching degree between the first annotation boxes in different annotated images, a group of first annotation boxes corresponding to the same target object in the different annotated images is determined.

[0014] Optionally, determining a group of first annotation boxes corresponding to the same target object in the different annotated images based on the matching degree between the first annotation boxes in the different annotated images includes:

[0015] For a first annotated image of the at least two annotated images, determining a reference frame based on a first annotated frame of the first annotated image, each of the reference frames corresponding to one of the target objects;

[0016] For a second annotated image other than the first annotated image in the at least two annotated images, matching a first annotated frame in the second annotated image with the reference frame to obtain a matching degree between the first annotated frame in the second annotated image and the reference frame;

[0017] It is determined that the first annotation frame in the second annotation image, whose matching degree with the reference frame meets a first preset requirement, belongs to a group of first annotation frames of the target object corresponding to the reference frame.

[0018] Optionally, determining the reference frame based on the first annotated frame of the first annotated image includes:

[0019] When the matching degree of the first annotation frames in different first annotation images meets a second preset requirement, determining the reference frame based on the matched first annotation frames;

[0020] or,

[0021] When the degree of matching of the first annotation frames in different first annotation images meets the second preset requirement, a first matching frame is determined based on the matched first annotation frame, and the first matching frame is determined as the reference frame; when the degree of matching of the first annotation frames in different first annotation images does not meet the second preset requirement, a second matching frame is determined based on the unmatched first annotation frame, and the second matching frame is determined as the reference frame.

[0022] Optionally, determining the reference frame based on the first annotated frame of the first annotated image includes:

[0023] Initializing n first annotated images from the at least two annotated images, and determining a degree of matching between first annotated frames in the n first annotated images, where n is an integer greater than or equal to 2;

[0024] Determining the reference frame based on the intersection of the first annotation frames in the n first annotated images whose matching degree meets the second preset requirement;

[0025] After determining the first annotation frame in the set of first annotation frames from the second annotated image, the second annotated image is used as the first annotated image, and an intersection between the first annotation frame in the set of first annotation frames and the corresponding reference frame is determined to obtain an updated reference frame. The updated reference frame is used as the reference frame for next matching with the second annotated image.

[0026] Optionally, the method further includes:

[0027] When the matching degree of the first annotation frame in the second annotation image does not meet the first preset requirement, a first prompt is output, where the first prompt is used to prompt a review of the unmatched first annotation frame.

[0028] Optionally, matching the first annotated frame in the second annotated image with the reference frame to obtain a matching degree of the first annotated frame in the second annotated image includes:

[0029] Determine the intersection-over-union ratio between the first annotated frame and each reference frame to obtain a matching degree of the first annotated frame in the second annotated image.

[0030] Optionally, comparing the first markup box information of the group of first markup boxes to obtain a comparison result includes:

[0031] Obtaining annotation categories corresponding to the group of first annotation boxes; and if the annotation categories are consistent, determining that the comparison result indicates that the group of first annotation boxes are consistent;

[0032] and / or,

[0033] Acquire position information corresponding to the group of first annotation frames; and determine, when the position information indicates that a distance between each first annotation frame is less than a preset distance, that the comparison result indicates that the group of first annotation frames is consistent.

[0034] Optionally, the annotation result includes the position of the second annotation box corresponding to the target object and the annotation category of the target object;

[0035] Generating a labeling result of the target object based on the first labeling frame information includes:

[0036] determining a position of the second markup box based on position information of each first markup box in the group of first markup boxes;

[0037] The labeling categories of the group of first labeling frames are obtained to obtain the labeling category of the target object.

[0038] Optionally, the method further includes:

[0039] In a case where the comparison result indicates that the set of first annotation boxes are inconsistent, a second prompt is output, where the second prompt is used to instruct the set of first annotation boxes to be reviewed.

[0040] In a second aspect, an electronic device is provided, wherein a program is stored in the memory, and the program is loaded and executed by the processor to implement the image processing method as described in the first aspect.

[0041] In a third aspect, a computer-readable storage medium is provided, wherein the storage medium stores a program, and when the program is executed by a processor, the program is used to implement the image processing method as described in the first aspect.

[0042] The beneficial effects of the present application are: by acquiring at least two annotated images obtained by annotating the same target image, each annotated image has a first annotated frame obtained by annotating one or more target objects in the target image; determining a group of first annotated frames corresponding to the target objects in the at least two annotated images; comparing the first annotated frame information corresponding to a group of first annotated frames to obtain a comparison result; when the comparison result indicates that a group of first annotated frames are consistent, generating an annotation result of the target object based on the first annotated frame information; it can solve the problem that different annotators may have inconsistent understandings of the same sample image, which may cause inconsistent image annotation results. In addition, when annotators face a large amount of data, there may be problems of missed or wrong labels; by generating the annotation result of the target object from the acquired annotated image, there is no need for manual participation in the annotation frame processing process, which can ensure the uniformity of the image annotation results and reduce the error rate of image annotation.

[0043] In addition, since missed labels may occur when annotating images, even if the first annotation box in one annotated image is correctly annotated, there may be a problem that the corresponding first annotation box cannot be matched in another annotated image. To address the above technical problem, the first annotation box whose matching degree does not meet the second preset requirement is also matched as a reference box. This can ensure the comprehensiveness of the matching, thereby improving the accuracy of determining a group of first annotation boxes corresponding to the same target object.

[0044] In addition, since mislabeling may occur during annotation, the matching degree of the first annotation box in the second annotated image may not meet the first preset requirement. To address the above technical problem, by outputting a first prompt when the matching degree of the first annotation box in the second annotated image does not meet the first preset requirement, the unmatched first annotation box can be reviewed, thereby improving the accuracy of the annotation of the first annotation box.

[0045] In addition, since the set of labeled boxes matched by the matching degree is only a set of first labeled boxes preliminarily screened by the intersection-over-union ratio, it cannot guarantee that the set of first labeled boxes are consistent. To address the above technical issues, this embodiment compares the position information and the label category at the same time, so that the consistency of the set of first labeled boxes can be guaranteed.

[0046] In addition, since annotation errors or erroneous annotations may occur during the annotation process, resulting in inconsistent comparison results, to address the above technical issues, when the comparison result indicates that a group of first annotation boxes is inconsistent, a second prompt is output, which is used to instruct the group of first annotation boxes to be reviewed. Therefore, the group of annotation boxes with inconsistent comparison results can be modified to generate a correct annotation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 is a flowchart of an image processing method provided by an embodiment of the present application;

[0049] Figure 2 This is a schematic diagram of an unlabeled target image provided by an embodiment of the present application;

[0050] Figure 3 is a schematic diagram of annotated images provided by one embodiment of the present application;

[0051] Figure 4 is a schematic diagram of annotated images provided by another embodiment of the present application;

[0052] Figure 5 is a schematic diagram of generating a reference frame provided by an embodiment of the present application;

[0053] Figure 6 is a schematic diagram of a first annotation frame matching process provided by an embodiment of the present application;

[0054] Figure 7 This is a schematic diagram of the positions of a group of first annotation frames provided in one embodiment of the present application;

[0055] Figure 8 is a block diagram of an image processing device provided by one embodiment of the present application;

[0056] Figure 9 This is a block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solutions of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present application, not all embodiments. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other unless there is a conflict.

[0058] It should be noted that the terms "first", "second", etc. in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0059] In this application, unless otherwise specified, directional words such as "up, down, top, bottom" are usually used with reference to the directions shown in the drawings, or with reference to the components themselves in the vertical, perpendicular or gravity direction; similarly, for ease of understanding and description, "inside and outside" refer to the inside and outside relative to the outline of each component itself, but the above directional words are not used to limit this application.

[0060] like Figure 1 The image processing method provided by an embodiment of the present application is shown. The execution subject of the image processing method is an electronic device, which can be a terminal such as a computer, a mobile phone, a tablet computer, or a server, or a mobile device such as a sweeping robot or a multi-legged robot. This embodiment does not limit the implementation method of the electronic device. Figure 1 , the method comprises at least the following steps:

[0061] Step 101 : Acquire at least two annotated images obtained by annotating the same target image, each annotated image having a first annotated frame obtained by annotating one or more target objects in the target image.

[0062] Methods for obtaining annotated images include but are not limited to the following:

[0063] The first method is to copy the target image to be labeled to obtain multiple copies of the target image; and receive labeled images obtained after different labelers label each image copy.

[0064] The second method involves obtaining annotated images corresponding to the same target image sent by multiple other devices. These annotated images are obtained by annotating copies of the same target image. The other devices are communicatively connected to the electronic device. Optionally, different annotators use different other devices, such as computers, mobile phones, tablet computers, and other terminals. This embodiment does not limit the type of other devices.

[0065] In actual implementation, the method for obtaining the annotated image may also be other methods, and this embodiment does not limit the method for obtaining the annotated image.

[0066] According to the above acquisition method, since different annotated images may be annotated by different annotators, the number of first annotation boxes in different annotated images may vary. To address this technical issue, this embodiment uses the following technical solution to review the first annotation boxes in the annotated images, and can automatically review annotated images with abnormal first annotation boxes.

[0067] The first annotation box is used to annotate the target object in the target image. The first annotation box is determined based on the edge contour of the target object. The first annotation box can be a rectangle, a diamond, or other shapes. This embodiment does not limit the shape of the first annotation box.

[0068] Optionally, when there are multiple first annotation boxes, the categories of target objects corresponding to different first annotation boxes are the same or different.

[0069] Optionally, each target object corresponds to a first annotation box. Based on this, since the target image includes one or more target objects, each annotated image includes one or more first annotation boxes.

[0070] The target object refers to an object that needs to be labeled in the target image, for example, the target object is a pedestrian in the target image, a car in the target image, or a non-motor vehicle in the target image.

[0071] like Figure 2 As shown, Figure 2 is an unlabeled target image containing multiple target objects of different categories, such as motor vehicles, non-motor vehicles, pedestrians, etc. Figure 3 and Figure 4 There are two annotated images respectively, and each annotated image has a first annotated frame for annotating a motor vehicle, a non-motor vehicle, and a pedestrian.

[0072] Step 102: Determine a group of first annotation boxes corresponding to the target objects in at least two annotated images.

[0073] In one example, determining a group of first annotation boxes corresponding to the same target object in at least two annotated images includes: determining a group of first annotation boxes corresponding to the same target object in different annotated images based on a degree of matching between the first annotation boxes in different annotated images.

[0074] Since the probability of the first annotation boxes of the target object in different annotated images overlapping is relatively high when annotating the same target object, the matching degree is positively correlated with the degree of overlap between the first annotation boxes in different annotated images.

[0075] Determining a group of first annotation boxes corresponding to the same target object in different annotated images based on the matching degree between the first annotation boxes in different annotated images includes at least the following steps S1-S3:

[0076] Step S1: for a first annotated image among the at least two annotated images, determining a reference frame based on a first annotated frame of the first annotated image, each reference frame corresponding to a target object.

[0077] Determining the reference frame based on the matched first annotation frame includes the following steps S11-S12:

[0078] Step S11: Initialize and determine n first annotated images from at least two annotated images, and determine the matching degree between each first annotated frame in the n first annotated images, where n is an integer greater than or equal to 2.

[0079] Optionally, determining the degree of matching between the first annotated frames in the n first annotated images includes calculating intersection-over-union ratios between the first annotated frames in different first annotated images to obtain the degree of matching between the first annotated frames in different first annotated images.

[0080] Specifically, an unmatched first annotation frame is determined in each of the n first annotation images to obtain n first annotation frames; and an intersection-over-union ratio of the n first annotation frames is determined as: the intersection of the first annotation frames divided by the union of the first annotation frames.

[0081] Step S12: determining a reference frame based on the intersection of the first annotated frames in the n first annotated images whose matching degree meets the second preset requirement;

[0082] The second preset requirement includes that the intersection-over-union ratio between the first annotation boxes in the n first annotated images is greater than or equal to a first intersection-over-union ratio threshold.

[0083] Determining the reference frame based on the first annotation frame of the first annotated image includes but is not limited to the following two methods:

[0084] The first method is to determine the reference frame based on the matched first annotation frames when the matching degrees of the first annotation frames in different first annotation images meet a second preset requirement.

[0085] The second method is to determine a first matching frame based on the matched first annotation frame, and determine the first matching frame as the reference frame, when the matching degree of the first annotation frame in different first annotation images meets the second preset requirement; and to determine a second matching frame based on the unmatched first annotation frame, and determine the second matching frame as the reference frame, when the matching degree of the first annotation frame in different first annotation images does not meet the second preset requirement.

[0086] Optionally, when the matching degree of the first annotation boxes in different first annotation images does not meet the second preset requirement, a preset prompt is output, and the preset prompt is used to prompt the unmatched first annotation boxes to be reviewed.

[0087] For example: Figure 5 As shown, there are two first annotated images 51 and 52. The first annotated image 51 has three first annotated frames A1, B1, and C1, and the first annotated image 52 has three first annotated frames A2, B2, and C2. At this time, if the intersection-over-union ratio of A1 and A2 is greater than 0.8 and the intersection-over-union ratio of B1 and B2 is greater than 0.8, the intersection of A1 and A2 and the intersection of B1 and B2 are used as reference frames. If the intersection-over-union ratio of C1 and C2 is less than 0.8, C1 and C2 are used as two reference frames respectively. That is, the reference frame 53 is composed of four parts, namely: the intersection of A1 and A2 (the shaded part in the figure), the intersection of B1 and B2 (the shaded part in the figure), C1, and C2.

[0088] Since missing labels may occur during image annotation, even if the first annotation box in one annotated image is correctly annotated, it may not match the corresponding first annotation box in another annotated image. To address this technical issue, by also matching first annotation boxes that do not meet the second preset matching requirement as reference boxes, the comprehensiveness of the matching can be guaranteed, thereby improving the accuracy of determining a group of first annotation boxes corresponding to the same target object.

[0089] Step S2: For a second annotated image other than the first annotated image in the at least two annotated images, matching the first annotated frame in the second annotated image with the reference frame to obtain a matching degree between the first annotated frame and the reference frame in the second annotated image.

[0090] Matching the first annotated frame in the second annotated image with the reference frame to obtain a matching degree between the first annotated frame and the reference frame in the second annotated image includes: determining an intersection-over-union ratio between the first annotated frame and each reference frame to obtain a matching degree of the first annotated frame in the second annotated image.

[0091] Step S3: Determine that a first annotated frame in the second annotated image whose matching degree with the reference frame meets a first preset requirement belongs to a group of first annotated frames of the target object corresponding to the reference frame.

[0092] The first preset requirement is that an intersection-over-union ratio (IoU) between the first annotation frame and the reference frame in the second annotation image satisfies a second IoU threshold.

[0093] Taking the second IoU threshold of 0.9 as an example, if the IoU of the first annotated box in the second annotated image and the reference box is 0.95, it is determined that the first annotated box in the second annotated image belongs to a group of first annotated boxes of the target object corresponding to the reference box.

[0094] After step S3, i.e., after a matching first annotated frame is determined from the second annotated image, the second annotated image is used as the first annotated image. If an unmatched second annotated image exists, step S1 is performed again. At this time, determining the reference frame based on the first annotated frame of the first annotated image includes: after determining a matching first annotated frame from the second annotated image, using the second annotated image as the first annotated image, determining the intersection between the first annotated frames in a set of first annotated frames and the corresponding reference frames to obtain an updated reference frame, and using the updated reference frame as the reference frame for the next matching with the second annotated image.

[0095] Optionally, when the matching degree of the first annotation frame in the second annotation image does not meet the first preset requirement, a first prompt is output, where the first prompt is used to prompt a review of the unmatched first annotation frame.

[0096] Optionally, the first prompt may be highlighting the unmatched first annotation box or bolding the unmatched first annotation box. This embodiment does not limit the implementation method of the first prompt.

[0097] In addition, since mislabeling may occur during annotation, the matching degree of the first annotation box in the second annotated image may not meet the first preset requirement. To address the above technical problem, by outputting a first prompt when the matching degree of the first annotation box in the second annotated image does not meet the first preset requirement, the unmatched first annotation box can be reviewed, thereby improving the accuracy of the annotation of the first annotation box.

[0098] In order to more clearly understand the matching process of the first annotation frame, the following is a detailed description of the process with an example. In this example, the second reference frame determination method mentioned above is used as an example. Figure 6 :

[0099] There are three annotated images 61, 62, and 63. Annotated image 61 has a first annotated box A1, B1, and C1; annotated image 62 has a first annotated box A2, B2, and C2; and annotated image 63 has a first annotated box A3, B3, and C3. Two annotated images are randomly selected from the three annotated images to obtain a first annotated image. For example, annotated image 61 and annotated image 62 are used as the first annotated images. Then, first annotated box A1 is sequentially matched with first annotated boxes A2, B2, and C2; first annotated box B1 is sequentially matched with first annotated boxes A2, B2, and C2; and first annotated box C1 is sequentially matched with first annotated boxes A2, B2, and C2. The intersection of the first annotated boxes with an intersection-over-union ratio greater than 0.8 is used as a reference box. First annotated boxes C1 and C2 with an intersection-over-union ratio less than 0.8 are also used as reference boxes 64. At the same time, first annotated boxes C1 and C2 are displayed in bold for review by the reviewer.

[0100] For the unmatched standard image 63, this standard image 63 is used as the second annotated image. Next, the first annotated frame A3 is sequentially matched with the reference frame 64, the first annotated frame B3 is sequentially matched with the reference frame 64, and the first annotated frame C3 is sequentially matched with the reference frame 64. The first annotated frames with an IoU ratio greater than 0.9 are used as a set of first annotated frames for the target objects corresponding to the reference frames. If the IoU ratio of the first annotated frame A3 and the first annotated frame B3 with the reference frame 64 is greater than 0.9, a set of first annotated frames corresponding to the two target objects is obtained. If the IoU ratio of the first annotated frame A3 with the reference frame 64 is less than 0.9, the first annotated frame C3 is displayed in bold for review by the reviewer.

[0101] Afterwards, if there are any unmatched second annotated images, the annotated image 63 is used as the first annotated image, and the intersection of the first annotated frames corresponding to the two groups of target objects (the black part in the figure) and the unmatched first annotated frames C1, C2, and C3 are used as reference frames. The first annotated frames in the unmatched second annotated images are matched with the reference frames respectively. The specific matching process is the same as that of the annotated image 63, and stops when all the second annotated images are matched. This embodiment will not be repeated here.

[0102] In another example, determining a group of first annotation boxes corresponding to the same target object in at least two annotated images includes: performing intersection and union matching on the first annotation boxes in the at least two annotated images with a pre-stored reference box to obtain a group of first annotation boxes corresponding to the same target object.

[0103] The pre-stored reference frame corresponds to the target object one by one. Optionally, the pre-stored reference frame can be set by the reviewer.

[0104] Step 103 : comparing the first annotation frame information corresponding to a group of first annotation frames to obtain a comparison result.

[0105] In one example, comparing first annotation box information of a group of first annotation boxes to obtain a comparison result includes: obtaining annotation categories corresponding to the group of first annotation boxes; and when the annotation categories are consistent, determining that the comparison result indicates that the group of first annotation boxes are consistent.

[0106] The annotation category may be obtained when annotating the target image, or may be obtained by identifying image information in the first annotation frame.

[0107] Recognizing the image information in the first labeled frames includes: inputting the image information of the group of first labeled frames into a pre-trained category recognition model to obtain a label category corresponding to each first labeled frame. The category recognition model is obtained by training a preset neural network model using training data.

[0108] Optionally, the training data includes sample image information and sample labels corresponding to the sample image information. The sample labels are used to indicate the annotation categories corresponding to the sample image information.

[0109] Among them, the category recognition model includes but is not limited to: Deep Neural Network (DNN), Recurrent Neural Network (RNN) or Convolutional Neural Network (CNN), etc. This embodiment does not limit the type of category recognition model.

[0110] In another example, first annotation box information of a group of first annotation boxes is compared to obtain a comparison result, including: obtaining position information corresponding to a group of first annotation boxes; when the position information indicates that the distance between each first annotation box is less than a preset distance, determining that the comparison result indicates that the group of first annotation boxes is consistent.

[0111] Obtaining position information corresponding to a group of first annotation frames includes: obtaining coordinate information corresponding to the middle position of each frame of the group of first annotation frames in a coordinate system.

[0112] Take the number of a group of first annotation boxes as an example, the coordinate information of each border vertex of the two first annotation boxes is as follows: Figure 7As shown, at this time, the coordinates corresponding to the middle position of the border AD are (50, 120), the coordinates corresponding to the middle position of the border AB are (20, 85), the coordinates corresponding to the middle position of the border BC are (50, 50), and the coordinates corresponding to the middle position of the border CD are (80, 85). At this time, the coordinates corresponding to the middle position of the border A1D1 are (60, 120), the coordinates corresponding to the middle position of the border A1B1 are (30, 70), the coordinates corresponding to the middle position of the border B1C1 are (60, 40), and the coordinates corresponding to the middle position of the border C1D1 are (90, 70). Taking the preset distance of 20 as an example, at this time, the distance between the middle position of AD and the middle position of A1D1 is 10, the distance between the middle position of AB and the middle position of A1B1 is 18, the distance between the middle position of BC and the middle position of B1C1 is 14, and the distance between the middle position of CD and the middle position of C1D1 is 18. Therefore, the distance between each border is less than the preset distance. At this time, it is determined that the comparison result indicates that a group of first annotation frames are consistent.

[0113] In another example, first annotation box information of a group of first annotation boxes is compared to obtain a comparison result, including: obtaining the annotation categories corresponding to the group of first annotation boxes and the position information corresponding to the group of first annotation boxes; when the annotation categories are consistent and the position information indicates that the distance between each first annotation box is less than a preset distance, determining that the comparison result indicates that the group of first annotation boxes is consistent.

[0114] Since the set of labeled boxes matched by the matching degree is only a set of first labeled boxes preliminarily screened by the intersection-over-union ratio, it cannot guarantee that the set of first labeled boxes are consistent. To address the above technical issues, this embodiment compares the position information with the label category at the same time, so that the consistency of the set of first labeled boxes can be guaranteed.

[0115] Step 104 : When the comparison result indicates that the set of first annotation boxes are consistent, generate an annotation result of the target object based on the first annotation box information.

[0116] Optionally, the annotation result includes the position of the second annotation box corresponding to the target object and the annotation category of the target object.

[0117] Accordingly, generating a labeling result of the target object based on the first labeling box information includes: determining the position of the second labeling box based on the position information of each first labeling box in a group of first labeling boxes; obtaining the labeling category of the group of first labeling boxes to obtain the labeling category of the target object.

[0118] Determining the position of the second annotation frame includes: obtaining vertex coordinates of each first annotation frame; performing weighted averaging of the corresponding vertex coordinates to generate new vertex coordinates; and determining the position of the second annotation frame based on the new vertex coordinates.

[0119] Optionally, when the comparison result indicates that the set of first annotation boxes are inconsistent, a second prompt is output, where the second prompt is used to instruct the set of first annotation boxes to be reviewed.

[0120] Optionally, the second prompt may be highlighting a group of first annotation boxes or bolding a group of first annotation boxes. This embodiment does not limit the implementation method of the second prompt.

[0121] In addition, since annotation errors or erroneous annotations may occur during the annotation process, resulting in inconsistent comparison results, to address the above technical issues, when the comparison result indicates that a group of first annotation boxes is inconsistent, a second prompt is output, which is used to instruct the group of first annotation boxes to be reviewed. Therefore, the group of annotation boxes with inconsistent comparison results can be modified to generate a correct annotation result.

[0122] In summary, the image processing method provided in this embodiment obtains at least two annotated images obtained by annotating the same target image, each annotated image has a first annotated frame obtained by annotating one or more target objects in the target image; determines a group of first annotated frames corresponding to the target objects in the at least two annotated images; compares the first annotated frame information corresponding to a group of first annotated frames to obtain a comparison result; when the comparison result indicates that a group of first annotated frames are consistent, generates an annotation result of the target object based on the first annotated frame information; can solve the problem that different annotators may have different understandings of the same sample image, which may cause inconsistent image annotation results. In addition, when annotators face a large amount of data, there may be problems of missed or wrong labels; by generating the annotation result of the target object through the acquired annotated image, there is no need for manual participation in the annotation frame processing process, which can ensure the uniformity of the image annotation results and reduce the error rate of image annotation.

[0123] In addition, since missed labels may occur when annotating images, even if the first annotation box in one annotated image is correctly annotated, there may be a problem that the corresponding first annotation box cannot be matched in another annotated image. To address the above technical problem, the first annotation box whose matching degree does not meet the second preset requirement is also matched as a reference box. This can ensure the comprehensiveness of the matching, thereby improving the accuracy of determining a group of first annotation boxes corresponding to the same target object.

[0124] In addition, since there may be mislabeling when labeling, the matching degree of the first labeling frame in the second labeling image does not meet the first preset requirement. In view of the above technical problems, the first prompt is output in the case that the matching degree of the first labeling frame in the second labeling image does not meet the first preset requirement, so that the first labeling frame that does not match can be audited, thereby improving the accuracy of the first labeling frame labeling.

[0125] In addition, since the set of labeling frames matched by the matching degree is only a set of first labeling frames preliminarily screened by the intersection over union, it cannot be guaranteed that the set of first labeling frames is consistent. In view of the above technical problems, the embodiment compares the position information and the labeling category at the same time, so that the consistency degree of the set of first labeling frames can be guaranteed.

[0126] In addition, since there may be labeling errors or incorrect labeling in the labeling process, the comparison result is inconsistent. In view of the above technical problems, the second prompt is output in the case that the comparison result indicates that the set of first labeling frames is inconsistent, and the second prompt is used to indicate that the set of first labeling frames is audited, so that the set of labeling frames with inconsistent comparison results can be modified, thereby generating correct labeling results.

[0127] Figure 8 is a block diagram of an image processing apparatus provided by an embodiment of the present application, and the embodiment is described by taking the case that the apparatus is applied to an electronic device. The apparatus at least includes the following modules: an image acquisition module 810, a labeling frame determination module 820, a labeling frame comparison module 830, and a result generation module 840.

[0128] The image acquisition module 810 is configured to acquire at least two labeling images obtained by labeling the same target image, and each labeling image has a first labeling frame obtained by labeling one or more target objects in the target image.

[0129] The labeling frame determination module 820 is configured to determine a set of first labeling frames corresponding to the target objects in the at least two labeling images respectively.

[0130] The labeling frame comparison module 830 is configured to compare first labeling frame information corresponding to the set of first labeling frames to obtain a comparison result.

[0131] The result generation module 840 is configured to generate a labeling result of the target object based on the first labeling frame information in the case that the comparison result indicates that the set of first labeling frames is consistent.

[0132] For related details, refer to the above embodiments.

[0133] It should be noted that the image processing device provided in the above embodiments is merely illustrated by the division of the above functional modules when performing image processing. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the image processing device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing device provided in the above embodiments and the image processing method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0134] This embodiment provides an electronic device, such as Figure 9 As shown, the electronic device at least includes a processor 901 and a memory 902 .

[0135] The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0136] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction, which is executed by the processor 901 to implement the image processing method provided in the method embodiment of the present application.

[0137] In some embodiments, the electronic device can further optionally include a peripheral device interface and at least one peripheral device. The processor 901, the memory 902 and the peripheral device interface can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface through a bus, a signal line or a circuit board. Illustratively, the peripheral devices include, but are not limited to, a radio frequency circuit, a touch display screen, an audio circuit, a power supply and the like.

[0138] Of course, the electronic device can also include fewer or more components, and the present embodiment is not limited in this regard.

[0139] Optionally, the present application also provides a computer readable storage medium, the computer readable storage medium stores a program, the program is loaded and executed by a processor to implement the image processing method of the above method embodiment.

[0140] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, however, as long as the combinations of the technical features do not contradict each other, they should be considered within the scope of the present disclosure.

[0141] The above-described embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire at least two annotated images obtained by annotating the same target image, each annotated image having a first annotated frame obtained by annotating one or more target objects in the target image; Determining a group of first annotation frames corresponding to the target objects respectively in the at least two annotated images; Comparing the first annotation frame information corresponding to the group of first annotation frames to obtain a comparison result; If the comparison result indicates that the group of first annotation boxes are consistent, generating an annotation result of the target object based on the first annotation box information; Wherein, determining a group of first annotation frames corresponding to the same target object in the at least two annotated images includes: determining, based on a degree of matching between first annotation boxes in different annotated images, a group of first annotation boxes corresponding to the same target object in the different annotated images; The step of determining a group of first annotation boxes corresponding to the same target object in the different annotated images based on the matching degree between the first annotation boxes in the different annotated images includes: For a first annotated image of the at least two annotated images, determining a reference frame based on a first annotated frame of the first annotated image, each of the reference frames corresponding to one of the target objects; For a second annotated image other than the first annotated image in the at least two annotated images, matching a first annotated frame in the second annotated image with the reference frame to obtain a matching degree between the first annotated frame in the second annotated image and the reference frame; It is determined that the first annotation frame in the second annotation image, whose matching degree with the reference frame meets a first preset requirement, belongs to a group of first annotation frames of the target object corresponding to the reference frame.

2. The image processing method according to claim 1, wherein: The determining of a reference frame based on a first annotation frame of the first annotated image includes: When the matching degree of the first annotation frames in different first annotation images meets a second preset requirement, determining the reference frame based on the matched first annotation frames; or, When the degree of matching of the first annotation frames in different first annotation images meets the second preset requirement, a first matching frame is determined based on the matched first annotation frame, and the first matching frame is determined as the reference frame; when the degree of matching of the first annotation frames in different first annotation images does not meet the second preset requirement, a second matching frame is determined based on the unmatched first annotation frame, and the second matching frame is determined as the reference frame.

3. The image processing method according to claim 1, wherein: The determining of a reference frame based on a first annotation frame of the first annotated image includes: Initializing n first annotated images from the at least two annotated images, and determining a degree of matching between first annotated frames in the n first annotated images, where n is an integer greater than or equal to 2; Determining the reference frame based on the intersection of the first annotation frames in the n first annotated images whose matching degree meets the second preset requirement; After determining the first annotation frame in the set of first annotation frames from the second annotated image, the second annotated image is used as the first annotated image, and an intersection between the first annotation frame in the set of first annotation frames and the corresponding reference frame is determined to obtain an updated reference frame. The updated reference frame is used as the reference frame for next matching with the second annotated image.

4. The image processing method according to claim 1, wherein: The method further comprises: When the matching degree of the first annotation frame in the second annotation image does not meet the first preset requirement, a first prompt is output, where the first prompt is used to prompt a review of the unmatched first annotation frame.

5. The image processing method according to claim 1, wherein: The matching of the first annotation frame in the second annotated image with the reference frame to obtain a matching degree of the first annotation frame in the second annotated image includes: Determine the intersection-over-union ratio between the first annotated frame and each reference frame to obtain a matching degree of the first annotated frame in the second annotated image.

6. The image processing method according to any one of claims 1 to 5, characterized in that: The comparing the first markup box information of the group of first markup boxes to obtain a comparison result includes: Obtaining annotation categories corresponding to the group of first annotation boxes; and if the annotation categories are consistent, determining that the comparison result indicates that the group of first annotation boxes are consistent; and / or, Acquire position information corresponding to the group of first annotation frames; and determine, when the position information indicates that a distance between each first annotation frame is less than a preset distance, that the comparison result indicates that the group of first annotation frames is consistent.

7. The image processing method according to any one of claims 1 to 5, characterized in that: The annotation result includes the position of the second annotation box corresponding to the target object and the annotation category of the target object; Generating a labeling result of the target object based on the first labeling frame information includes: determining a position of the second markup box based on position information of each first markup box in the group of first markup boxes; The labeling categories of the group of first labeling frames are obtained to obtain the labeling category of the target object.

8. The image processing method according to any one of claims 1 to 5, characterized in that: The method further comprises: In a case where the comparison result indicates that the set of first annotation boxes are inconsistent, a second prompt is output, where the second prompt is used to instruct the set of first annotation boxes to be reviewed.

9. An electronic device, characterized in that: The device includes a processor and a memory; the memory stores a program, and the program is loaded and executed by the processor to implement the image processing method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The storage medium stores a program, and when the program is executed by the processor, it is used to implement the image processing method according to any one of claims 1 to 8.

Images