Method, system, medium and electronic device for manual image annotation processing

Abstract

The application provides a kind of manual image annotation processing method, system, medium and electronic equipment, comprising: the location of image target is obtained and different thickness degree is set for the annotation frame according to the area size of target;After the target in the image is annotated, the pixel points in the annotation frame and the pixel points on the boundary line of the annotation frame are respectively set different confidence levels.The present application has different annotation frame thickness degree according to the target of different area size, and the target with larger area has thicker annotation frame, which can more accurately surround the target and reduce the error caused by manual annotation.The pixel points on the boundary line of the annotation frame are set to a lower confidence level to reduce the impact of pixel points not belonging to the target, improve the learning effect of the image segmentation algorithm based on deep learning, and thus improve the recognition accuracy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method, system, medium, and electronic device for manual image annotation processing. More particularly, it relates to a method, storage medium, and electronic device for improving the robustness of manual image annotation. Background Technology

[0002] In recent years, the development of deep learning has greatly promoted the progress of artificial intelligence-related algorithms, including those in computer vision and natural language processing. The datasets required for deep learning algorithms are generally divided into training and testing subsets, used independently. The algorithm uses the training subset during the training phase, and after training, an algorithm model is generated. To verify the model's learning effect, it needs to be tested on the testing subset to calculate the accuracy. Deep learning has many learning methods. In supervised learning, deep learning algorithms need to label the dataset with real values. During the training phase, these real labels serve as supervisory information for the model's learning. During the testing phase, these real labels are used in conjunction with the model's predictions to calculate the accuracy and evaluate the model's learning ability.

[0003] Taking pixel-level segmentation and annotation of equipment defects in industrial scenarios as an example, when manually annotating a large number of pixels in an image, annotation tools such as LabelMe are generally used to annotate the target with polygons. This may result in inaccurate bounding boxes, causing annotation errors, usually because the boundary lines of the bounding boxes cannot completely surround the target. Inaccurate ground truth labels directly affect the model's learning ability and the test results of the model.

[0004] Therefore, there is an urgent need to develop a method to improve the robustness of manual image annotation. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the purpose of this invention is to provide a method, system, medium and electronic device for manual image annotation processing.

[0006] A method for manual image annotation processing provided by the present invention includes:

[0007] Step S1: Obtain the location of the target in the image and set different thicknesses for the bounding box according to the area occupied by the target;

[0008] Step S2: After labeling the target in the image, set different confidence levels for the pixels inside the label box and the pixels on the boundary line of the label box.

[0009] Preferably, step S1 includes: the larger the target area to be labeled, the thicker the boundary line is set for the labeling box; and the smaller the target area, the thinner the boundary line is set for the labeling box.

[0010] Preferably, if the pixels on the boundary line contain both pixels that do not belong to the target and pixels that belong to the target, then based on the difficulty of outlining the target contour and the uncertainty of the target edge, a higher confidence level is set for the pixels within the annotation box, and a lower confidence level is set for the pixels on the boundary line of the annotation box.

[0011] Preferably, the difficulty of outlining the target contour is determined based on the degree and number of concavities and convexities of the target contour;

[0012] The uncertainty of the target edge is determined based on the similarity between the outside and inside of the target edge.

[0013] A system for manual image annotation processing according to the present invention includes:

[0014] Module M1: Obtains the location of the target in the image and sets different thicknesses for the bounding box according to the area occupied by the target;

[0015] Module M2: After labeling the target in the image, different confidence levels are set for the pixels inside the label box and the pixels on the boundary line of the label box.

[0016] Preferably, module M1 includes: the larger the target area to be labeled, the thicker the boundary line is set for the labeling box; and the smaller the target area, the thinner the boundary line is set for the labeling box.

[0017] Preferably, if the pixels on the boundary line contain both pixels that do not belong to the target and pixels that belong to the target, then based on the difficulty of outlining the target contour and the uncertainty of the target edge, a higher confidence level is set for the pixels within the annotation box, and a lower confidence level is set for the pixels on the boundary line of the annotation box.

[0018] Preferably, the difficulty of outlining the target contour is determined based on the degree and number of concavities and convexities of the target contour;

[0019] The uncertainty of the target edge is determined based on the similarity between the outside and inside of the target edge.

[0020] According to the present invention, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the steps of the manual image annotation processing method are implemented.

[0021] An electronic device for manual image annotation processing according to the present invention includes: at least one processor electrically connected to the storage medium, for executing a computer program stored in the storage medium to implement the manual image annotation processing method as described in the present invention.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. In this invention, different thicknesses of the annotation frame are used for targets of different sizes. Larger targets have thicker annotation frames, which can more accurately surround the target and reduce errors caused by manual annotation.

[0024] 2. In this invention, the pixels on the boundary line of the bounding box are set with a lower confidence level to reduce the influence of pixels that do not belong to the target, improve the learning effect of the deep learning-based image segmentation algorithm, and thus improve the recognition accuracy. Attached Figure Description

[0025] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0026] Figure 1 This is a schematic diagram of the process of the present invention.

[0027] Figure 2 This is a schematic diagram of a fine-lined frame for a device defect in an industrial setting, provided in an embodiment of the present invention.

[0028] Figure 3 This is a flowchart illustrating equipment defect labeling in an industrial setting, as described in this invention.

[0029] Figure 4 This is a schematic diagram illustrating the confidence level settings for pixels within the annotation box and on the boundary line in this invention.

[0030] Figure 5 This is a schematic diagram of the electronic device in this invention. Detailed Implementation

[0031] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0032] This invention sets different thicknesses for the annotation boxes based on the area occupied by the target. Furthermore, considering the difficulty of outlining the target contour and the uncertainty of the target edges, it assigns higher confidence scores to pixels within the annotation boxes and lower confidence scores to pixels on the boundary lines, thus including pixels on the boundary lines in the annotation range. This solves the problems of inaccurate annotation and incomplete target enclosure in manual image segmentation and annotation, thereby improving the learning effect of deep learning algorithms and ultimately increasing recognition accuracy.

[0033] Example 1

[0034] According to the present invention, a method for manual image annotation processing is provided, such as... Figure 1 As shown, it includes:

[0035] Step S1: Obtain the location of the target in the image and set different thicknesses for the bounding boxes based on the area occupied by the target. The target includes equipment defects that need to be labeled in an industrial scene, such as... Figure 2 As shown, an image can contain one or more device defects.

[0036] Additionally, image bounding boxes are generated based on image annotation information. This annotation information includes the annotation information of the dataset images used by deep learning-based object detection or image segmentation algorithms. The bounding boxes can be rectangles or polygons. The image annotation information is the positional information of the image bounding boxes. Specifically, the positional information of the bounding boxes can include the coordinates of each annotation point, and the bounding box is formed by these annotation points closing out from end to end. For example... Figure 3 As shown, the vertices of the polygonal annotation box are the annotation points.

[0037] Specifically, step S1 includes: the larger the target area to be labeled, the thicker the boundary line of the label box is set; conversely, the smaller the target area, the thinner the boundary line is set. This allows for more precise enclosure of the target, reducing labeling errors. It is worth noting that regardless of the size of the target area, the minimum width of the label box boundary line is 1 pixel. Because different thicknesses are set for the label box based on the target area, the boundary line width value of the label box needs to be additionally included in the image annotation information.

[0038] It should be noted that those skilled in the art will understand that the thickness of the annotation box can be set as needed in practical applications. For example, the width of the annotation box boundary line can be between 1% and 5% of the target area. This invention does not limit the specific method of setting the thickness of the annotation box.

[0039] Step S2: As Figure 4As shown, after labeling the target in the image, different confidence levels are assigned to pixels within the label box and pixels on the boundary line of the label box. Specifically, pixels on the boundary line may contain both pixels that do not belong to the target and pixels that do belong to the target. Therefore, if pixels on the boundary line contain both pixels that do not belong to the target and pixels that do belong to the target, then based on the difficulty of outlining the target contour and the uncertainty of the target edge, a higher confidence level is assigned to pixels within the label box, and a lower confidence level is assigned to pixels on the boundary line of the label box.

[0040] The difficulty of outlining the target contour depends on the degree and number of protrusions and concavities in the contour. The greater the degree and number of protrusions in the contour, the more difficult it is to outline. Conversely, the less the degree and number of protrusions, the easier it is to outline.

[0041] The uncertainty of a target edge is determined by the similarity between the outer and inner parts of the target edge. The higher the similarity between the outer and inner parts of the target edge, the greater the uncertainty of the target edge; conversely, the lower the similarity between the outer and inner parts of the target edge, the smaller the uncertainty of the target edge.

[0042] It should be noted that those skilled in the art will understand that in practical applications, the confidence level can be set as needed. Provided that the confidence level of the pixels within the annotation box is greater than the confidence level of the pixels on the boundary line of the annotation box, the confidence level can be set to any value in the range of 0 to 1. This invention does not limit the range of confidence level values.

[0043] Example 2

[0044] The present invention also provides a system for manual image annotation processing. Those skilled in the art can implement the system by executing the steps of the manual image annotation processing method. That is, the manual image annotation processing method can be understood as a preferred embodiment of the system.

[0045] A system for manual image annotation processing according to the present invention includes:

[0046] Module M1: Acquires the location of the target in the image and sets different thicknesses for the bounding box based on the area occupied by the target. Module M1 includes: the larger the area of ​​the target to be labeled, the thicker the boundary line of the bounding box is set, and the smaller the area of ​​the target, the thinner the boundary line of the bounding box is set.

[0047] Module M2: After labeling the target in the image, different confidence levels are assigned to pixels within the labeling box and pixels on the boundary line of the labeling box. If pixels on the boundary line contain both pixels that do not belong to the target and pixels that do belong to the target, then based on the difficulty of outlining the target contour and the uncertainty of the target edge, a higher confidence level is assigned to pixels within the labeling box, and a lower confidence level is assigned to pixels on the boundary line. The difficulty of outlining the target contour is determined based on the degree and number of concavities and convexities in the target contour; the uncertainty of the target edge is determined based on the similarity between the external and internal parts of the target edge.

[0048] According to the present invention, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the steps of the manual image annotation processing method are implemented.

[0049] An electronic device for manual image annotation processing provided by the present invention, such as Figure 5 As shown, it includes: at least one processor electrically connected to the storage medium, for executing a computer program stored in the storage medium to implement the manual image annotation processing method as described in this invention.

[0050] Electronic devices can be computer devices, including but not limited to: personal computers, servers, general-purpose computers, special-purpose computers, network equipment, embedded devices, programmable devices, smart mobile terminals, smart home devices, wearable smart devices, and in-vehicle smart devices. Storage media include but are not limited to: RAM, ROM, magnetic disks, magnetic tapes, optical discs, flash memory, USB flash drives, external hard drives, memory cards, memory sticks, network server storage, and network cloud storage. Processors include but are not limited to CPUs (Central Processing Units), GPUs (Graphics Processing Units), and MCUs (Microprocessors).

[0051] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0052] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for manual image annotation processing, characterized in that, include: Step S1: Obtain the location of the target in the image and set different thicknesses for the bounding box according to the area occupied by the target; Step S2: After labeling the targets in the image, set different confidence levels for the pixels inside the label box and the pixels on the boundary line of the label box. If the pixels on the boundary line contain both pixels that do not belong to the target and pixels that belong to the target, then based on the difficulty of outlining the target contour and the uncertainty of the target edge, a higher confidence level is set for the pixels inside the annotation box, and a lower confidence level is set for the pixels on the boundary line of the annotation box. The difficulty of outlining the target contour is determined by the degree and number of concavities and convexities in the target contour. The uncertainty of the target edge is determined based on the similarity between the outside and inside of the target edge.

2. The method for manual image annotation processing according to claim 1, characterized in that, Step S1 includes: the larger the target area to be labeled, the thicker the boundary line is set for the label box, and the smaller the target area, the thinner the boundary line is set for the label box.

3. A system for manual image annotation processing, characterized in that, include: Module M1: Obtains the location of the target in the image and sets different thicknesses for the bounding box according to the area occupied by the target; Module M2: After labeling the target in the image, different confidence levels are set for the pixels inside the label box and the pixels on the boundary line of the label box. If the pixels on the boundary line contain both pixels that do not belong to the target and pixels that belong to the target, then based on the difficulty of outlining the target contour and the uncertainty of the target edge, a higher confidence level is set for the pixels inside the annotation box, and a lower confidence level is set for the pixels on the boundary line of the annotation box. The difficulty of outlining the target contour is determined by the degree and number of concavities and convexities in the target contour. The uncertainty of the target edge is determined based on the similarity between the outside and inside of the target edge.

4. The system for manual image annotation processing according to claim 3, characterized in that, Module M1 includes: the larger the target area to be labeled, the thicker the boundary line is set for the label box; the smaller the target area, the thinner the boundary line is set for the label box.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the manual image annotation processing method according to any one of claims 1 to 2.

6. An electronic device for manual image annotation processing, comprising: At least one processor, electrically connected to the storage medium of claim 5, is configured to execute a computer program stored in the storage medium to implement the manual image annotation processing method as described in claims 1 to 2 of the present invention.

Images