Industrial labeling method and device
By introducing an industrial annotation method and device in the field of industrial testing, semi-automatic annotation is used to use user input to solve the problem of time-consuming and labor-consuming manual annotation, and the labeling efficiency and quality are improved.
Patent Information
- Application Number
- CN202411504061.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-25
- Publication Date
- 2025-03-14
AI Technical Summary
In the field of industrial testing, the prior art method of manually labeling data sets by manual and manual labeling results in the processing of large-scale data consuming a lot of time and labor costs, which leads to slow project progress and reduces labeling quality.
An industrial annotation method and device are proposed. By receiving various forms of input from the target area to which the user is to mark the image, annotating the feature images associated with the target area based on the user input, and adopting semi-automatic annotation method to reduce the threshold for use and save labor and time costs.
Semi-automatic annotation of images is realized, which lowers the threshold for use, saves a lot of manpower and time costs, and improves the labeling efficiency and quality.
Smart Images
Figure CN119625449A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of industrial detection technology, and in particular, relates to an industrial labeling method and device. Background Art
[0002] In the field of industrial inspection, most tasks are performed based on deep learning models, where deep learning tasks require the labeling of large data sets. In the related art, there is a method of manually labeling data sets. This method takes a lot of time and manpower costs when processing large-scale data, resulting in slow project progress. In addition, long-term repetitive work often makes labelers feel tired, which easily leads to a decline in labeling quality. Summary of the invention
[0003] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes an industrial labeling method and device, which lowers the threshold for use, brings greater convenience to the labeling of staff, saves a lot of manpower and time costs, and improves the labeling efficiency and labeling quality.
[0004] In a first aspect, the present application provides an industrial labeling method, the method comprising:
[0005] Receive a first input from a user regarding a target area of the image to be annotated; the first input includes at least one of a box selection input and a click input;
[0006] In response to the first input, annotating a feature image associated with the target area to obtain an annotated image;
[0007] The annotated image is output, where the annotated image includes image features and annotation information of the image to be annotated.
[0008] According to the industrial labeling method provided in the embodiment of the present application, by receiving various forms of user input on the target area in the image to be labeled, the feature image associated with the target area can be labeled based on the user input to obtain a labeled image. The labeling is performed in a manner based on prompt points and boxes, and features of different morphological types can be labeled, which has a wide range of applicable scenarios. It also realizes semi-automatic labeling of images, lowers the usage threshold, brings great convenience to the staff's labeling, saves a lot of manpower and time costs, and improves labeling efficiency and quality.
[0009] In an industrial labeling method according to an embodiment of the present application, the first input includes a plurality of sub-inputs for a plurality of different target areas; in response to the first input, labeling a feature image associated with the target area to obtain a labeled image includes:
[0010] In response to the multiple sub-inputs, based on the input order of the sub-inputs, the feature images associated with the sub-inputs are annotated in sequence to obtain the annotated images.
[0011] An industrial labeling method according to an embodiment of the present application, in response to the multiple sub-inputs, sequentially labeling the feature images associated with the sub-inputs based on the input order of the sub-inputs to obtain the labeled image, includes:
[0012] In response to a previous sub-input among the multiple sub-inputs, annotating a feature image associated with the previous sub-input to obtain a first annotated image corresponding to the previous sub-input;
[0013] In response to a latter sub-input among the multiple sub-inputs, a feature image associated with the latter sub-input is annotated on the first annotated image to obtain the annotated image.
[0014] In an industrial labeling method according to an embodiment of the present application, labeling the feature image associated with the target area to obtain a labeled image includes:
[0015] The feature image associated with the target area is annotated based on a target annotation model to obtain the annotated image output by the target annotation model; the target annotation model includes a lightweight encoder and a decoder, and the lightweight encoder is obtained by distillation training based on an industrial dataset.
[0016] In an industrial labeling method according to an embodiment of the present application, labeling the feature image associated with the target area to obtain a labeled image includes:
[0017] Extracting the contour of the feature image;
[0018] The feature image is annotated based on the contour to obtain the annotated image.
[0019] In an industrial labeling method according to an embodiment of the present application, after labeling the feature image based on the contour to obtain the labeled image, the method further includes:
[0020] Calculate the area of the region within the outline based on the outline;
[0021] When the area of the region is smaller than a first area threshold, deleting the annotation information of the feature image corresponding to the contour;
[0022] When the area of the region is not less than a second area threshold, the annotation information of the region exceeding the second area threshold is deleted; the second area threshold is greater than the first area threshold.
[0023] In a second aspect, the present application provides an industrial labeling device, comprising:
[0024] A first processing module, configured to receive a first input from a user of a target area of an image to be annotated; the first input includes at least one of a box selection input and a click input;
[0025] A second processing module, configured to annotate the feature image associated with the target area in response to the first input to obtain an annotated image;
[0026] The third processing module is used to output the annotated image, where the annotated image includes image features and annotation information of the image to be annotated.
[0027] According to the industrial labeling device provided in the embodiment of the present application, by receiving various forms of user input on the target area in the image to be labeled, the feature image associated with the target area can be labeled based on the user input to obtain a labeled image. The labeling is performed in a manner based on prompt points and boxes, and features of different morphological types can be labeled, which has a wide range of applicable scenarios. It also realizes semi-automatic labeling of images, lowers the usage threshold, brings great convenience to the staff's labeling, saves a lot of manpower and time costs, and improves labeling efficiency and labeling quality.
[0028] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the industrial labeling method as described in the first aspect above is implemented.
[0029] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the industrial labeling method as described in the first aspect above.
[0030] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the industrial labeling method as described in the first aspect above.
[0031] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:
[0032] By receiving various forms of user input on the target area in the image to be annotated, the feature image associated with the target area can be annotated based on the user input to obtain the annotated image. The annotation is performed in a way based on prompt points and boxes, and features of different morphological types can be annotated, which has a wide range of applicable scenarios. It also realizes semi-automatic annotation of images, lowers the usage threshold, brings great convenience to the annotation of staff, saves a lot of manpower and time costs, and improves the annotation efficiency and quality.
[0033] Furthermore, the annotation results are obtained through the user's input, and the user is allowed to continue annotating based on the previous result, so that the user can gradually annotate multiple target areas in the image step by step and in sequence, so that the user can interact in the software interface in real time, which improves the flexibility of annotation and ensures the accuracy and consistency of the annotation process.
[0034] Furthermore, a lightweight encoder is obtained through distillation training on industrial datasets, and then a target labeling model for industrial labeling is obtained, so that the target labeling model can be deployed on mobile and PC terminals, broadening the applicable scenarios; and the lightweight encoder is more suitable for scenarios in industrial datasets while ensuring model performance, improving the labeling accuracy and precision of the target labeling model in industrial scenarios.
[0035] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0037] Figure 1 This is one of the flow diagrams of the industrial labeling method provided in the embodiment of the present application;
[0038] Figure 2 This is one of the principle schematic diagrams of the industrial labeling method provided in the embodiment of the present application;
[0039] Figure 3 This is the second principle schematic diagram of the industrial labeling method provided in the embodiment of the present application;
[0040] Figure 4 This is the third principle schematic diagram of the industrial labeling method provided in the embodiment of the present application;
[0041] Figure 5 This is the fourth principle schematic diagram of the industrial labeling method provided in the embodiment of the present application;
[0042] Figure 6This is the fifth principle schematic diagram of the industrial labeling method provided in the embodiment of the present application;
[0043] Figure 7 This is the sixth principle schematic diagram of the industrial labeling method provided in the embodiment of the present application;
[0044] Figure 8 This is the seventh principle schematic diagram of the industrial labeling method provided in the embodiment of the present application;
[0045] Fig. 9 This is the second flow chart of the industrial labeling method provided in the embodiment of the present application;
[0046] Fig.10 This is the eighth schematic diagram of the principle of the industrial labeling method provided in the embodiment of the present application;
[0047] Fig.11 This is the ninth principle schematic diagram of the industrial labeling method provided in the embodiment of the present application;
[0048] Fig.12 This is the tenth principle schematic diagram of the industrial labeling method provided in the embodiment of the present application;
[0049] Fig.13 This is the eleventh principle schematic diagram of the industrial labeling method provided in the embodiment of the present application;
[0050] Fig.14 is a schematic diagram of the structure of an industrial labeling device provided in an embodiment of the present application;
[0051] Fig.15 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0053] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0054] The industrial labeling method, industrial labeling device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0055] Among them, the industrial labeling method can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.
[0056] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).
[0057] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.
[0058] The industrial labeling method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the industrial labeling method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The industrial labeling method provided in the embodiment of the present application is described below using an electronic device as an example of the execution subject.
[0059] like Figure 1 As shown, the industrial labeling method includes: step 110, step 120 and step 130.
[0060] The industrial labeling method can be applied to the field of industrial inspection. For example, when detecting industrial defects, the industrial defects can be labeled based on the industrial labeling method to obtain a large number of labeled data sets.
[0061] Step 110: receiving a first input from a user regarding a target area of an image to be annotated;
[0062] In this step, the image to be annotated includes features to be annotated. For example, the image to be annotated may include defect features to be annotated.
[0063] like Figure 2 , Figure 3 and Figure 4 The following example shows an image to be labeled that contains multiple types of defects.
[0064] The target area is the feature area that needs to be annotated in the image to be annotated.
[0065] The first input includes at least one of a box selection input and a click input.
[0066] Users can select the target area in the image by drawing a rectangle, circle, or other shaped box.
[0067] Alternatively, the user can indicate the location of the target area by clicking on a specific point in the image.
[0068] For example, when the user uses the mouse to input clicks, when the user clicks the left mouse button, the user's positive single-click input can be received, and by identifying the positive single-click input, it can be determined that the area corresponding to the input is the foreground area; when the user clicks the right mouse button (or double-clicks, etc.), the user's negative single-click input can be received, and by identifying the negative single-click input, it can be determined that the area corresponding to the input is the background area.
[0069] In some embodiments, the first input may be in at least one of the following ways:
[0070] First, the first input may be a touch operation, including but not limited to a click operation, a slide operation, and a press operation.
[0071] In this implementation, receiving the first input from the user may be receiving a touch operation of the user in the display area of the terminal display screen.
[0072] In order to reduce the user error operation rate, the effective area of the first input can be limited to a specific area, such as the upper middle area of the interface of the image to be annotated; or when the interface of the image to be annotated is displayed, the target control is displayed on the current interface, and the first input can be implemented by touching the target control; or the first input can be set to a continuous multiple tapping operation on the display area within a target time interval.
[0073] Secondly, the first input may be a physical key input.
[0074] In this embodiment, a physical button corresponding to the target area in the image to be annotated is provided on the terminal body, and receiving the user's first input can be an operation of receiving the user pressing the corresponding physical button; the first input can also be a combined operation of pressing multiple physical buttons at the same time.
[0075] Third, the first input may be voice input.
[0076] In this implementation, when the terminal receives a voice message such as "mark the apple in the image", it can trigger the display of an interface for marking the apple.
[0077] Of course, in other embodiments, the first input may also be in other forms, including but not limited to character input, etc., which can be determined according to actual needs and is not limited in this embodiment of the present application.
[0078] For example, the user can provide a mask image of the same size as the image, where each pixel is labeled as belonging to foreground, background, or another class.
[0079] Mask hints can be used to provide precise spatial information.
[0080] Step 120: in response to the first input, annotate the feature image associated with the target area to obtain an annotated image;
[0081] In this step, according to the first input of the user, the system can identify the target area and mark the target area.
[0082] For example, you can draw a bounding box on the target area, fill it with color, add a text label, or modify certain properties of the image (such as brightness or contrast) to highlight the target area.
[0083] The feature image associated with the target area is the image framed by the target area in the image to be annotated.
[0084] The feature image includes feature information in the image to be annotated. For example, the feature image may include defect information in the image to be annotated.
[0085] The labeled image is an image that has been labeled.
[0086] The annotated image includes the image features of the image to be annotated and the annotation information added by the user.
[0087] The annotation information may include a bounding box, a polygon, a point, a text, or a mask.
[0088] The annotation information may also include category labels and location information of objects in the target area.
[0089] Step 130: Output the annotated image.
[0090] In this step, the annotated image may be displayed on a screen; or the annotated image may be saved in a file system.
[0091] In some embodiments, the annotation information may be output in the form of structured data (such as JSON or XML, etc.).
[0092] According to the industrial labeling method provided in the embodiment of the present application, by receiving various forms of user input on the target area in the image to be labeled, the feature image associated with the target area can be labeled based on the user input to obtain a labeled image. The labeling is performed in a manner based on prompt points and boxes, and features of different morphological types can be labeled, which has a wide range of applicable scenarios. It also realizes semi-automatic labeling of images, lowers the usage threshold, brings great convenience to the staff's labeling, saves a lot of manpower and time costs, and improves labeling efficiency and quality.
[0093] In some embodiments, step 120 may include:
[0094] In response to the multiple sub-inputs, based on the input order of the sub-inputs, the feature images associated with the sub-inputs are labeled in sequence to obtain labeled images.
[0095] In this embodiment, the first input includes a plurality of sub-inputs for a plurality of different target areas.
[0096] The user can input multiple times, each input targeting a specific target area in the image to be annotated.
[0097] During the labeling process, labeling can be performed based on the input order of each sub-input.
[0098] That is, the target area entered first will be marked first, and the target area entered later will be marked afterwards.
[0099] Based on the order of the sub-inputs, the feature image associated with each sub-input will be annotated to obtain an annotated image.
[0100] In some embodiments, in response to multiple sub-inputs, based on the input order of each sub-input, sequentially annotating feature images associated with the sub-inputs to obtain an annotated image may include:
[0101] In response to a previous sub-input among the multiple sub-inputs, annotating a feature image associated with the previous sub-input to obtain a first annotated image corresponding to the previous sub-input;
[0102] In response to a latter sub-input among the multiple sub-inputs, a feature image associated with the latter sub-input is annotated on the first annotated image to obtain an annotated image.
[0103] In this embodiment, the previous sub-input refers to the input that has been received and is ready to be marked.
[0104] The subsequent sub-input is an input received after the previous sub-input.
[0105] After receiving the previous sub-input, the feature image associated with the sub-input is first annotated to generate a preliminary annotated image, namely the first annotated image.
[0106] When the next sub-input is received, the system will continue to annotate the feature image associated with the next sub-input on the existing first annotated image. This process will be repeated until all sub-inputs have been processed and a complete annotated image is finally generated.
[0107] In the actual implementation process, Figure 5 As shown, the previous sub-input is received, for example, the user's selection of the target area can be received; Figure 6 As shown, in response to the previous sub-input, the first annotation corresponding to the image to be annotated can be obtained, and the first annotated image can be output. For example, the red area can be used to cover the object to be annotated in the image to be annotated; Figure 7 As shown, receiving the user's subsequent sub-input for the first annotated image; as shown Figure 8 As shown, the second annotation corresponding to the image to be annotated can be obtained, and the annotated image can be output.
[0108] According to the industrial labeling method provided in the embodiment of the present application, the labeling result is obtained through the user's input, and the user is allowed to continue labeling based on the previous result, so that the user can gradually label multiple target areas in the image step by step and in sequence, so that the user can interact in the software interface in real time, which improves the flexibility of labeling and ensures the accuracy and consistency of the labeling process.
[0109] In some embodiments, labeling the feature image associated with the target area to obtain the labeled image may include:
[0110] The feature image associated with the target area is annotated based on the target annotation model to obtain an annotated image output by the target annotation model.
[0111] In this embodiment, the target annotation model is a machine learning model for image annotation, which receives a feature image associated with a target region as input and outputs an annotated image.
[0112] The object annotation model may include an image encoder, a prompt encoder, and a mask decoder.
[0113] Among them, the image encoder uses the ViT model to convert the input image into a high-dimensional and information-rich representation (i.e., encoding), which can capture the key features of the image.
[0114] Visual Transformer (ViT) is an image processing model based on the Transformer architecture. ViT can divide an image into a series of small patches, and then regard these small patches as "words" (tokens) in the sequence, and learn the global and local features of the image through the self-attention mechanism.
[0115] The prompt encoder can convert the input prompt (e.g., natural language text) into an encoded form compatible with the image encoder output.
[0116] Hint The encoder can be a lightweight encoder.
[0117] The lightweight encoder can be obtained by distillation training based on industrial datasets, and the lightweight encoder is a mask decoder.
[0118] The encoder is used to extract useful information or features from the input feature image.
[0119] The model’s prompt encoder can be adjusted to support multiple types of prompt inputs.
[0120] Knowledge distillation can be performed on the basis of the large industrial model, so that the target labeling model can maintain the segmentation ability of the large industrial model.
[0121] For example, the knowledge distillation method can be used on the model encoder to generate a student model. During this period, the acquired industrial data set is used for distillation training to enable it to learn the basic segmentation capabilities of the original model's large model and adapt it to the industrial data set scenario. The student model is the result of knowledge distillation, and a lighter model encoder can be obtained.
[0122] The parameter size of the lightweight encoder is 10M. The obtained lightweight encoder is coupled and compatible with the decoder of the industrial large model to obtain the target annotation model.
[0123] The mask decoder can predict and decode the labels or attributes of these masked areas based on the output of the image encoder and prompt encoder, as well as the context information. The results generated by the mask decoder can be used to fill the mask area to complete the image annotation.
[0124] According to the industrial labeling method provided in the embodiment of the present application, a lightweight encoder is obtained by performing distillation training on an industrial data set, and then a target labeling model for industrial labeling is obtained, so that the target labeling model can be deployed on mobile terminals and PC terminals, thereby broadening the applicable scenarios; and the lightweight encoder is more suitable for scenarios in industrial data sets while ensuring model performance, thereby improving the labeling accuracy and precision of the target labeling model in industrial scenarios.
[0125] In some embodiments, labeling the feature image associated with the target area to obtain the labeled image may include:
[0126] Extract the contour of the feature image;
[0127] The feature image is annotated based on the contour to obtain an annotated image.
[0128] In this embodiment, the contour is a collection of object boundaries in an image, and the contour is used to represent shape information of the labeled object.
[0129] By extracting the contour of the feature image, the target area can be located and labeled.
[0130] Feature images can be annotated based on contours, for example, a boundary line can be drawn around the contour, a color can be filled, or other visual markers can be applied to highlight the target area.
[0131] In some embodiments, after annotating the feature image based on the contour to obtain the annotated image, the industrial annotation method may further include:
[0132] Calculate the area of the region within the contour based on the contour;
[0133] When the area of the region is smaller than the first area threshold, deleting the annotation information of the feature image corresponding to the contour;
[0134] When the area of the region is not less than the second area threshold, the annotation information of the region exceeding the second area threshold is deleted.
[0135] In this embodiment, the area of the region may be obtained by counting the number of pixels within the contour (or other measurement methods).
[0136] When the area of a region within a certain contour is smaller than the first area threshold, it can be considered that this region may not be a valid target region (for example, it may be noise or false detection), and the annotation information of the feature image corresponding to the contour can be deleted.
[0137] When the area of a region within a certain contour is not less than the second area threshold (and the second area threshold is greater than the first area threshold), it can be considered that this region may contain too much non-target information (for example, background or other irrelevant objects), and the annotation information of the partial area exceeding the second area threshold can be deleted to retain the part most relevant to the target area.
[0138] In the actual implementation process, Fig.10 As shown, when the area of the region is smaller than the first area threshold, the annotation information of the feature image corresponding to the contour can be deleted by hole filling.
[0139] The threshold for hole filling (AreaThresholdReal) is dynamically adjusted by combining the manually set area threshold lower limit (AreaThreshold) and the automatically calculated percentage of the current maximum contour area (MaxContousArea):
[0140] AreaThresholdReal=Max(AreaThreshold,MaxContousArea / 100)
[0141] Among them, MaxContousArea is the area of the contour with the largest area among all contours.
[0142] When the defect area is larger, the threshold for filling the internal holes also increases.
[0143] The defect contour can also be corroded and expanded to make the defect edge smoother.
[0144] When there are discrete small contours around the contour (for example, the area corresponding to the contour is smaller than the set threshold), it can be filtered out.
[0145] When the area of the region is not less than the second area threshold, the annotation information of the region exceeding the second area threshold can be deleted by edge clipping.
[0146] When inputting and annotating a rectangular box, if the outline of the output result exceeds the range of the rectangular box, the area that exceeds the rectangular box more can be cropped.
[0147] like Fig.11 As shown, by outputting the annotation result image of the target annotation model ( Fig.10 ) is corrected to obtain the corrected annotation result image.
[0148] like Fig. 9 As shown, if the annotation effect does not meet your expectations, you can use the brush to expand the mask area, and you can also use the eraser tool to erase the area.
[0149] The target annotation model outputs the annotation results in the form of a mask. For large-area defects, the mask can also be converted into a polygonal annotation, such as Fig.12 shown.
[0150] You can also set the outline polygon fit to get a suitable polygon annotation, such as Fig.13 As shown, the annotation can be adjusted by adding, deleting or dragging polygon outline points.
[0151] In the actual implementation process, Fig. 9As shown, you can load the weight file of the pre-trained target annotation model. After the software is started, you only need to load it once.
[0152] Then load the images to be annotated. You can load them in the form of folders. Load multiple images to be annotated into the image list and switch to load annotations according to the order in which the images are arranged.
[0153] The image to be annotated can be displayed in the software image interface, and the user can mark the defect location with prompt points or prompt boxes.
[0154] Then, the image to be annotated and the prompt information input by the user can be input into the target annotation model to obtain the annotated image output by the target annotation model.
[0155] The annotated image may be processed, for example, small holes inside the annotated area may be removed; the edge of the annotated area may be smoothed; or the annotated area may be expanded to include some background information.
[0156] SuperAI can be used to load and display the annotation results. SAM can be integrated into the annotation software SuperAI. Users only need to click a point or draw a bounding box to generate a mask for the object, thereby reducing the amount of manual operation by the user.
[0157] If the annotation result meets expectations, the annotated image is saved directly, and the next image to be annotated is annotated; if the annotation result does not meet expectations, the annotation area can be adjusted using a brush or eraser according to the annotation situation. After the adjustment is completed, the annotated image is saved, and the next image to be annotated is annotated.
[0158] The industrial labeling device provided in the present application is described below. The industrial labeling device described below and the industrial labeling method described above can be referenced to each other.
[0159] The industrial labeling method provided in the embodiment of the present application can be executed by an industrial labeling device. In the embodiment of the present application, an industrial labeling device executing the industrial labeling method is taken as an example to illustrate the industrial labeling device provided in the embodiment of the present application.
[0160] The embodiment of the present application also provides an industrial labeling device.
[0161] like Fig.14 As shown, the industrial labeling device includes: a first processing module 1410 , a second processing module 1420 and a third processing module 1430 .
[0162] The first processing module 1410 is used to receive a first input from a user of a target area to be annotated in the image; the first input includes at least one of a box selection input and a click input;
[0163] The second processing module 1420 is used to annotate the feature image associated with the target area in response to the first input to obtain an annotated image;
[0164] The third processing module 1430 is used to output a labeled image, where the labeled image includes image features of the image to be labeled and labeling information.
[0165] According to the industrial labeling device provided in the embodiment of the present application, by receiving various forms of user input on the target area in the image to be labeled, the feature image associated with the target area can be labeled based on the user input to obtain a labeled image. The labeling is performed in a manner based on prompt points and boxes, and features of different morphological types can be labeled, which has a wide range of applicable scenarios. It also realizes semi-automatic labeling of images, lowers the usage threshold, brings great convenience to the staff's labeling, saves a lot of manpower and time costs, and improves labeling efficiency and labeling quality.
[0166] In some embodiments, the first input includes multiple sub-inputs for multiple different target areas; the second processing module 1420 may also be used to:
[0167] In response to the multiple sub-inputs, based on the input order of the sub-inputs, the feature images associated with the sub-inputs are labeled in sequence to obtain labeled images.
[0168] In some embodiments, the second processing module 1420 may also be used to:
[0169] In response to a previous sub-input among the multiple sub-inputs, annotating a feature image associated with the previous sub-input to obtain a first annotated image corresponding to the previous sub-input;
[0170] In response to a latter sub-input among the multiple sub-inputs, a feature image associated with the latter sub-input is annotated on the first annotated image to obtain an annotated image.
[0171] In some embodiments, the second processing module 1420 may also be used to:
[0172] The feature image associated with the target area is annotated based on the target annotation model to obtain an annotated image output by the target annotation model; the target annotation model includes a lightweight encoder and a decoder, and the lightweight encoder is obtained by distillation training based on an industrial dataset.
[0173] In some embodiments, the second processing module 1420 may also be used to:
[0174] Extract the contour of the feature image;
[0175] The feature image is annotated based on the contour to obtain an annotated image.
[0176] In some embodiments, the industrial labeling device may further include a fourth processing module, which is used to label the feature image based on the contour to obtain the labeled image, and then calculate the area of the region within the contour based on the contour;
[0177] When the area of the region is smaller than the first area threshold, deleting the annotation information of the feature image corresponding to the contour;
[0178] When the area of the region is not less than the second area threshold, the annotation information of the region exceeding the second area threshold is deleted; the second area threshold is greater than the first area threshold.
[0179] The industrial labeling device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0180] The industrial labeling device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0181] The industrial labeling device provided in the embodiment of the present application can achieve Figures 1 to 13 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0182] In some embodiments, Fig.15As shown, an embodiment of the present application also provides an electronic device 1500, including a processor 1501, a memory 1502, and a computer program stored in the memory 1502 and executable on the processor 1501. When the program is executed by the processor 1501, each process of the above-mentioned industrial labeling method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0183] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0184] On the other hand, the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the various processes of the above-mentioned industrial labeling method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0185] On the other hand, the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it is implemented to perform the various processes of the above-mentioned industrial labeling method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0186] On the other hand, an embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned industrial labeling method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0187] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0188] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0189] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution can be essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.
[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An industrial labeling method, characterized in that: include: Receive a first input from a user regarding a target area of the image to be annotated; the first input includes at least one of a box selection input and a click input; In response to the first input, annotating a feature image associated with the target area to obtain an annotated image; The annotated image is output, where the annotated image includes image features and annotation information of the image to be annotated.
2. The industrial labeling method according to claim 1, characterized in that: The first input includes a plurality of sub-inputs for a plurality of different target regions; in response to the first input, annotating a feature image associated with the target region to obtain an annotated image includes: In response to the multiple sub-inputs, based on the input order of the sub-inputs, the feature images associated with the sub-inputs are annotated in sequence to obtain the annotated images.
3. The industrial labeling method according to claim 2, characterized in that: The step of annotating the feature images associated with the sub-inputs in sequence based on the input order of the sub-inputs in response to the multiple sub-inputs to obtain the annotated images includes: In response to a previous sub-input among the multiple sub-inputs, annotating a feature image associated with the previous sub-input to obtain a first annotated image corresponding to the previous sub-input; In response to a latter sub-input among the multiple sub-inputs, a feature image associated with the latter sub-input is annotated on the first annotated image to obtain the annotated image.
4. The industrial labeling method according to any one of claims 1 to 3, characterized in that: The step of labeling the characteristic image associated with the target area to obtain a labeled image includes: The feature image associated with the target area is annotated based on a target annotation model to obtain the annotated image output by the target annotation model; the target annotation model includes a lightweight encoder and a decoder, and the lightweight encoder is obtained by distillation training based on an industrial dataset.
5. The industrial labeling method according to any one of claims 1 to 3, characterized in that: The step of labeling the characteristic image associated with the target area to obtain a labeled image includes: Extracting the contour of the feature image; The feature image is annotated based on the contour to obtain the annotated image.
6. The industrial labeling method according to claim 5, characterized in that: After the feature image is annotated based on the contour to obtain the annotated image, the method further includes: Calculate the area of the region within the outline based on the outline; When the area of the region is smaller than a first area threshold, deleting the annotation information of the feature image corresponding to the contour; When the area of the region is not less than a second area threshold, the annotation information of the region exceeding the second area threshold is deleted; the second area threshold is greater than the first area threshold.
7. An industrial marking device, characterized in that: include: A first processing module, configured to receive a first input from a user of a target area of an image to be annotated; the first input includes at least one of a box selection input and a click input; A second processing module, configured to annotate the feature image associated with the target area in response to the first input to obtain an annotated image; The third processing module is used to output the annotated image, where the annotated image includes image features and annotation information of the image to be annotated.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the industrial labeling method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the industrial labeling method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the industrial labeling method according to any one of claims 1 to 6 is implemented.