Labeling assistance device and labeling assistance method

Abstract

Provided is a technique for efficiently performing annotation while comparing with a similar image. An annotation assistance device (100) is provided with: a clustering unit (35) that clusters image data included in an image data group (60); and a label reception unit (206) that receives the application of a label to the image data included in the cluster generated by the clustering.

Description

TECHNICAL FIELD

[0001] The present application relates to an annotation assisting apparatus and an annotation assisting method. BACKGROUND

[0002] In the annotation work of the pass and fail in AI, since the initial image data group is in a state where various types of images exist in a large amount and in disorder, it is necessary to concentrate on adding a plurality of labels uniformly to any image. In the pass and fail determination, there are many cases where the evaluation of pass and fail and the pass and fail are ambiguous, and the work takes time.

[0003] Therefore, a technology of assisting the annotation of a user is proposed (for example, refer to Patent Literature 1).

[0004] Here, the display apparatus displays an image in which annotation is not performed among images registered in the image database, receives text data or numerical data in which an image input by the user via the input apparatus is described, and records the data in the training data field of the image. The image data management information of each image including the training data field is registered in the image database. The attribute 2 feature amount field is included in the image data management information, and a value indicating a category based on the similarity to any one of the images registered in the image database 422 is recorded. For the images having the same attribute 2 feature amount, at the time when the above data is input in the training data field of any image, the same data is also recorded in the training data field for the images having the same attribute 2 feature amount, thereby reducing the number of annotations of the user.

[0005] PRIOR ART DOCUMENT

[0006] PATENT LITERATURE

[0007] Patent Literature 1: Japanese Patent Application Laid-Open No. 2017-224184 SUMMARY

[0008] PROBLEMS TO BE SOLVED BY THE INVENTION

[0009] The present application is achieved in view of the above-described problems, and an object thereof is to provide a technology of efficiently performing annotation while comparing with similar images.

[0010] MEANS FOR SOLVING THE PROBLEMS

[0011] The present application for solving the above problems is an annotation assisting apparatus characterized by having: a clustering section that clusters image data included in an image data group; and an annotation receiving section that receives the assignment of a label for image data included in a cluster generated by the clustering.

[0012] Thus, clustering can be performed also for image data that has not been labeled in advance. In addition, since image data included in a cluster is labeled as a target, labeling can be performed efficiently while comparing with similar images.

[0013] Further, in the present application, there can be provided a reference value providing unit that provides a reference value indicating a feature of image data when the image data included in the cluster is subjected to the assignment of the label.

[0014] Thus, labeling can be performed while referring to a reference value indicating a feature of image data, and therefore efficient labeling can be performed.

[0015] Further, in the present application, the label receiving unit can arrange the image data included in the cluster based on the reference value.

[0016] Thus, image data that is a target of labeling is arranged based on a reference value, and therefore similar images are arranged closer, and more efficient labeling can be performed.

[0017] Further, in the present application, there can be provided a reference data receiving unit that receives a selection of image data to which a predetermined label is to be assigned from among the image data group, as a reference image data group, and the reference value is an index indicating similarity to reference image data included in the reference image data group.

[0018] Thus, as a reference value, an index indicating similarity to reference image data is used, and therefore labeling based on reference image data can be performed more efficiently.

[0019] Further, in the present application, there can be provided an automatic labeling unit that assigns a predetermined label to the image data included in the image data group based on the index indicating similarity to reference image data included in the reference image data group.

[0020] Thus, a predetermined label is automatically assigned based on an index indicating similarity to reference image data included in a reference image data group, and therefore the time required for labeling of image data similar to reference image data can be shortened, and efficient labeling can be performed.

[0021] Further, in the present application, the reference data receiving unit can provide an auxiliary index indicating appropriateness of the selection when the selection of the reference image data group is received.

[0022] Thus, a more appropriate reference data group can be selected based on an auxiliary index.

[0023] Further, in the present application, it can also be that a preprocessing section is provided which performs prescribed image processing on the image data included in the image data group for the clustering.

[0024] Thus, the clustering can be performed on the image data on which prescribed image processing has been performed in advance, and therefore, unnecessary information and the like can be removed in advance, and high-precision clustering can be performed.

[0025] Further, in the present application, it can also be that the label reception section collectively receives the assignment of the label for a plurality of the image data included in the cluster.

[0026] Thus, compared to the case where the label is assigned to the image data individually, efficient labeling can be performed.

[0027] Further, in the present application, it can also be that a reference setting reception section is provided which receives the setting of the number of the image data selected as the reference image data group.

[0028] Thus, by appropriately setting the number of the image data selected as the reference image data group, the precision of the clustering can be adjusted.

[0029] Further, in the present application, it can also be that the label reception section changes the display mode of the image data included in the cluster in accordance with the distance between the image data included in the cluster and the center of the cluster.

[0030] Thus, the distance between the image data and the center of the cluster can be recognized in accordance with the display mode of the image data, and therefore, efficient labeling can be performed.

[0031] Further, in the present application, it can also be that the label reception section receives the setting of an attention region in the image data, and displays the attention region of the image data.

[0032] Thus, the labeling can be performed while paying attention to the image of the attention region, and therefore, high-precision labeling can be performed.

[0033] Further, in the present application, it can also be that a re-clustering reception section is provided which receives the re-clustering of the image data in the image data group to which the label has not been assigned, and the label reception section receives the assignment of the label for the image data included in the cluster generated by the re-clustering.

[0034] Thus, the image data to which the label has not been assigned can be labeled based on the relationship between different image data included in a new cluster, and therefore, high-precision labeling can be performed.

[0035] Further, in the present application, the label reception unit can provide a test result of a learning model generated using the labeled image data set as learning data, and can receive a change of the assigned label.

[0036] Thus, the assigned label can be changed based on the test result of the learning model generated using the labeled image data as learning data, and thus the labeling can be efficiently performed.

[0037] Further, the present application is a labeling assistance method including the steps of clustering image data included in an image data set, and receiving assignment of a label for image data included in a cluster generated by the clustering.

[0038] Thus, the image data that has not been labeled in advance can also be clustered. Further, since the image data included in the cluster is labeled as an object, the labeling can be efficiently performed while comparing with similar images.

[0039] Effects of the Invention

[0040] According to the present application, a technology that can efficiently perform labeling while comparing with similar images can be provided. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a functional block diagram of a labeling assistance apparatus of an embodiment of the present application.

[0042] Figure 2 is a diagram showing a hardware structure of a labeling assistance apparatus of an embodiment of the present application.

[0043] Figure 3 is a flowchart showing a process of a determiner generation process of an embodiment of the present application.

[0044] Figure 4 is a flowchart showing a process of a labeling assistance process of an embodiment of the present application.

[0045] Figure 5 is a diagram showing a display example of a typical good product selection screen of an embodiment of the present application.

[0046] Figure 6 is a diagram showing a concept of clustering of an embodiment of the present application.

[0047] Figure 7 is a diagram showing a display example of a labeling screen of an embodiment of the present application.

[0048] Figure 8 is a diagram showing a display example of a region selection screen of an embodiment of the present application.

[0049] Figure 9is a drawing showing an example of a display of a test result screen of an embodiment of the present application. DETAILED DESCRIPTION

[0050] [Application Example]

[0051] An application example of the present application will be described below with reference to the drawings.

[0052] Figure 1 is a functional block diagram showing the labeling assistance device 100. Figure 4 is a flowchart showing a labeling assistance method.

[0053] The labeling assistance device 100 includes an input section 10, a display section 20, a processing section 30, a data management section 40, a labeling data storage section 50, and can be constituted by a general computer device. In addition, the processing section 30 includes a preprocessing section 31, a learning processing section 32, a reference information assignment processing section 33, an automatic labeling section 34, a grouping processing section 35, an ROI processing section 36, and a screen information generation section 37.

[0054] First, according to an instruction by a user through the input section 10, the data management section 40 reads in the image data group 60 (step Sll).

[0055] Next, the images included in the read-in image data group 60 are displayed on the display section 20, and the user selects 20 typical good product images (hereinafter, referred to as "typical good product images") (step S12). Figure 5 An example of a good product selection screen 201 generated by the screen information generation section 37 and displayed on the display section 20 is shown. The user switches the images to be displayed in the image display area 202 by operating the scroll bars 203 and 204, selects any of the images displayed in the image display area 202 by right-clicking a mouse or the like, and confirms the selection of the typical good product images by selecting the "OK" button 205 displayed on the good product selection screen 201.

[0056] In parallel with the selection of the typical good product images by the user in step S2, in the preprocessing section 31 of the processing section 30, the image data of the read-in image data group 60 is subjected to a resist color and wiring pattern color removal process as a preprocessing (step S13).

[0057] Next, the learning processing section 32 of the processing section 30 learns the typical good product image data selected by the user in step S2 (step S14), and generates a model that outputs a similarity with respect to the typical good product image data as an output value with respect to the input image data.

[0058] Next, image data with output values ​​below a specified threshold are removed from the overall data (step S15). The closer an image is to a typical qualified product image, the smaller its output value. In this way, by removing image data with output values ​​below a specified threshold from the overall data, obviously qualified products can be removed from the labeled objects, thus achieving efficient labeling.

[0059] Next, the grouping processing unit 35 of the processing unit 30 groups the remaining image data after removing image data with output values ​​below a specified threshold from the total in step S5 (step S16).

[0060] Next, the user labels the image data grouped in step S6 according to each group. Figure 7 This example shows a labeling screen 206 displayed on the display unit 20. Images IM11 to IM19 are displayed in the image display area 207. Output values ​​217 are also displayed in the upper right corner of each image IM11 to IM19. When the user moves the pointer to each image IM11 to IM19 displayed in the image display area 207 and left-clicks the mouse, the label for that image changes sequentially to OK, NG, and Gray, allowing the user to select the desired label. In the image display area 207, images IM11 to IM19 are arranged and displayed according to the order of their output values. When the output values ​​represent similarity, the images are displayed in a similar order; therefore, images with output values ​​higher than a certain value can be collectively classified as NG or Gray.

[0061] Image IM17, displayed in image display area 207, is highlighted with a different background color (HL). Images that are off-center by a specified percentage (here, 10%) are also highlighted (HL).

[0062] The user switches between groups of images displayed in image display area 207 as described above, and annotates at least a portion of the images in each group.

[0063] If there are images that the user has not labeled, the user can choose whether to regroup the unlabeled images (step S18). By pressing the "Regroup Unlabeled Images" button 222, the grouping processing unit 35 of the processing unit 30 regroups the unlabeled images, and the user can assign labels to the regrouped images.

[0064] In this way, by using the annotation aid 100, similar images can be annotated continuously, thus shortening the time required for annotation work.

[0065] [Example 1]

[0066] Hereinafter, the annotation assistance device 100 of Embodiment 1 of the present invention will be described in more detail using the accompanying drawings. However, the structures of the devices and systems described in this embodiment should be appropriately changed according to various conditions. That is, the scope of the present invention is not limited to the following embodiments.

[0067] Figure 1 A functional block diagram showing the annotation assistance device 100. Figure 2 A hardware structure diagram showing the annotation assistance device 100. Figure 3 A flowchart for explaining the generation process of a determination device based on machine learning. Figure 4 A flowchart for explaining the annotation assistance method.

[0068] When generating a determination device for determining pass / fail based on images of components and pads through machine learning, the processing is performed according to the Figure 3 flowchart shown.

[0069] First, images of object areas such as components and pads are collected (step S1).

[0070] Next, annotation for attaching pass / fail labels to the collected images is performed (step S2).

[0071] Learning data is extracted from the annotated image data, and the learning model performs machine learning on the extracted learning data (step S3). Thus, a determination device that outputs a pass / fail determination result based on the input image is generated.

[0072] Next, the performance of the generated determination device is tested using the test data extracted from the annotated image data (step S4).

[0073] If the performance of the generated determination device is sufficient and the test result is acceptable, the learned determination device is applied to the teaching software for actual operation (step S5).

[0074] If the performance of the generated determination device is insufficient and there is a problem with the test result, return to step S2) and perform annotation again.

[0075] In order to generate a determination device with good performance, a large amount of high-quality learning data needs to be prepared. Therefore, the annotation operation of attaching labels to the collected image data plays an important role.

[0076] The annotation assistance device 100 described below assists such annotation and efficiently generates high-quality learning data.

[0077] The annotation assistance device 100 includes an input unit 10, a display unit 20, a processing unit 30, a data management unit 40, and an annotation data storage unit 50.

[0078] The annotation assistance device 100 is composed of a general-purpose computer device including a processor 101 such as a CPU, a main storage unit (memory) 102, an auxiliary storage unit (hard disk, etc.) 103, an input unit (keyboard, mouse, controller, touch panel, etc.) 10, a display unit (monitor) 20, and a communication unit 104. The annotation assistance device 100 may also be a computer system in which multiple computer devices cooperate.

[0079] The processing unit 30 and the data management unit 40 are implemented by expanding the program stored in the auxiliary storage unit 103 in the main storage unit and executing it in the processor 101.

[0080] The following is in accordance with Figure 4 The flowchart illustrates the annotation assistance method performed in the annotation assistance device 100.

[0081] First, according to the user's instructions via the input unit 10, the data management unit 40 reads the image data group 60 (step S11). The image data group 60 can be read from an external storage device connected via a network via the communication unit 104, or it can be read from the auxiliary storage unit 103.

[0082] Next, the images contained in the read image data group 60 are displayed on the display unit 20, and the user selects 20 typical qualified product images (hereinafter referred to as "typical qualified product images") (step S12). Figure 5 An example of a qualified product selection screen 201 displayed on the display unit 20 is shown. It displays approximately 1000 images contained in the image data group 60, allowing the user to select an image from which they determine that the product is a typical qualified product. Figure 5In the image display area 202, nine images, IM1 to IM9, are displayed. The user switches between images displayed in the image display area 202 using scroll bars 203 and 204, selects any image in the image display area 202 by right-clicking the mouse, and confirms the selection of typical qualified product images by selecting the "OK" button 205 displayed on the qualified product selection screen 201. The number of typical qualified product images selected is an example; the number of typical qualified product images can be entered or selected using an input field or a reference setting receiving unit in the form of a spinner, and an appropriate number of images can be selected. The selection of typical qualified product images in this stage is performed on each component that is targeted. When the user selects an image, an indicator indicating the appropriateness of the selection can also be displayed. Such an indicator is, for example, an indicator indicating the similarity of the selected images. If an indicator whose score in the similarity indicator is close can be selected, it can be confirmed that the selection is appropriate. The qualified product selection screen 201 corresponds to the reference data receiving unit of this invention. Furthermore, the typical qualified product images correspond to the reference image data group and reference image data of this invention. The indicators representing the appropriateness of the selection are equivalent to the auxiliary indicators of this invention.

[0083] In parallel with the typical qualified product image selection performed by the user in step S2, the preprocessing unit 31 of the processing unit 30 performs preprocessing on the image data of the read image data group 60 by removing resist and wiring pattern colors (step S13). By removing parts that are not needed in the judgment, such as resist and wiring pattern colors, the accuracy of the judgment can be improved. The preprocessed image data is then processed further. Here, the removal of resist and wiring pattern colors corresponds to the image processing specified for clustering in this invention.

[0084] Next, in step S2, the learning processing unit 32 of the processing unit 30 learns typical qualified product image data selected by the user (step S14) and generates a model that outputs a similarity to the typical qualified product image data for the input image data. As a learning model, an autoencoder can be used, but it is not limited to this. The similarity to the typical qualified product image data can also be calculated using a similarity calculation method for two images, such as SSIM (Structural Similarity), to calculate the average distance. Here, similarity or average distance is equivalent to an index representing similarity or a reference value representing a feature.

[0085] Next, the reference information from the processing unit 30 is used by the processing unit 33 to calculate output values ​​such as the similarity of the image data contained in the image data group 60 using the learned model and other methods. The automatic annotation unit 34 then removes image data with output values ​​below a predetermined threshold from the overall data (step S15). The closer an image is to a typical qualified product image, the smaller its output value. In this way, by removing image data with output values ​​below the predetermined threshold from the overall data, obviously qualified products can be removed from the labeled objects, thus achieving efficient annotation. By assigning the "OK" label to images with output values ​​below the predetermined threshold, separate annotation work is not required. The threshold can be set as an average value, or it can be an appropriate threshold that can be set by the user.

[0086] Next, the grouping processing unit 35 of the processing unit 30 groups (clusters) the remaining image data after removing image data with output values ​​below a specified threshold from the population through step S5 (step S16). Figure 6 This diagram conceptually illustrates the grouping performed by the grouping processing unit 35. Here, an example is given where the grouped image data is arranged in a two-dimensional feature space. Here, a typical acceptable image GIM without defective features is located near the origin of the feature space. A colored circle IM corresponds to one image, and a circle (ellipse) CL containing multiple circles corresponds to one group (cluster). Arrow Ar1 represents the output value, expressed as the distance from the typical acceptable image in the feature space. Arrow Ar2 represents the distance from the cluster center C of the image belonging to cluster CL, and is highlighted based on this distance, as described later. The grouping processing unit 35 corresponds to the clustering unit of this invention.

[0087] Next, the user labels the image data grouped in step S6 according to each group. Figure 7 An example of a labeling screen 206 generated by the image information generation unit 37 based on grouping results and displayed on the display unit 20 is shown. Here, on the upper side of the image display area 207, the group name 210 containing the image data displayed in the image display area 207 is displayed as "Group A". In addition, the group number 211 of all groups, i.e., "1 / 20" (the first group out of all 20 groups), is also displayed. Moreover, a "Previous Group" button 212 and a "Next Group" button 213 for switching the image data displayed in the image display area 207 to the previous or next group are displayed respectively. Below the group name 210, "OK" button 214, "NG" button 215, and "Gray" button 216 are displayed for assigning labels "OK", "NG", and "Gray" to the image data contained in the group. The labeling screen 206 corresponds to the labeling receiving unit of the present invention.

[0088] Images IM11 to IM19 are displayed in image display area 207. Output values ​​217 are also displayed in the upper right corner of each image IM11 to IM19. When the user moves the pointer to each image IM11 to IM19 displayed in image display area 207 and left-clicks the mouse, the label for that image changes sequentially to OK, NG, and Gray, allowing the user to select the desired label. The selected label 218 is displayed as "OK" in the upper left corner of each image IM11 to IM19. The output values ​​217 for each image IM11 to IM19 are displayed in image display area 207, and the images IM11 to IM19 are arranged and displayed according to the order of these output values. Since the output values ​​represent similarity, the images are displayed in order of similarity; therefore, images with output values ​​above a certain value can be judged as either NG or Gray. Here, the images are displayed in image display area 207 according to the order of their output values, but it is also possible to allow the user to choose whether to arrange the images according to their output values ​​or to arrange them using other methods such as the scoring order based on past judgment models. Here, the image displayed in the image display area 207 can also be changed by manipulating scroll bars 219 and 220. Here, the output value corresponds to a reference value representing the characteristics of the image data of the present invention, and the annotation screen 206 corresponds to the reference value providing unit of the present invention.

[0089] For the image IM17 displayed in the image display area 207, a different background color is used for highlighting (HL). The HL highlighting is performed on images that are offset from the center of the cluster by a predetermined proportion, as described above. A slider 221 is displayed on the right side of the image display area 207, allowing the user to set the proportion of the image to be highlighted. Here, the image that is 10% offset from the center of the cluster is set to be highlighted. The different background colors used for highlighting (HL) correspond to the display method of the present invention. Display methods that vary according to the distance from the center of the cluster are not limited to this.

[0090] On the right side of the image display area 207, a "Regroup Unlabeled Images" button 222 and a "Save Results" button 223 are displayed. When the user presses the "Save Results" button 223, the user's annotations are determined (step S17). The determined annotation results are stored in the annotation data storage unit 50 by the data management unit 40. The attached tags and image IDs that identify the image data are stored in the annotation data storage unit 50 in association with each other. The user may not assign tags to all images in each group, or may determine annotations only for a portion of the grouped images. When the user regroups unlabeled images, the "Regroup Unlabeled Images" button 222 is pressed. The "Regroup Unlabeled Images" button 222 corresponds to the re-clustering receiving unit of the present invention.

[0091] Users can also designate a portion of the image as a region of interest (ROI), which will be enlarged and displayed in the image display area 235. Figure 8 This is an example of a focus area setting acceptance screen 230. For example, a user can select a focus area 233 by enclosing a portion of the pop-up magnified image 231 with a rectangular frame 232. By pressing the "OK" button 234, the focus area 233 setting is confirmed. Based on this setting, the ROI processing unit 36 ​​of the processing unit 30 generates images IM21 to IM29 of the focus area. The screen information generation unit 37 then displays the focus area setting acceptance screen 230, which shows the images IM21 to IM29 of the focus area in the image display area 235, on the display unit 20. The setting of the focus area 233 is not limited to the user selecting a desired range; alternatively, the ROI processing unit 36 ​​of the processing unit 30 may recommend an appropriate focus area 233, which the user can then approve and set.

[0092] The user switches between groups of images displayed in image display area 207 as described above, and annotates at least a portion of the images in each group.

[0093] If there are images that the user has not labeled, the user can choose whether to regroup the unlabeled images (step S18). If regrouping the unlabeled images is necessary, the user presses the "Regroup Unlabeled Images" button 222. When the "Regroup Unlabeled Images" button 222 is pressed, the process returns to step S6, and the grouping processing unit 35 of the processing unit 30 regroups the unlabeled images. Then, with... Figure 7 Similarly, the regrouped images are displayed on the display unit 20, and the user can assign labels to the regrouped images (step S17).

[0094] Labeling is complete when all data is labeled (step S9).

[0095] As described above, learning data and test data are extracted from the labeled image data. Then, the decision model is used to perform machine learning on the learning data (step S3), and the test data is input into the learned decision model for testing (step S4).

[0096] Figure 9 This is an example of a test result screen with resolution 240.

[0097] The test results screen 240 displays a summary of the judgments 241. In summary 241, the number of data items judged as OK and the number of data items judged as NG are displayed in tabular form for OK data, NG data, and gray data, respectively. Here, 1000 items in the OK data were judged as OK, and 20 items were judged as NG. Additionally, 0 items in the NG data were judged as OK, and 100 items were judged as NG. Furthermore, regarding the gray data, there were 0 OK and 0 NG judgments.

[0098] The test result screen 240 displays detailed result bars 242 for each image. If there are issues with the test results, i.e., the performance of the generated judgment model is insufficient, the detailed result bar 242 displays the image IM31 (the target image), the output value 243, and a "move" button 244 for the annotation screen 206. Pressing the "move" button 244 moves the image to the annotation screen 206 and changes the assigned label. Additionally, the test result screen 240 displays histograms 245 representing the measurement values ​​for OK and NG, respectively. These histograms 245 confirm whether the OK measurement value 245a and the NG measurement value 245b are separated. Furthermore, the test result screen 240 displays a "model application" button 246. If the model generated by the AI ​​learning tool is without problems based on the test results, the generated model is applied to the teaching software by the user pressing the "model application" button 246 (step S6).

[0099] In addition, in order to compare the structural elements of this disclosure with the structures of the embodiments, the structural elements of this disclosure are described using reference numerals.

[0100] <Postscript 1>

[0101] A labeling assistance device (100) is characterized by having: a clustering unit (35) that clusters image data contained in an image data group (60); and a labeling acceptance unit (206) that accepts the assignment of labels to the image data contained in the clusters generated by the clustering.

[0102] <Appendix 2>

[0103] According to Appendix 1, the annotation assisting device (100) is characterized in that it has a reference value providing unit (201) which provides a reference value representing the characteristics of the image data when the label is assigned to the image data contained in the cluster.

[0104] <Appendix 3>

[0105] According to Appendix 2, the annotation assist device (100) is characterized in that the annotation receiving unit (206) arranges the image data contained in the cluster according to the reference value.

[0106] <Appendix 4>

[0107] According to Appendix 2 or 3, the annotation aid device (100) is characterized in that it has a reference data receiving unit that receives the selection of image data to be assigned a specified label from the image data group (60) as a reference image data group, wherein the reference value is an index representing the similarity to the reference image data contained in the reference image data group.

[0108] <Appendix 5>

[0109] According to Appendix 4, the annotation aid device (100) is characterized in that it has an automatic annotation unit (34) that assigns a predetermined label to the image data contained in the image data group based on the index representing the similarity to the reference image data contained in the reference image data group.

[0110] <Appendix 6>

[0111] The annotation aid device (100) according to Appendix 4 or 5 is characterized in that the reference data receiving unit (201) provides an auxiliary index indicating the appropriateness of the selection when receiving the selection of the reference image data group.

[0112] <Appendix 7>

[0113] In any of the annotation aids (100) described in notes 1 to 6, the annotation aids (100) has a preprocessing unit (31) that performs prescribed image processing on the image data contained in the image data group (60) for the clustering.

[0114] <Appendix 8>

[0115] The annotation assisting device (100) according to any one of appendices 1 to 7 is characterized in that the annotation receiving unit (206) accepts the assignment of labels to the multiple image data contained in the cluster at the same time.

[0116] <Appendix 9>

[0117] The annotation aid device (100) according to any one of appendices 4 to 6 is characterized in that the annotation aid device (100) has a reference setting receiving unit that accepts the setting of the number of image data selected as the reference image data group.

[0118] <Postscript 10>

[0119] The annotation aid device (100) according to any one of appendices 1 to 9 is characterized in that the annotation receiving unit (206) changes the display mode of the image data contained in the cluster according to the distance between the image data contained in the cluster and the center of the cluster.

[0120] <Postscript 11>

[0121] The annotation aid device (100) according to any one of appendices 1 to 10 is characterized in that the annotation receiving unit (206) accepts the setting of the region of interest in the image data and displays the region of interest in the image data.

[0122] <Appendix 12>

[0123] The labeling aid device (100) according to any one of Appendix 1 to 11 is characterized in that the labeling aid device (100) has a re-clustering acceptance unit (222) which accepts the re-clustering of the image data in the image data group (60) that has not been assigned the label, and the labeling acceptance unit (206) accepts the assignment of the label to the image data contained in the cluster generated by the re-clustering.

[0124] <Appendix 13>

[0125] The annotation assist device (100) according to any one of appendices 1 to 12 is characterized in that the annotation receiving unit (206) provides test results of the learning model generated by using the annotated image data set as learning data, and accepts changes to the assigned labels.

[0126] <Appendix 14>

[0127] A labeling-assisted method includes: step (step S16), clustering image data (60) contained in an image data group; and step (step S17), assigning labels to the image data contained in the clusters generated by the clustering.

[0128] Label Explanation

[0129] 35: Group Processing Department

[0130] 60: Image Data Group

[0131] 100: Labeling auxiliary devices

[0132] 206: Annotated screen

Claims

1. A labeling auxiliary device, characterized in that, have: The clustering department clusters the image data contained in the image data group; and The labeling and acceptance department assigns labels to the image data contained in the clusters generated by the clustering.

2. The labeling auxiliary device according to claim 1, characterized in that, The labeling assistance device has a reference value providing unit that provides reference values ​​representing the characteristics of the image data when assigning the label to the image data contained in the cluster.

3. The labeling auxiliary device according to claim 2, characterized in that, The annotation receiving department arranges the image data contained in the cluster according to the reference value.

4. The labeling auxiliary device according to claim 2 or 3, characterized in that, The labeling aid device has a reference data receiving unit, which receives a selection of image data from the image data set for which a specified label is to be assigned, and uses this selection as the reference image data set. The reference value is an indicator representing the similarity to the reference image data contained in the reference image data set.

5. The labeling auxiliary device according to claim 4, characterized in that, The annotation aid device has an automatic annotation unit that assigns a predetermined label to the image data contained in the image data group based on the index representing the similarity to the reference image data contained in the reference image data group.

6. The labeling auxiliary device according to claim 4, characterized in that, When accepting the selection of the reference image data group, the reference data receiving unit provides auxiliary indicators to indicate the appropriateness of the selection.

7. The annotation auxiliary device according to any one of claims 1 to 3, wherein, The annotation aid device has a preprocessing unit that performs prescribed image processing on the image data contained in the image data group for the clustering.

8. The annotation auxiliary device according to any one of claims 1 to 3, characterized in that, The labeling processing department accepts the assignment of labels to multiple image data contained in the cluster.

9. The annotation auxiliary device according to claim 4, characterized in that, The annotation aid has a reference setting receiving unit that accepts the setting of the number of image data selected as the reference image data group.

10. The annotation auxiliary device according to any one of claims 1 to 3, characterized in that, The annotation receiving department changes the display method of the image data contained in the cluster according to the distance between the image data contained in the cluster and the center of the cluster.

11. The annotation auxiliary device according to any one of claims 1 to 3, characterized in that, The annotation acceptance department accepts the setting of the region of interest in the image data and displays the region of interest in the image data.

12. The annotation auxiliary device according to any one of claims 1 to 3, characterized in that, The labeling aid device has a re-clustering acceptance unit, which accepts the re-clustering of image data in the image data group that have not been assigned the label. The labeling and acceptance department accepts the assignment of labels to the image data contained in the clusters generated by the re-clustering.

13. The annotation auxiliary device according to any one of claims 1 to 3, characterized in that, The labeling acceptance department provides test results of the learning model generated by using the labeled image data group as learning data, and accepts changes to the assigned labels.

14. A labeling assistance method, comprising the following steps: Cluster the image data contained in the image data set; and The image data contained in the clusters generated by the clustering are assigned acceptance labels.

Images