Annotation support device and annotation support method

Abstract

The present invention provides a technique for efficiently performing annotation while comparing with a similar image. An annotation support device (100) is provided with: a clustering unit (35) for clustering image data included in an image data group (60); and an annotation reception unit (206) for receiving the assignment of a label to image data included in the cluster generated by the clustering.

Description

Annotation support device and annotation support method

[0001] The present invention relates to an annotation support device and an annotation support method.

[0002] In the annotation process for determining good and bad products using AI, the initial image data set contains a large number of scattered images with various variations, so it is necessary to uniformly and centrally assign many labels to each image. In determining whether a product is good or bad, the evaluation of good or bad and defective products is often ambiguous, making the process time-consuming.

[0003] For this reason, a technology has been proposed to support user annotation (see, for example, Patent Document 1). In this technology, unannotated images from among images registered in an image database are displayed on a display device, text or numerical data describing the image is received via an input device by the user, and this data is recorded in the teacher data field of the image. Image data management information for each image, including a teacher data field, is registered in the image database. The image data management information includes an attribute 2 feature field, which records a numerical value or the like indicating a category based on the degree of similarity with one of the images registered in the image database 422. For images with the same attribute 2 feature, when the data is entered in the teacher data field of any of the images, the same data is also recorded in the teacher data field for the images with the same attribute 2 feature, thereby reducing the number of annotations performed by the user.

[0004] Japanese Patent Application Laid-Open No. 2017-224184

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technology for efficiently performing annotation while comparing with similar images.

[0006] In order to solve the above problem, the present invention provides an annotation support device comprising: a clustering unit that clusters image data included in an image data group; and an annotation receiving unit that receives assignment of labels to image data included in the clusters generated by the clustering.

[0007] This allows clustering of image data that has not been annotated in advance. Furthermore, since annotation is performed on image data included in a cluster, annotation can be performed efficiently while comparing similar images.

[0008] The present invention may further comprise a reference value providing unit that provides a reference value indicating a feature of the image data when the label is assigned to the image data included in the cluster.

[0009] This allows annotation to be performed while referring to reference values ​​that indicate the characteristics of the image data, thereby enabling efficient annotation.

[0010] In the present invention, the annotation receiving unit may arrange the image data included in the cluster according to the reference value.

[0011] According to this, image data to be annotated is arranged according to the reference value, so that similar images are arranged close to each other, enabling more efficient annotation.

[0012] In addition, in the present invention, a reference data receiving unit is provided that receives a selection of image data from the image data group to which a predetermined label is to be assigned as a reference image data group, and the reference value may be an index indicating the similarity with the reference image data included in the reference image data group.

[0013] According to this, an index indicating the degree of similarity with the reference image data is used as the reference value, so that annotation based on the reference image data can be performed more effectively.

[0014] In addition, the present invention may include an automatic annotation unit that assigns a predetermined label to the image data included in the image data group based on the index indicating the similarity with reference image data included in the reference image data group.

[0015] According to this, a predetermined label is automatically assigned based on an index indicating the degree of similarity with the reference image data included in the reference image data group, thereby shortening the time required to annotate image data that is similar to the reference image data, enabling efficient annotation.

[0016] In the present invention, the reference data receiving section may provide an auxiliary index indicating the appropriateness of the selection when receiving the selection of the reference image data group.

[0017] This makes it possible to select a more appropriate reference data group based on the auxiliary index.

[0018] In the present invention, a preprocessing unit may be provided that performs predetermined image processing for the clustering on the image data included in the image data group.

[0019] According to this, clustering can be performed on image data that has been subjected to predetermined image processing in advance, so that unnecessary information can be removed in advance, and highly accurate clustering can be achieved.

[0020] In the present invention, the annotation receiving unit may collectively receive the assignment of the label to a plurality of the image data included in the cluster.

[0021] This allows for more efficient annotation than when labels are individually assigned to image data.

[0022] In the present invention, a reference setting receiving section may be provided that receives a setting of the number of image data to be selected as the reference image data group.

[0023] According to this, the accuracy of clustering can be adjusted by appropriately setting the number of image data selected as the reference image data group.

[0024] In the present invention, the annotation receiving unit may change a display mode of the image data included in the cluster depending on a distance from the center of the cluster.

[0025] This allows the distance between the image data and the center of the cluster to be recognized based on the display mode of the image data, thereby enabling efficient annotation.

[0026] In the present invention, the annotation receiving unit may receive a setting of a region of interest in the image data, and display the region of interest in the image data.

[0027] This allows annotation while focusing on the image of the region of interest, making it possible to perform annotation with high accuracy.

[0028] In addition, in the present invention, the image data group may include a re-clustering receiving unit that receives re-clustering of image data to which the label has not been assigned, and the annotation receiving unit may be configured to receive assignment of the label to image data included in the cluster generated by the re-clustering.

[0029] This allows unannotated image data to be annotated based on the relationship with different image data included in the new cluster, thereby enabling highly accurate annotation.

[0030] In addition, in the present invention, the annotation receiving unit may provide test results of a learning model generated using the annotated image data group as learning data, and may also receive changes to the assigned labels.

[0031] This allows the assigned labels to be changed according to the test results of a learning model generated using annotated image data as learning data, thereby enabling efficient annotation.

[0032] The present invention also provides an annotation support method including: a step of clustering image data included in an image data group; and a step of accepting assignment of labels to image data included in the clusters generated by the clustering.

[0033] This allows clustering of image data that has not been annotated in advance. Furthermore, since annotation is performed on image data included in a cluster, annotation can be performed efficiently while comparing similar images.

[0034] According to the present invention, it is possible to provide a technology for efficiently performing annotation while comparing with similar images.

[0035] FIG. 1 is a functional block diagram of an annotation support device according to an embodiment of the present invention. FIG. 2 is a diagram showing a hardware configuration of the annotation support device according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating a procedure for a classifier generation process according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a procedure for an annotation support process according to an embodiment of the present invention. FIG. 5 is a diagram showing a display example of a typical non-defective product selection screen according to an embodiment of the present invention. FIG. 6 is a diagram explaining the concept of clustering according to an embodiment of the present invention. FIG. 7 is a diagram showing a display example of an annotation screen according to an embodiment of the present invention. FIG. 8 is a diagram showing a display example of an area selection screen according to an embodiment of the present invention. FIG. 9 is a diagram showing a display example of a test result screen according to an embodiment of the present invention.

[0036] [Application Examples] Application examples of the present invention will be described below with reference to the drawings.

[0037] Fig. 1 shows a functional block diagram of an annotation support device 100. Fig. 4 is a flowchart illustrating an annotation support method.

[0038] The annotation support device 100 includes an input unit 10, a display unit 20, a processing unit 30, a data management unit 40, and an annotation data storage unit 50, and can be configured using a general computer device. The processing unit 30 also includes a preprocessing unit 31, a learning processing unit 32, a reference information assignment processing unit 33, an automatic annotation unit 34, a grouping processing unit 35, an ROI processing unit 36, and a screen information generation unit 37.

[0039] First, in response to a user instruction via the input unit 10, the data management unit 40 reads the image data group 60 (step S11).

[0040] Next, the images included in the loaded image data group 60 are displayed on the display unit 20, and the user is prompted to select 20 images of typical non-defective products (hereinafter referred to as "typical non-defective product images") (step S12). Fig. 5 shows an example of a non-defective product selection screen 201 generated by the screen information generating unit 37 and displayed on the display unit 20. The user switches between images to be displayed in the image display area 202 by operating scroll bars 203 and 204, selects one of the images displayed in the image display area 202 by right-clicking the mouse, or the like, and then selects an "OK" button 205 displayed on the non-defective product selection screen 201 to confirm the selection of the typical non-defective product image.

[0041] In parallel with the user's selection of a typical non-defective image in step S2, the pre-processing unit 31 of the processing unit 30 performs pre-processing on the image data of the read image data group 60 to remove resist and wiring pattern colors (step S13).

[0042] Next, the learning processing unit 32 of the processing unit 30 learns the typical good image data selected by the user in step S2 (step S14), and generates a model for the input image data whose output value is the similarity to the typical good image data.

[0043] Next, image data whose output value is equal to or less than a predetermined threshold is removed from the data population (step S15). The output value becomes smaller as the image is closer to a typical non-defective image. By removing image data whose output value is equal to or less than the predetermined threshold from the data population in this way, it is possible to remove obviously non-defective products from the annotation target, thereby improving the efficiency of annotation.

[0044] Next, the grouping processing unit 35 of the processing unit 30 groups the remaining image data after excluding the image data whose output value is equal to or less than the predetermined threshold value from the population in step S5 (step S16).

[0045] Next, the user annotates the image data grouped in step S6 for each group. FIG. 7 shows an example of an annotation screen 206 displayed on the display unit 20. Images IM11 to IM19 are displayed in the image display area 207. An output value 217 is also displayed in the upper right corner of each image IM11 to IM19. When the user moves the pointer over each image IM11 to IM19 displayed in the image display area 207 and left-clicks the mouse, the label for that image changes from OK to NG to Gray, allowing the user to select the desired label. Images IM11 to IM19 are displayed in the image display area 207 in order of output value. When the output value indicates similarity, the images are displayed in order of similarity, so that images with output values ​​above a certain value can be collectively determined as NG or Gray.

[0046] The image IM17 displayed in the image display area 207 is highlighted with a different background color (HL). The highlighted display HL is applied to a predetermined percentage (10% in this case) of images that are outside the center of the cluster.

[0047] As described above, the user switches between groups of images displayed in the image display area 207 and performs annotations on at least some of the images in each group.

[0048] If there are images to which the user has not assigned a label, the user can select whether or not to regroup the unlabeled images (step S18). By pressing the above-mentioned "Regroup unannotated images" button 222, the grouping processing unit 35 of the processing unit 30 regroups the unlabeled images, and the user can assign labels to the regrouped images.

[0049] In this way, by performing annotation using the annotation support device 100, similar images can be annotated consecutively, thereby reducing the time required for annotation work.

[0050] [First Embodiment] An annotation support device 100 according to a first embodiment of the present invention will be described in more detail below with reference to the accompanying drawings. However, the configurations of the device and system described in this embodiment should be modified as appropriate depending on various conditions. In other words, the scope of the present invention is not intended to be limited to the following embodiment.

[0051] Fig. 1 shows a functional block diagram of the annotation support device 100. Fig. 2 shows a hardware configuration diagram of the annotation support device 100. Fig. 3 is a flowchart explaining a process for generating a classifier by machine learning. Fig. 4 is a flowchart explaining an annotation support method.

[0052] When a judger that judges whether a component or land is good or bad based on an image of the component or land is generated by machine learning, the process proceeds according to the flowchart shown in FIG.

[0053] First, images of target areas such as components and lands are collected (step S1). Next, the collected images are annotated to assign pass / fail labels (step S2). Learning data is extracted from the annotated image data, and the extracted learning data is machine-learned into a learning model (step S3). This generates a classifier that outputs a pass / fail judgment result from the input image. Next, the performance of the generated classifier is tested using test data extracted from the annotated image data (step S4). If the performance of the generated classifier is sufficient and there are no problems with the test results, the trained classifier is applied to teaching software and put into actual use (step S5). If the performance of the generated classifier is not sufficient and there are problems with the test results, return to step S2 and redo the annotation.

[0054] To generate a high-performance classifier, it is necessary to prepare a large amount of high-quality training data, and for this purpose, annotation work, which assigns labels to collected image data, plays an important role.

[0055] The annotation support device 100 described below supports such annotation and efficiently generates high-quality training data.

[0056] The annotation support device 100 includes an input unit 10 , a display unit 20 , a processing unit 30 , a data management unit 40 , and an annotation data storage unit 50 .

[0057] The annotation support device 100 is configured by a general-purpose computer device including a processor 101 such as a CPU, a main memory unit (memory) 102, an auxiliary memory unit (hard disk, etc.) 103, an input unit (keyboard, mouse, controller, touch panel, etc.) 10, a display unit (display) 20, a communication unit 104, etc. The annotation support device 100 may be a computer system in which multiple computers work together. The processing unit 30 and the data management unit 40 are realized by expanding a program stored in the auxiliary memory unit 103 into the main memory unit and executing it in the processor 101.

[0058] 4, the annotation support method executed in the annotation support device 100 will be described. First, in response to a user's instruction via the input unit 10, the data management unit 40 reads the image data group 60 (step S11). The image data group 60 may be read from an external storage device connected via a network via the communication unit 104, or may be read from the auxiliary storage unit 103.

[0059] Next, the images included in the loaded image data group 60 are displayed on the display unit 20, and the user is prompted to select 20 images of typical non-defective products (hereinafter referred to as "typical non-defective product images") (step S12). FIG. 5 shows an example of a non-defective product selection screen 201 displayed on the display unit 20. Approximately 1,000 images included in the image data group 60 are displayed, and the user is prompted to select an image that is determined to be a typical non-defective product from among them. In FIG. 5, nine images, images IM1 to IM9, are displayed in the image display area 202. The user can switch between images displayed in the image display area 202 by operating scroll bars 203 and 204, select one of the images displayed in the image display area 202 by right-clicking the mouse, or the like, and then confirm the selection of the typical non-defective product image by selecting an "OK" button 205 displayed on the non-defective product selection screen 201. The number of typical non-defective product images to be selected is an example, and the number of typical non-defective product images to be selected can be input or selected using an input field or a spinner-type standard setting receiving unit, allowing an appropriate number of images to be selected. At this stage, the selection of typical non-defective product images is performed for each target part. When the user selects images, an indicator indicating the appropriateness of the selection may be displayed. One such indicator is, for example, an indicator indicating the similarity of the selected images. If images with similarity index scores are selected, it can be confirmed that the selection is appropriate. The non-defective product selection screen 201 corresponds to the standard data receiving unit of the present invention. Furthermore, the typical non-defective product images correspond to the standard image data group and standard image data of the present invention. The indicator indicating the appropriateness of the selection corresponds to the auxiliary indicator of the present invention.

[0060] In parallel with the user's selection of typical non-defective images in step S2, the preprocessing unit 31 of the processing unit 30 performs preprocessing on the image data of the read image data group 60 to remove resist and wiring pattern colors (step S13). By removing parts unnecessary for judgment, such as resist and wiring pattern colors, the accuracy of judgment can be improved. Subsequent processing is performed on the preprocessed image data. Here, the removal of resist and wiring pattern colors corresponds to the predetermined image processing for clustering in this invention.

[0061] Next, the learning processing unit 32 of the processing unit 30 learns the typical non-defective image data selected by the user in step S2 (step S14) and generates a model for the input image data, in which the similarity to the typical non-defective image data is used as an output value. An autoencoder, for example, can be used as the learning model, but is not limited to this. The similarity to the typical non-defective image data may be calculated by calculating the average distance using a two-image similarity calculation method such as SSIM (Structural Similarity). Here, the similarity or the average distance corresponds to an index indicating the similarity or a reference value indicating the feature.

[0062] Next, the reference information assignment processing unit 33 of the processing unit 30 calculates output values ​​such as similarity of the image data included in the image data group 60 using a trained model or other methods, and the automatic annotation unit 34 removes image data whose output values ​​are below a predetermined threshold from the data population (step S15). The output value becomes smaller the closer the image is to a typical good-quality image. In this way, by removing image data whose output values ​​are below a predetermined threshold from the data population, it is possible to remove clearly good-quality products from the annotation targets, thereby improving annotation efficiency. Images whose output values ​​are below a predetermined threshold can be collectively labeled "OK," eliminating the need for individual annotation work. The threshold can be set to, for example, an average value, or the user may be allowed to set an appropriate threshold.

[0063] Next, in step S5, the grouping processing unit 35 of the processing unit 30 removes image data whose output value is equal to or less than a predetermined threshold from the population and groups (clusters) the remaining image data (step S16). FIG. 6 is a diagram conceptually illustrating grouping by the grouping processing unit 35. Here, an example is described in which the grouped image data are arranged in a two-dimensional feature space. Here, a typical non-defective image GIM with no defective features is located near the origin of the feature space. Each colored circle IM corresponds to one image, and a circle (ellipse) CL containing multiple circles corresponds to one group (cluster). Arrow Ar1 indicates the output value, represented by the distance from the typical non-defective image in the feature space. Arrow Ar2 indicates the distance from the cluster center C of the image belonging to cluster CL, and highlighting is performed according to the distance from the cluster center C, as described below. The grouping processing unit 35 corresponds to the clustering unit of the present invention.

[0064] Next, the user annotates the image data grouped in step S6 for each group. FIG. 7 shows an example of an annotation screen 206 generated by the screen information generating unit 37 based on the grouping results and displayed on the display unit 20. Here, the group name 210 containing the image data displayed in the image display area 207 is displayed as "Group A" above the image display area 207. Also displayed is the group number 211 of all groups, "1 / 20" (the first group out of a total of 20 groups). A "Previous Group" button 212 and a "Next Group" button 213 are displayed for switching the group of image data displayed in the image display area 207 to the previous group or the next group, respectively. Below the group name 210, an "OK" button 214, an "NG" button 215, and a "Gray" button 216 are displayed for collectively labeling the image data included in the group with "OK," "NG," or "Gray." The annotation screen 206 corresponds to the annotation receiving unit of the present invention.

[0065] The image display area 207 displays each of the images IM11 to IM19. An output value 217 is also displayed in the upper right corner of each of the images IM11 to IM19. When the user moves the pointer over each of the images IM11 to IM19 displayed in the image display area 207 and left-clicks the mouse, the label of the image changes in sequence from OK to NG to Gray, allowing the user to select the desired label. The selected label 218 is displayed as "OK" in the upper left corner of each of the images IM11 to IM19. The image display area 207 displays the output value 217 of each of the images IM11 to IM19, and the images IM11 to IM19 are arranged and displayed in order of their output values. When the output value indicates similarity, the images are displayed in order of similarity, so that images with output values ​​above a certain value can be collectively judged as NG or Gray. Here, the images are displayed in the image display area 207 in order of output value, but the user may be allowed to select whether to arrange the images in order of output value, or in other manners such as in order of scores based on past judgment models. Again, the images displayed in the image display area 207 can be changed by operating the scroll bars 219 and 220. Here, the output value corresponds to the reference value indicating the characteristics of the image data of the present invention, and the annotation screen 206 corresponds to the reference value providing unit of the present invention.

[0066] Image IM17 displayed in the image display area 207 is highlighted with a different background color (HL). As described above, the highlighting HL is performed on a predetermined percentage of images that are off the center of the cluster. On the right side of the image display area 207, a slider 221 is displayed that allows the user to set the percentage of images to be highlighted. In this example, it is set to highlight 10% of the images that are off the center of the cluster. The different background colors used for the highlighting HL correspond to the display mode of the present invention. The display mode that changes depending on the distance from the center of the cluster is not limited to this.

[0067] On the right side of the image display area 207, a "Regroup Unannotated Images" button 222 and a "Save Results" button 223 are displayed. When the user presses the "Save Results" button 223, the annotations made by the user are confirmed (step S17). The confirmed annotation results are stored in the annotation data storage unit 50 by the data management unit 40. The annotation data storage unit 50 stores the assigned labels in association with image IDs that identify the image data. The user does not need to assign labels to all images in each group, and may confirm annotations for only some of the grouped images. When the user wants to regroup images to which no labels have been assigned, the user presses the "Regroup Unannotated Images" button 222. The "Regroup Unannotated Images" button 222 corresponds to the reclustering reception unit of the present invention.

[0068] A user may set a portion of an image as a region of interest (ROI) and have the region of interest displayed in an enlarged form in the image display area 235. FIG. 8 shows an example of a region of interest setting acceptance screen 230. The user can select a region of interest 233 by, for example, surrounding a portion of an enlarged image 231 displayed as a pop-up with a rectangular frame 232. The setting of the region of interest 233 is confirmed by pressing an "OK" button 234. The ROI processing unit 36 ​​of the processing unit 30 generates images IM21 to IM29 of the region of interest based on this setting, and the screen information generating unit 37 displays the region of interest setting acceptance screen 230, on the display unit 20, in which the images IM21 to IM29 of the region of interest are displayed in the image display area 235. The setting of the region of interest 233 is not limited to when the user selects the desired range, but may also be set by the ROI processing unit 36 ​​of the processing unit 30 recommending an appropriate region of interest 233 and the user approving the recommended region of interest 233.

[0069] As described above, the user switches between groups of images displayed in the image display area 207 and performs annotations on at least some of the images in each group.

[0070] If there are images to which the user has not assigned a label, the user can select whether or not to regroup the unlabeled images (step S18). To regroup the unlabeled images, the user presses the above-mentioned "Regroup Unannotated Images" button 222. When the "Regroup Unannotated Images" button 222 is pressed, the process returns to step S6, and the grouping processing unit 35 of the processing unit 30 regroups the unlabeled images. The regrouped images are then displayed on the display unit 20 in the same manner as in FIG. 7, and the user can assign labels to the regrouped images (step S17).

[0071] When all the data have been labeled, the annotation is complete (step S9).

[0072] As described above, training data and test data are extracted from the annotated image data. Then, the training data is subjected to machine learning by a determination model (step S3), and the test data is input to the trained determination model to perform a test (step S4).

[0073] 9 shows an example of a test result screen 240. A judgment summary 241 is displayed on the test result screen 240. The summary 241 displays, in a table format, the number of data judged as OK and the number of data judged as NG for each of OK data, NG data, and gray data. Here, 1,000 of the OK data items have been judged as OK, and 20 have been judged as NG. Furthermore, of the NG data items, 0 have been judged as OK, and 100 have been judged as NG. Furthermore, for gray data, there are 0 cases of both OK and NG judgments.

[0074] The test result screen 240 displays a detailed result field 242 for each image. If the test results indicate a problem, i.e., if the performance of the generated judgment model is insufficient, the detailed result field 242 displays the target image IM31, its output value 243, and a "Go" button 244 to the annotation screen 206. Pressing the "Go" button 244 moves the user to the annotation screen 206 for that image, allowing them to change the label assigned. The test result screen 240 also displays a histogram 245 showing OK and NG measurement values. This histogram 245 allows users to confirm whether the OK measurement values ​​245a and the NG measurement values ​​245b are separated. The test result screen 240 also displays an "Apply Model" button 246. If the test results indicate that there are no problems with the model generated by the AI ​​learning tool, the user can press the "Apply Model" button 246 to apply the generated model to the teaching software (step S6).

[0075] In addition, the following describes the components of the present disclosure with reference to the reference numerals in the drawings to enable comparison between the components of the present disclosure and the configurations of the embodiments. <Supplementary Note 1> An annotation support device (100) comprising: a clustering unit (35) that clusters image data included in an image data group (60); and an annotation receiving unit (206) that accepts assignment of labels to image data included in the clusters generated by the clustering. <Supplementary Note 2> The annotation support device (100) according to Supplementary Note 1, further comprising: a reference value providing unit (201) that provides a reference value indicating a characteristic of the image data when accepting assignment of the label to the image data included in the cluster. <Supplementary Note 3> The annotation support device (100) according to Supplementary Note 2, characterized in that the annotation receiving unit (206) arranges the image data included in the cluster according to the reference value. <Supplementary Note 4> The annotation support device (100) according to Supplementary Note 2 or 3, further comprising: a reference data accepting unit that accepts selection of image data from the image data group (60) to be assigned a predetermined label as a reference image data group, wherein the reference value is an index indicating similarity with reference image data included in the reference image data group. <Supplementary Note 5> The annotation support device (100) according to Supplementary Note 4, further comprising: an automatic annotation unit (34) that assigns the predetermined label to the image data included in the image data group based on the index indicating similarity with reference image data included in the reference image data group. <Supplementary Note 6> The annotation support device (100) according to Supplementary Note 4 or 5, further comprising: when accepting the selection of the reference image data group, the reference data accepting unit (201) provides an auxiliary index indicating the appropriateness of the selection. <Supplementary Note 7> The annotation support device (100) according to any one of Supplements 1 to 6, further comprising: a preprocessing unit (31) that performs predetermined image processing for the clustering on the image data included in the image data group (60).<Supplementary Note 8> The annotation support device (100) according to any one of Supplements 1 to 7, characterized in that the annotation receiving unit (206) collectively receives the assignment of the label to the plurality of image data included in the cluster. <Supplementary Note 9> The annotation support device (100) according to any one of Supplements 4 to 6, characterized in that it includes a reference setting receiving unit that receives a setting of the number of image data to be selected as the reference image data group. <Supplementary Note 10> The annotation support device (100) according to any one of Supplements 1 to 9, characterized in that the annotation receiving unit (206) changes the display mode of the image data included in the cluster depending on the distance from the center of the cluster. <Supplementary Note 11> The annotation support device (100) according to any one of Supplements 1 to 10, characterized in that the annotation receiving unit (206) receives a setting of a region of interest in the image data, and displays the region of interest in the image data. <Supplementary Note 12> The annotation support device (100) according to any one of Supplements 1 to 11, further comprising: a reclustering receiving unit (222) that receives re-clustering of image data from the image data group (60) to which the label has not been assigned, wherein the annotation receiving unit (206) receives assignment of the label to image data included in the clusters generated by the re-clustering. <Supplementary Note 13> The annotation support device (100) according to any one of Supplements 1 to 12, further comprising: the annotation receiving unit (206) provides test results of a learning model generated using the annotated image data group as training data, and receives changes to the assigned label. <Supplementary Note 14> An annotation support method including: a step of clustering image data included in the image data group (60) (step S16); and a step of receiving assignment of a label to image data included in the clusters generated by the clustering (step S17).

[0076] 35: Grouping processing unit 60: Image data group 100: Annotation support device 206: Annotation screen

Claims

1. An annotation support device comprising: a clustering unit that clusters image data included in an image data group; and an annotation receiving unit that receives the assignment of labels to image data included in the clusters generated by the clustering.

2. The annotation support device according to claim 1, further comprising a reference value providing unit that provides a reference value indicating the characteristics of the image data included in the cluster when accepting the assignment of the label to the image data.

3. The annotation support device according to claim 2, characterized in that the annotation receiving unit arranges the image data included in the cluster according to the reference value.

4. An annotation support device as described in claim 2 or 3, characterized in that it is provided with a reference data accepting unit that accepts a selection of image data from the image data group to which a predetermined label is to be assigned as a reference image data group, and the reference value is an index indicating the degree of similarity with the reference image data included in the reference image data group.

5. The annotation support device described in claim 4, further comprising an automatic annotation unit that assigns a predetermined label to the image data included in the image data group based on the index indicating the similarity with reference image data included in the reference image data group.

6. The annotation support device according to claim 4, characterized in that the reference data receiving unit, when receiving a selection of the reference image data group, provides an auxiliary indicator representing the appropriateness of the selection.

7. An annotation support device according to any one of claims 1 to 3, further comprising a pre-processing unit that performs predetermined image processing for the clustering on the image data included in the image data group.

8. An annotation support device according to any one of claims 1 to 3, characterized in that the annotation receiving unit receives the assignment of the labels collectively for multiple image data included in the cluster.

9. The annotation support device according to claim 4, further comprising a reference setting receiving section that receives a setting of the number of the image data to be selected as the reference image data group.

10. The annotation support device according to any one of claims 1 to 3, characterized in that the annotation receiving unit changes the display mode of the image data contained in the cluster depending on the distance from the center of the cluster.

11. The annotation support device according to any one of claims 1 to 3, characterized in that the annotation receiving unit receives settings of an area of ​​interest in the image data, and displays the area of ​​interest in the image data.

12. An annotation support device as described in any one of claims 1 to 3, characterized in that it further comprises a re-clustering reception unit that receives re-clustering for image data among the image data group to which the label has not been assigned, and the annotation reception unit accepts the assignment of the label to image data included in the cluster generated by the re-clustering.

13. The annotation support device according to any one of claims 1 to 3, characterized in that the annotation receiving unit provides test results of a learning model generated using the annotated image data group as learning data, and accepts changes to the assigned labels.

14. An annotation support method comprising: a step of clustering image data included in an image data group; and a step of accepting assignment of labels to image data included in the clusters generated by the clustering.

Images