Learning Support Device, Learning Support Method, and Learning Support Program

Abstract

To provide a learning support device capable of ensuring the quality of annotations for AI models for image recognition.SOLUTION: A learning support device in an embodiment includes an extraction unit, a creation unit, and a selection unit. The extraction unit extracts the edge of an object in a target image of annotation. The creation unit creates a closed edge rectangle tangent to the closed edge which is an edge extracted as a closed curve out of the edges extracted by the extraction unit. The selection unit selects a closed-edge rectangle with the highest similarity by calculating the similarity between the input rectangle input by the user for the target image and the closed-edge rectangle.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The disclosed embodiments relate to a learning support device, a learning support method, and a learning support program.

Background Art

[0002] Conventionally, when training an AI (Artificial Intelligence) model using algorithms such as deep learning, annotation, which is the task of creating teacher data that serves as positive examples, is known (see, for example, Patent Document 1).

[0003] Among such annotations, for an annotation for an AI model for image recognition, a rectangle called a bounding box (hereinafter referred to as "BB") that encloses each object in the image is created through so-called VDT (Visual Display Terminals) work performed manually. In addition, metadata indicating the name, attributes, etc. of each object is assigned to each created BB.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the prior art still has room for further improvement in ensuring the quality of annotations.

[0006] For example, to improve the accuracy of an AI model, it is necessary to collect a large number of images and perform annotations on each of these images. However, in the above-described VDT work, there are problems such as extremely large man-hours and increased mental and physical fatigue of workers.

[0007] In addition, in particular, the above-mentioned BB is preferably created so as not to include extra areas other than the object as much as possible in order to improve the quality as teacher data. However, this is not easy by manual operation, and there is a risk that the quality will depend on the skill level of the operator.

[0008] One aspect of the embodiment is made in view of the above, and an object is to provide a learning support device, a learning support method, and a learning support program that can ensure the quality of annotation for an AI model for image recognition.

Means for Solving the Problems

[0009] The learning support device according to one aspect of the embodiment includes Controller and is provided with. The above Controller is In teacher data creation Extract the edges of the object in the annotation target image and Among the extracted edges, create a closed edge rectangle that touches the closed edge, which is the edge extracted as a closed curve and Calculate the similarity between the input rectangle input by the user for the target image and the closed edge rectangle, and select the closed edge rectangle with the largest similarity and adopt the selected closed-edge rectangle as the bounding box for the annotation to do.

Effects of the Invention

[0010] According to one aspect of the embodiment, the quality of annotation for an AI model for image recognition can be ensured.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments of the learning support device, learning support method, and learning support program disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments shown below.

[0013] First, an overview of the learning support method according to the embodiment will be described with reference to FIGS. 1 to 5. FIG. 1 is a diagram showing a schematic configuration of the learning support device 10 according to the embodiment. FIGS. 2 to 5 are schematic explanatory diagrams (parts 1) to (4) of the learning support method according to the embodiment.

[0014] The learning support device 10 is a computer used by a user U who is an annotation worker when annotating an AI model for image recognition. The learning support device 10 is, for example, a desktop or notebook PC (Personal Computer), a tablet terminal, a smartphone, a server, a workstation, or the like.

[0015] As shown in FIG. 1, the learning support device 10 according to the embodiment has an HMI (Human Machine Interface) unit 3. The learning support device 10 also has a target image DB 11a and a rectangular information DB 11e.

[0016] The HMI unit 3 is a component that provides interface components for the user U. The HMI unit 3 includes an input unit 3a and an output unit 3b.

[0017] The input unit 3a is an input device that receives input operations from the user U, and is realized by, for example, a keyboard, a mouse, a tablet, a touch panel, etc. Note that the input unit 3a may be realized by software components.

[0018] The output unit 3b is an output device that displays and outputs the target image for annotation and BBs etc. input on such a target image, and is realized by a display etc. Note that it may be configured integrally with the input unit 3a by a touch panel display.

[0019] The target image DB 11a is a database in which each image to be the target of annotation is stored. The rectangular information DB 11e is a database in which rectangular information regarding the position, size, etc. of BBs created on each image in annotation is stored.

[0020] The user U performs annotation on each image stored in the target image DB 11a through VDT work via the HMI unit 3, and as a result, the rectangular information regarding the created BBs is stored in the rectangular information DB 11e.

[0021] By the way, existing technologies related to such annotation have room for further improvement in ensuring the quality of annotation.

[0022] For example, in order to improve the accuracy of the AI model, it is necessary to collect a large number of images and perform annotation on each of these images. However, in VDT work, there is a problem that it takes a huge amount of man-hours and the physical and mental fatigue of the user U also increases.

[0023] In addition, particularly for BB, in order to improve the quality as teacher data, it is preferably created so as not to include extra areas other than the object as much as possible. However, this is not easy by manual operation, and there is a risk that the quality will depend on the proficiency of the user U.

[0024] Therefore, in the learning support method according to the embodiment, the edges of the object in the target image for annotation are extracted, and among the extracted edges, a closed-edge rectangle that is in contact with the closed edge, which is the edge extracted as a closed curve, is created. The similarity between the input rectangle input by the user U for the target image and the closed-edge rectangle is calculated, and the closed-edge rectangle with the largest similarity is selected.

[0025] Specifically, as shown in FIG. 2, in the learning support method according to the embodiment, first, the learning support device 10 creates an input rectangle IR by the input operation of the user U for the target image for annotation (step S1). As shown in the figure, the user U designates the input rectangle IR by, for example, a region selection operation using a mouse or the like from the start point P1 to the end point P2 on the target image.

[0026] On the other hand, as shown in FIG. 3, the learning support device 10 extracts edges in advance for the same target image (step S2). For such edge extraction, a known algorithm such as the Canny edge detector may be used, or an original algorithm may be used.

[0027] Then, as shown in FIG. 4, the learning support device 10 automatically creates a rectangle in contact with such a closed edge for the extracted edges that are extracted as a closed curve, that is, a closed edge (hereinafter referred to as a "closed edge") with both ends matching (step S3). The rectangle in contact with such a closed edge (hereinafter referred to as a "closed-edge rectangle") is shown by a broken line in the figure.

[0028] Then, as shown in the figure, the learning support device 10 selects, from among the closed edge rectangles, the closed edge rectangle that is most similar to the input rectangle IR as the automatically adjusted rectangle AR (step S4). The method of selecting the most similar closed edge rectangle will be described later with reference to FIG. 8 and subsequent figures.

[0029] Then, as shown in FIG. 5, the learning support device 10 displays, on the output unit 3b, the rectangle AR after the automatic adjustment, as well as a dialog inquiring of the user U as to whether or not to adopt the rectangle AR (step S5).

[0030] As shown in the figure, if the user U selects "Yes", the learning support device 10 adopts the adjusted rectangle AR as BB. On the other hand, if the user U selects "No", the learning support device 10 adopts the rectangle before adjustment, i.e., the input rectangle IR, as BB.

[0031] In this way, in the learning support method of the embodiment, the edges of an object in the target image for annotation are extracted, and a closed edge rectangle is created that is tangent to a closed edge, which is an edge extracted as a closed curve from among the extracted edges. The similarity between the input rectangle IR input to the target image by user U and the closed edge rectangle is calculated, and the closed edge rectangle with the greatest similarity is selected.

[0032] Therefore, according to the learning support method of the embodiment, it is possible to significantly reduce the man-hours required for creating BBs, which in the existing technology required the user U to create BBs one by one based on his / her own senses and judgments for each image collected in large quantities. This also makes it possible to reduce the mental and physical fatigue of the user U in VDT ​​work.

[0033] In addition, according to the learning support method of the embodiment, a closed edge rectangle that is automatically created using image analysis technology including edge extraction and that is adjacent to the boundary of an object in an image can be adopted as the BB, thereby reducing the influence of the user U's level of proficiency on the quality of the training data.

[0034] That is, according to the learning support method according to the embodiment, the quality of annotations for an AI model for image recognition can be ensured. Hereinafter, a configuration example of the learning support device 10 to which the learning support method according to the embodiment is applied will be described more specifically.

[0035] FIG. 6 is a block diagram showing a configuration example of the learning support device 10 according to the embodiment. Note that in FIG. 6, only the components necessary for explaining the features of the embodiment are shown, and the description of general components is omitted.

[0036] In other words, each component illustrated in FIG. 6 is a functional concept, and it is not necessarily physically configured as illustrated. For example, the specific form of the distribution and integration of each block is not limited to that illustrated, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads, usage situations, and the like.

[0037] Also, in the description using FIG. 6, the description of the components that have already been described may be simplified or omitted.

[0038] As shown in FIG. 6, the learning support device 10 according to the embodiment includes a storage unit 11 and a control unit 12. Further, the learning support device 10 is connected to the HMI unit 3 via wired or wireless, or directly.

[0039] Since the HMI unit 3 has already been described, the description here is omitted. The storage unit 11 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. In the example of FIG. 6, the storage unit 11 stores the target image DB 11a, the extraction algorithm information 11b, the edge information 11c, the similarity calculation information 11d, and the rectangle information DB 11e.

[0040] Since the target image DB11a and the rectangle information DB11e have been described, the description here is omitted. The extraction algorithm information 11b is library information of the algorithm used in the edge extraction process executed by the edge extraction unit 12ba described later.

[0041] The edge information 11c stores information about the edges extracted by the edge extraction unit 12ba. The similarity calculation information 11d stores algorithms and various parameters that serve as the criteria for calculating the similarity between the input rectangle IR and the closed edge rectangle. The similarity calculation information 11d can be, for example, pre-selected by the user U and set in advance as static information.

[0042] The control unit 12 is a controller and is realized, for example, by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc., when various programs (not shown) stored in the storage unit 11 are executed using the RAM as a work area. Also, the control unit 12 can be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0043] The control unit 12 has an image drawing unit 12a, an image analysis unit 12b, and a rectangle creation unit 12c, and realizes or executes the functions and operations of the information processing described below.

[0044] The image drawing unit 12a outputs the annotated target image stored in the target image DB11a to the output unit 3b. The image drawing unit 12a includes a rectangle drawing unit 12aa. The rectangle drawing unit 12aa draws the input rectangle IR input via the input unit 3a on the target image.

[0045] The image analysis unit 12b performs image analysis processing on the target image. The image analysis unit 12b includes an edge extraction unit 12ba. The edge extraction unit 12ba performs edge extraction processing on the target image based on the extraction algorithm information 11b.

[0046] Note that, in addition to the Canny method described above, the edge extraction unit 12ba can use Sobel, Laplacian, etc. as known algorithms for edge extraction.

[0047] The edge extraction unit 12ba may execute edge extraction processing using an algorithm arbitrarily selected by the user U from among these known algorithms and the original algorithm.

[0048] By extracting the boundaries of the objects in the image through such image analysis processing, a certain quality can be maintained without depending on the user U's sense and judgment. Also, by enabling the arbitrary selection of algorithms, it becomes possible to realize functions independent of the characteristics of the training data. For example, it is possible to appropriately use different algorithms depending on the color and shape of the object, or the brightness and saturation of the image, and accurately extract the edges of the object.

[0049] The rectangle creation unit 12c includes a closed edge rectangle creation unit 12ca and a most similar rectangle selection unit 12cb. The closed edge rectangle creation unit 12ca creates a closed edge rectangle that touches the closed edge based on the edge information 11c extracted by the edge extraction unit 12ba.

[0050] Here, FIG. 7 is an explanatory diagram of the closed edge rectangle creation process. As shown in FIG. 7, the closed edge rectangle creation unit 12ca obtains the minimum horizontal position X1, the maximum horizontal position X2, the minimum vertical position Y1, and the maximum vertical position Y2 of the closed edge Ce from the edge information 11c.

[0051] Then, as shown in the same figure, the closed edge rectangle creation unit 12ca creates a closed edge rectangle R using these four coordinates (X1, Y1), (X1, Y2), (X2, Y1), and (X2, Y2) defined thereby.

[0052] As a result, it is no longer necessary for the user U to adjust the BB one by one according to the boundary of the object, the man-hours of the VDT work can be significantly reduced, and the burden on the user U can be reduced.

[0053] Returning to the description of FIG. 6. The most similar rectangle selection unit 12cb calculates the similarity between each closed-edge rectangle R created by the closed-edge rectangle creation unit 12ca and the input rectangle IR based on the similarity calculation information 11d, and selects the closed-edge rectangle R with the maximum similarity.

[0054] The algorithm for calculating the similarity can be arbitrarily selected by the user U, similar to the case of edge extraction described above. The most similar rectangle selection unit 12cb uses, for example, the aspect ratio, the center position, the width / height, etc. as elements for measuring the degree of similarity. Then, the most similar rectangle selection unit 12cb obtains the similarity from the sum of the squares of the differences of each element between the input rectangle IR and each closed-edge rectangle R, and selects the closed-edge rectangle R with the maximum similarity.

[0055] Here, a specific example of the most similar rectangle selection process will be described with reference to FIGS. 8 to 12. FIGS. 8 to 12 are diagrams (Part 1) to (Part 5) showing a specific example of the most similar rectangle selection process.

[0056] First, as shown in FIG. 8, assume that the input rectangle IR with a center position of (4, 3), a width of 8, and a height of 6 contains closed edges in the shape of a star, a circle, and a pentagon.

[0057] In such a case, as shown in FIG. 9, when only the aspect ratio is compared, the most similar rectangle selection unit 12cb selects the closed-edge rectangle R1 whose aspect ratio is substantially the same as that of the input rectangle IR among the closed-edge rectangle R1 in contact with the closed edge of the star, the closed-edge rectangle R2 in contact with the closed edge of the circle, and the closed-edge rectangle R3 in contact with the closed edge of the pentagon.

[0058] Also, as shown in FIG. 10, when only the center positions are compared, the most similar rectangle selection unit 12cb selects the closed-edge rectangle R2 among the closed-edge rectangles R1, R2, and R3, whose center position is substantially the same as that of the input rectangle IR.

[0059] Also, as shown in FIG. 11, when only the width / height ratios are compared, the most similar rectangle selection unit 12cb selects the closed-edge rectangle R3 among the closed-edge rectangles R1, R2, and R3, whose width / height ratio is most similar to that of the input rectangle IR.

[0060] Also, as shown in FIG. 12, assume that for the input rectangle IR1, for example, only a part of the closed edge is included and no closed edge is detected within the input rectangle IR1. In such a case, as shown in the same figure, the most similar rectangle selection unit 12cb expands the input rectangle IR1 by a certain amount, detects a closed edge within the expanded input rectangle IR2, and creates a closed-edge rectangle R1.

[0061] In this way, by assisting in determining the boundary of the object from inside and outside the input rectangle IR, it becomes unnecessary to create a precise input rectangle IR, and the burden on the user U can be reduced.

[0062] By the way, in the description using FIG. 5, an example of a screen UI (User Interface) that asks the user U whether to adopt the rectangle AR after automatic adjustment for the input rectangle IR with "Yes" or "No" was given, but the screen UI is not limited to this.

[0063] Hereinafter, various specific examples of the screen UI will be described with reference to FIGS. 13 to 26. FIGS. 13 to 26 are diagrams (Part 1) to (Part 14) showing specific examples of the screen UI.

[0064] As shown in FIG. 13, for example, the learning support device 10 may automatically adopt the rectangle AR after automatic adjustment as BB for the input rectangle IR input by the user U without asking about the above-mentioned adoption or not.

[0065] Also, as shown in FIG. 5, as shown in FIG. 14, for example, the learning support device 10 may inquire whether to adopt the automatically adjusted rectangle AR for the input rectangle IR input by the user U.

[0066] In such a case, as shown in FIG. 15, if "Yes" is selected by the user U, the learning support device 10 will adopt the automatically adjusted rectangle AR as BB. On the other hand, as shown in FIG. 16, if "No" is selected by the user U, the learning support device 10 will adopt the input rectangle IR as BB.

[0067] Also, as shown in FIG. 17, assume that the input rectangle IR input by the user U contains a plurality of closed edges. Note that such a case includes the case where the input rectangle IR is expanded. Also, here, assume that it contains two closed edges, a pentagon and a circle.

[0068] In such a case, as shown in the same figure, for example, the learning support device 10 may automatically adopt, as BB, the rectangle AR with the maximum similarity for the input rectangle IR input by the user U without inquiring about the above-mentioned adoption or not.

[0069] Also, in a similar case, as shown in FIG. 18, for example, the learning support device 10 may first inquire whether to adopt the first candidate rectangle AR1 with the maximum similarity for the input rectangle IR input by the user U.

[0070] In such a case, as shown in FIG. 19, if "Yes" is selected by the user U, the learning support device 10 will adopt the first candidate rectangle AR1 as BB. On the other hand, as shown in FIG. 20, if "No" is selected by the user U, the learning support device 10 will inquire whether to adopt the second candidate rectangle AR2 with the next highest similarity.

[0071] And in such a case, as shown in FIG. 21, if "Yes" is selected by the user U, the learning support device 10 will adopt the second candidate rectangle AR2 as BB. On the other hand, as shown in FIG. 22, if "No" is selected by the user U, the learning support device 10 will adopt the input rectangle IR as BB.

[0072] Also, for example, the learning support device 10 may be configured such that the flow shown in FIGS. 18 to 22 can be selected using the direction keys of the keyboard and adopted using the Enter key, as shown in FIG. 23.

[0073] Similarly, for example, the learning support device 10 may be configured to be selectable and adoptable by a click operation using a mouse or a tap operation using a finger, as shown in FIG. 24.

[0074] Next, consider the case where the input rectangle IR already contains an annotated rectangle AR. As shown in FIG. 25, assume that the input rectangle IR contains the annotated rectangles AR1 and AR2.

[0075] In such a case, as shown in the same figure, for example, the learning support device 10 selects and adopts a rectangle AR3 that has not yet been annotated by any of the methods shown in FIGS. 13 to 16.

[0076] Also, as shown in FIG. 26, assume that the input rectangle IR contains the annotated rectangle AR1 and a plurality of unannotated closed edges.

[0077] In such a case, as shown in the same figure, the learning support device 10 selects and adopts, for example, the rectangle AR2 from among the rectangles AR2 and AR3 that have not yet been annotated, by any of the methods shown in FIGS. 17 to 24.

[0078] Next, the processing sequence executed by the learning support device 10 will be described with reference to FIG. 27. FIG. 27 shows the processing sequence executed by the learning support device 10 according to the embodiment. Note that FIG. 27 shows the processing sequence until one BB is created in one target image.

[0079] First, when a target image for annotation is selected from the user U via the HMI unit 3 (step S101), the image drawing unit 12a displays the target image on the HMI unit 3 (step S102). Also, the image drawing unit 12a transmits the target image to the image analysis unit 12b.

[0080] The image analysis unit 12b causes the edge extraction unit 12ba to execute edge extraction processing based on the target image and the extraction algorithm information 11b in the storage unit 11 (step S103), and writes the edge information 11c, which is the processing result, to the storage unit 11.

[0081] Then, when a rectangular input is received from the user U via the HMI unit 3 (step S104), the image drawing unit 12a draws the input rectangle IR on the HMI unit 3 (step S105). Also, the image drawing unit 12a transmits input rectangle information regarding the input rectangle IR to the rectangle creation unit 12c.

[0082] The rectangle creation unit 12c causes the closed edge rectangle creation unit 12ca to execute closed edge rectangle creation processing based on the input rectangle IR and the edge information 11c in the storage unit 11 (step S106).

[0083] Then, the most similar rectangle selection unit 12cb executes most similar rectangle selection processing based on the processing result of the closed edge rectangle creation processing (step S107), and transmits most similar rectangle information regarding the most similar rectangle, which is the processing result, to the image drawing unit 12a. The image drawing unit 12a draws the most similar rectangle on the HMI unit 3 based on the received most similar rectangle information (step S108).

[0084] Then, when the image drawing unit 12a receives the selection of the rectangular AR to be adopted from the user U via the HMI unit 3 (step S109), it draws such a rectangular AR on the HMI unit 3 (step S110), and transmits the adopted rectangle information regarding the adopted rectangular AR to the rectangle creation unit 12c (step S111).

[0085] Then, the rectangle creation unit 12c writes the received adopted rectangle information into the rectangle information DB11e of the storage unit 11 (step S112) and ends the process.

[0086] As described above, the learning support device 10 according to the embodiment includes an edge extraction unit 12ba (corresponding to an example of an "extraction unit"), a closed edge rectangle creation unit 12ca (corresponding to an example of a "creation unit"), and a most similar rectangle selection unit 12cb (corresponding to an example of a "selection unit"). The edge extraction unit 12ba extracts the edges of the object in the annotation target image. The closed edge rectangle creation unit 12ca creates a closed edge rectangle that contacts the closed edge, which is an edge extracted as a closed curve, among the edges extracted by the edge extraction unit 12ba. The most similar rectangle selection unit 12cb calculates the similarity between the input rectangle IR input by the user U for the target image and the closed edge rectangle, and selects the closed edge rectangle with the largest similarity.

[0087] Therefore, according to the learning support device 10 according to the embodiment, the quality of the annotation for the AI model for image recognition can be ensured.

[0088] In addition, the edge extraction unit 12ba extracts the edges of the object using an edge extraction algorithm arbitrarily selected by the user U.

[0089] Therefore, according to the learning support device 10 according to the embodiment, it is possible to realize functions regardless of the characteristics of the teacher data. For example, it is possible to properly select algorithms according to the color and shape of the object or the brightness and saturation of the image, and accurately extract the edges of the object.

[0090] Further, the closed-edge rectangle creation unit 12ca creates the closed-edge rectangle having the coordinate positions of the four corners defined by the minimum horizontal position, the maximum horizontal position, the minimum vertical position, and the maximum vertical position of the closed edge as the vertex positions of each corner.

[0091] Therefore, according to the learning support device 10 according to the embodiment, it is possible to automatically and accurately create a closed-edge rectangle in contact with the closed edge, and the input rectangle IR can be automatically adjusted with high accuracy.

[0092] In addition, the most similar rectangle selection unit 12cb calculates the similarity between the input rectangle IR and the closed-edge rectangle using at least the aspect ratio, the center position, or either the horizontal width or the vertical width as the comparison element.

[0093] Therefore, according to the learning support device 10 according to the embodiment, it is possible to accurately calculate the similarity using at least the aspect ratio, the center position, or either the horizontal width or the vertical width as the comparison element, and appropriately select the most similar rectangle AR based on such similarity.

[0094] In addition, when the closed edge is only partially included in the input rectangle IR, the most similar rectangle selection unit 12cb expands the input rectangle IR by a certain amount so that the closed edge can be detected within the input rectangle IR.

[0095] Therefore, according to the learning support device 10 according to the embodiment, even if the input rectangle IR is not accurately created, for example, it is possible to automatically correct it.

[0096] In the above-described embodiment, the similarity calculation information 11d serving as a criterion for calculating similarity is, for example, pre-selected by the user U and can be preset as static information. However, it may be configured to be dynamically changeable. For example, based on the calculated similarity and the history of the user U's adoption results for the closed-edge rectangles presented based on such similarity, the tendency of the user U's adoption pattern, etc. may be learned by machine learning, and the learning result may be dynamically reflected in the similarity calculation information 11d. Further, such dynamic reflection may be applied to the extraction algorithm information 11b.

[0097] Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the present invention are not limited to the specific details and representative embodiments described and represented as above. Accordingly, various changes can be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents.

Explanation of Reference Numerals

[0098] 3 HMI unit 3a Input unit 3b Output unit 10 Learning support device 11 Storage unit 11a Target image DB 11b Extraction algorithm information 11c Edge information 11d Similarity calculation information 11e Rectangle information DB 12 Control unit 12a Image drawing unit 12aa Rectangle drawing unit 12b Image analysis unit 12ba Edge extraction unit 12c Rectangle creation unit 12ca Closed-edge rectangle creation unit 12cb Most similar rectangle selection unit IR Input rectangle U User

Claims

1. Extract the edges of an object in the target image for annotation in teacher data creation, create a closed edge rectangle that touches a closed edge, which is an edge extracted as a closed curve, among the extracted edges, calculate the similarity between the input rectangle input by the user for the target image and the closed edge rectangle, select the closed edge rectangle with the greatest similarity, and adopt the selected closed edge rectangle as the bounding box for the annotation, A learning support device comprising a controller.

2. The controller extracts the edges of the object using an edge extraction algorithm arbitrarily selected by the user, The learning support device according to claim 1.

3. The controller creates the closed edge rectangle with the coordinate positions of the four corners defined by the minimum horizontal position, maximum horizontal position, minimum vertical position, and maximum vertical position of the closed edge as each vertex position, The learning support device according to claim 1 or 2.

4. The controller calculates the similarity between the input rectangle and the closed edge rectangle using at least the aspect ratio, center position, or either the horizontal width and vertical width as elements for comparison, The learning support device according to claim 1, 2, or 3.

5. The controller when the closed edge is only partially included in the input rectangle, expands the input rectangle by a certain amount so that the closed edge can be detected within the input rectangle, The learning support device according to any one of claims 1 to 4.

6. Extract the edges of an object in the target image for annotation in teacher data creation, Create a closed-edge rectangle that touches a closed edge, which is an edge extracted as a closed curve, among the extracted edges, Calculate the similarity between the input rectangle input by the user for the target image and the closed-edge rectangle, Select the closed-edge rectangle with the greatest similarity, Adopt the selected closed-edge rectangle as the bounding box for the annotation, A learning support method executed by a controller.

7. Extract the edges of an object in the target image for annotation in teacher data creation, Create a closed-edge rectangle that touches a closed edge, which is an edge extracted as a closed curve, among the extracted edges, Calculate the similarity between the input rectangle input by the user for the target image and the closed-edge rectangle, Select the closed-edge rectangle with the greatest similarity, Adopt the selected closed-edge rectangle as the bounding box for the annotation, A learning support program executed by a controller.

Images