Information processing method, information processor, and program

Abstract

To improve accuracy of a machine learning model.SOLUTION: An information processing method by an information processor 1, the information processing method includes: receiving manual annotation on an object indicated by a first image to apply a first label to the first image; generating a first model by executing learning using the first image to which the first label is applied as teacher data; applying a second label to each of a plurality of second images by inputting the plurality of second images to the first model to acquire the plurality of second images to which the second label is applied; receiving correction operation of correcting the second label for at least some of the plurality of second images to which the second label is applied; and generating a second model by executing learning using the plurality of second images to which the corrected second label is applied as teacher data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing method, an information processing apparatus, and a program.

Background Art

[0002] Conventionally, there has been known a technique for preparing a certain number of learning data 23 by a person visually annotating some of the acquired data 22 using an annotation tool and assigning a correct label (for example, Patent Document 1). In this technique, a learning unit 31 machine-learns the learning data 23 to generate a prediction model 21, and a prediction unit 32 predicts a pseudo label using the prediction model 21. It is determined by human visual inspection whether the pseudo label is valid as a correct label or whether it can be determined to be valid or not. The pseudo label determined to be invalid is discarded.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above prior art documents, since the pseudo label is discarded, the final number of teacher data may be reduced and become insufficient. Therefore, there is room for improvement from the viewpoint of the accuracy of the machine learning model.

[0005] An object of the present disclosure made in view of such circumstances is to improve the accuracy of a machine learning model.

Means for Solving the Problems

[0006] An information processing method according to an embodiment of the present disclosure is an information processing method by an information processing apparatus, Receiving manual annotation for an object shown in the first image and assigning a first label to the first image; Generating a first model by performing learning using the first image with the first label as teacher data; Inputting a plurality of second images into the first model to assign a second label to each of the plurality of second images, and obtaining a plurality of second images each with the second label assigned thereto; Receiving a correction operation for correcting the second label for at least a part of the plurality of second images each with the second label assigned thereto; Generating a second model by performing learning using the plurality of second images each with the corrected second label as teacher data; including.

[0007] An information processing apparatus according to an embodiment of the present disclosure is an information processing apparatus including a control unit, wherein the control unit receives manual annotation for an object shown in the first image and assigns a first label to the first image; generates a first model by performing learning using the first image with the first label as teacher data; inputs a plurality of second images into the first model to assign a second label to each of the plurality of second images, and obtains a plurality of second images each with the second label assigned thereto; receives a correction operation for correcting the second label for at least a part of the plurality of second images each with the second label assigned thereto; generates a second model by performing learning using the plurality of second images each with the corrected second label as teacher data; and executes an operation including.

[0008] A program according to an embodiment of the present disclosure is for an information processing apparatus Receiving manual annotation for the object shown in the first image and assigning a first label to the first image; Generating a first model by performing learning using the first image with the first label as training data; Inputting a plurality of second images into the first model to assign a second label to each of the plurality of second images, and obtaining a plurality of second images each with a second label assigned thereto; Receiving a correction operation for correcting the second label for at least a part of the plurality of second images each with a second label assigned thereto; Generating a second model by performing learning using the plurality of second images each with the corrected second label as training data; Executing operations including the above.

Advantages of the Invention

[0009] According to one embodiment of the present disclosure, the accuracy of the machine learning model can be improved.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Best Mode for Carrying Out the Invention

[0011] FIG. 1 is a schematic diagram of the information processing apparatus 1 according to the present embodiment. The information processing apparatus 1 can communicate with one or more other terminals via a network. The network includes, for example, a mobile communication network, the Internet, or a fixed communication network.

[0012] In FIG. 1, for simplicity of explanation, only one information processing apparatus 1 is shown. However, the number of information processing apparatuses 1 is not limited to this. For example, the processing executed by the information processing apparatus 1 may be executed by a plurality of information processing apparatuses 1 arranged in a distributed manner.

[0013] The information processing apparatus 1 may be a general-purpose device such as a PC or a dedicated device such as a workstation. "PC" is an abbreviation for personal computer. As an alternative example, the information processing apparatus 1 is a computer such as a server belonging to a cloud computing system or other computing system. The information processing apparatus 1 may be installed, for example, in a facility dedicated to an operator or a shared facility including a data center.

[0014] The internal configuration of the information processing apparatus 1 is described in detail in FIG. 2. The information processing apparatus 1 includes a control unit 11, a communication unit 12, a storage unit 13, a display unit 14, and an input unit 15. Each component of the information processing apparatus 1 is communicably connected to each other.

[0015] The control unit 11 includes, for example, one or more general-purpose processors including a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 11 may include one or more dedicated processors specialized for specific processing. Instead of including a processor, the control unit 11 may include one or more dedicated circuits. The dedicated circuit may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 11 may include an ECU (Electronic Control Unit). The control unit 11 controls the communication unit 12 to transmit and receive any information.

[0016] The control unit 11 can control the display content of the display unit 14. That is, when the control unit 11 receives control from the user, it is configured to be controllable to cause the display unit 14 to execute arbitrary display processing.

[0017] The communication unit 12 includes a communication module corresponding to one or more wired or wireless LAN (Local Area Network) standards for connecting to a network. The communication unit 12 may include a module corresponding to one or more mobile communication standards including LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation). The communication unit 12 may include a communication module corresponding to one or more short-range communication standards or specifications including Bluetooth (registered trademark), AirDrop (registered trademark), IrDA, ZigBee (registered trademark), Felica (registered trademark), or RFID. The communication unit 12 transmits and receives any information via the network.

[0018] The storage unit 13 includes, for example, a semiconductor memory, a magnetic memory, an optical memory, or a combination of at least two of these, but is not limited thereto. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 13 may function as, for example, a main memory device, an auxiliary memory device, or a cache memory. The storage unit 13 may store the information of the result analyzed or processed by the control unit 11. The storage unit 13 may store various information related to the operation or control of the information processing apparatus 1. The storage unit 13 may store a system program, an application program, and embedded software, etc. The storage unit 13 may be provided outside the information processing apparatus 1 and accessed from the information processing apparatus 1. For example, the storage unit 13 may store at least one of a first label, a first model, a second label, a plurality of second images each with the second label assigned thereto, a modified second label, and a second model in association with the first image.

[0019] The display unit 14 is, for example, a display. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electro luminescence. Instead of being provided in the information processing apparatus 1, the display unit 14 may be connected to the information processing apparatus 1 as an external output device. As the connection method, for example, any method such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used. The display unit 14 can display any image acquired or generated in the information processing apparatus 1.

[0020] The input unit 15 includes, for example, a touch sensor provided integrally with a display, a microphone, physical keys, capacitive keys, or a pointing device. The input unit 15 receives an operation for inputting information used for the operation of the information processing apparatus 1. When the input unit 15 includes a touch sensor, the input unit 15 detects contact of a user's finger or a stylus pen or the like and identifies the contact position. The input unit 15 may be integrated with the display to form a touch panel display.

[0021] The information processing method in this embodiment will be described. In this embodiment, as shown in FIG. 3, the control unit 11 receives a manual (i.e., by hand) annotation (i.e., an annotation) for an object shown in a single first image G1 via the input unit 15. The number of the first images G1 is arbitrary. The control unit 11 assigns a first label (e.g., the first label L1) to the object (e.g., the object T1) in the first image G1 and obtains a first image GA1 to which the first label is assigned. By annotation, one label is assigned to one object. The first label L1 may be a mask image covering the object T1. The mask image in this embodiment is semi-transparent. In this case, the user can intuitively determine whether the first label L1 correctly surrounds the object. As an alternative, the mask image may be opaque. The mask image may be a polygon. In this case, the user can easily trace the mask image on the input unit 15. The control unit 11 generates a first model M1, which is a first process learned model, by performing learning using the first image GA1 to which the first label is assigned as teacher data.

[0022] As shown in FIG. 4, the control unit 11 inputs N (for example, 99) second images G2 different from the first image G1 in the first step into the first model M1, and assigns a second label to each of the N (N: a natural number of 2 or more) second images G2 by performing annotation using, for example, AI (Artificial Intelligence). The control unit 11 acquires N second images GA2 to each of which the second label is assigned. The N second images G2 are different from each other. The control unit 11 receives, from the user via the input unit 15, a correction operation for the second label of the second image GA2. Details of the correction operation will be described later. The control unit 11 generates a second model M2, which is a second-step learned model, by performing learning using the N second images GA2 as teacher data.

[0023] As shown in FIG. 5, the control unit 11 inputs P third images G3 different from the total of 1+N images including one first image G1 and N second images G2 into the second model M2, performs annotation using AI, and acquires the P third images G3. The P third images G3 are different from each other. The control unit 11 generates a third model M3, which is a third-step learned model obtained by training the P third images G3. Since the third model M3 has relatively high accuracy, it may be used on-site.

[0024] Details of the correction operation for the second label will be described. As shown in FIG. 6, a second label 72 is assigned to the object 71 in the second image. The second label 72 is an image that covers a part of the object 71. The contour line 72R of the second label 72 includes a contour line assigned to at least a part of the contour of the object 71. The control unit 11 receives, via the input unit 15, a user operation that starts the correction operation at the starting point P1 on the contour line 72R and ends the correction operation at the ending point P4 on the contour line 72R passing through the point P2 and the point P3. The user operation may be an operation of tracing the surface of the touch panel display. In this case, the correction operation of the contour line 72R includes correcting the contour line 72R outwardly from the object 71 so as to newly add, as a closed area, to the second label 72 an area of the object 71 that is not covered by the second label 72. When the control unit 11 receives a user operation to end the correction operation, it generates a corrected second label 73. In the correction operation of the second label 72, the control unit 11 corrects the second label 72 without generating a new label.

[0025] In other embodiments, the second label may protrude from the object. As an additional example or an alternative example to the example shown in FIG. 6, the correction operation of the contour line 72R may include a correction operation of correcting the contour line inwardly from the object so as to cut off an area of the second label that protrudes from the object.

[0026] As an additional example or an alternative example, the control unit 11 can assign different second labels to the second image G2 for each type. Here, the second label is, as an example, a mask image. As shown by reference numeral 76 in FIG. 7, labels of different colors or patterns are assigned for each type of the second label (that is, label A, label B, and label C). In the process of assigning the second label, the control unit 11 determines the type for each object and assigns different second labels for each type. For example, the control unit 11 assigns different types for each shape of the object. The number of types of the second label can be arbitrarily set.

[0027] The second image GA2 after the second label is assigned is shown in FIG. 8. For example, a second label M92 is assigned to the object 92. The control unit 11 can receive a modification operation for the type of the object with respect to the second image GA2 via the input unit 15. For example, a case where a label M93 indicating label A is erroneously assigned to the object 93 will be described. In this case, the control unit 11 receives, via the input unit 15, the selection of the second label (here, the area 91 where label B is displayed) corresponding to the modified type. When the control unit 11 receives the selection for the object 93 that is the target of modification, a new second label (here, label B) is assigned to the object 93. As an alternative example of the method for assigning a new second label, the control unit 11 can receive, via the input unit 15, a modification operation from the user to correct the label M93 itself indicating the incorrect label A into a correct label by a polygon. The modification operation in this case is an operation of operating only the outer contour of the incorrect label M93 to change the incorrect label into a correct label. As another alternative example of the method for assigning a new second label, the control unit 11 may receive, via the input unit 15, a modification operation of deleting the label M93 indicating the incorrect label A and then assigning a new correct second label (here, label B).

[0028] In FIG. 9, a flowchart showing the operation of the information processing apparatus 1 is described.

[0029] In S1, the control unit 11 receives a manual annotation for the object shown in the first image. In S2, the control unit 11 assigns a first label to the first image. In S3, the control unit 11 generates a first model by performing learning using the first image with the first label assigned as teacher data. In S4, the control unit 11 inputs a plurality of second images to the first model to assign a second label to each of the plurality of second images, and obtains a plurality of second images each with a second label assigned thereto.

[0030] In S5, the control unit 11 receives a correction operation for correcting the second label for at least a part of the plurality of second images each provided with the second label. In S6, the control unit 11 generates a second model by performing learning using, as teacher data, the plurality of second images each provided with the corrected second label.

[0031] As described above, according to this embodiment, the operation of the control unit 11 of the information processing apparatus 1 includes receiving a manual annotation for the object shown in the first image G1 and assigning a first label to the first image G1, and generating a first model by performing learning using, as teacher data, the first image G1 to which the first label is assigned. The operation of the control unit 11 further includes assigning a second label to each of the plurality of second images G2 by inputting the plurality of second images G2 into the first model to obtain a plurality of second images GA2 each provided with the second label, receiving a correction operation for correcting the second label for at least a part of the plurality of second images GA2 each provided with the second label, and generating a second model by performing learning using, as teacher data, the plurality of second images GA2 each provided with the corrected second label. With this configuration, the information processing apparatus 1 receives the correction of the second label and uses, as teacher data, the plurality of second images to which the corrected second label is assigned. Thus, the information processing apparatus 1 can enrich the teacher data and thereby improve the accuracy of the machine learning model.

[0032] Also, according to this embodiment, the correction operation of the second label includes a correction operation of a contour line assigned to at least a part of the contour of the object. For example, the operation of the control unit 11 includes receiving a user operation that starts a correction operation on the contour line and ends the correction operation on the contour line, and receiving the correction operation of the second label. With this configuration, the information processing apparatus 1 can receive the correction operation of the contour line of the second label, so that the teacher data can be further enriched.

[0033] Also, according to the present embodiment, the contour line correction operation includes correcting the contour line toward the inside of the object so as to cut out the region protruding from the object among the second labels. The contour line correction operation includes correcting the contour line toward the outside of the object so as to newly add the region of the object that is not covered by the second label to the second label. With this configuration, when the second label and the object do not match, the information processing apparatus 1 can change the second label to an appropriate shape, so that the teacher data can be further enriched.

[0034] Also, according to the present embodiment, the correction operation of the second label includes a correction operation of the type of the object. With this configuration, the information processing apparatus 1 can correct the second label that does not correspond to the type of the object, so that the second label can be made appropriate corresponding to the type, and thus the teacher data can be further enriched.

[0035] It should be noted that although the present disclosure has been described based on the drawings and examples, those skilled in the art may make various modifications and alterations based on the present disclosure. In addition, changes can be made without departing from the spirit of the present disclosure. For example, the functions and the like included in each means or each step can be rearranged so as not to be logically contradictory, and a plurality of means or steps can be combined into one or divided.

[0036] The drawings for explaining the embodiments according to the present disclosure are schematic. The dimensional ratios and the like on the drawings do not necessarily match the actual ones.

[0037] For example, in the above-described embodiment, a program for executing all or part of the functions or processes of the information processing apparatus 1 can be recorded on a computer-readable recording medium. The computer-readable recording medium includes a non-temporary computer-readable medium, and is, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory. The distribution of the program is performed, for example, by selling, transferring, or lending a portable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. The distribution of the program may also be performed by storing the program in the storage of an arbitrary server and transmitting the program from the arbitrary server to another computer. The program may also be provided as a program product. As an embodiment of the present disclosure, it is also possible to take an embodiment as a system, a program, a storage medium (for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card, etc.) on which the program is recorded.

[0038] The implementation form of the program is not limited to application programs such as object code compiled by a compiler and program code executed by an interpreter, and may be in the form of a program module incorporated in an operating system. Further, the program does not have to be configured such that all processes are executed only on the CPU on the control board. The program may be configured such that part or all of it is executed by another processing unit mounted on an expansion board or an expansion unit added to the board as needed.

Description of Reference Numerals

[0039] 1 Information processing apparatus

Claims

1. An information processing method by an information processing apparatus, comprising: receiving manual annotation for an object shown in a first image, and attaching a first label to the first image; generating a first model by performing learning using the first image with the first label attached thereto as teacher data; inputting a plurality of second images into the first model to attach a second label to each of the plurality of second images, and obtaining a plurality of second images each with the second label attached thereto; receiving a correction operation for correcting the second label for at least a part of the plurality of second images each with the second label attached thereto; generating a second model by performing learning using the plurality of second images each with the corrected second label attached thereto as teacher data; An information processing method including the above.

2. The information processing method according to claim 1, wherein the correction operation for the second label includes a correction operation for a contour line attached to at least a part of the contour of the object.

3. The information processing method according to claim 2, wherein when receiving a user operation that starts a correction operation on the contour line and ends the correction operation on the contour line, receiving the correction operation for the second label is included.

4. The information processing method according to claim 3, wherein the correction operation for the contour line includes correcting the contour line inward toward the object so as to cut out an area protruding from the object among the second labels.

5. The information processing method according to claim 3, wherein the correction operation for the contour line includes correcting the contour line outward toward the object so as to newly add an area of the object that is not covered by the second label to the second label.

6. The information processing method according to claim 1, wherein the correction operation for the second label includes a correction operation for the type of the object.

7. An information processing apparatus including a control unit, wherein the control unit: receives manual annotation for an object shown in a first image, and attaches a first label to the first image; generates a first model by performing learning using the first image with the first label attached thereto as teacher data; By inputting a plurality of second images into the first model, a second label is assigned to each of the plurality of second images, and a plurality of second images each with a second label assigned thereto are obtained. Receiving a correction operation for correcting the second label for at least a part of the plurality of second images each with a second label assigned thereto. Generating a second model by performing learning using, as teacher data, a plurality of second images each with the corrected second label assigned thereto. An information processing apparatus that executes operations including the above.

8. In an information processing apparatus, Receiving manual annotation for an object shown in a first image and assigning a first label to the first image. Generating a first model by performing learning using, as teacher data, the first image with the first label assigned thereto. By inputting a plurality of second images into the first model, a second label is assigned to each of the plurality of second images, and a plurality of second images each with a second label assigned thereto are obtained. Receiving a correction operation for correcting the second label for at least a part of the plurality of second images each with a second label assigned thereto. Generating a second model by performing learning using, as teacher data, a plurality of second images each with the corrected second label assigned thereto. A program for causing the above operations to be executed.

Images