Information processing method

Abstract

To provide an information processing method for reducing a load of annotation work for counting.SOLUTION: An information processing method of an information processing device 1 includes processes of: receiving selection of an image pattern for one unit of counted objects; detecting one or more candidates of the counted objects in an image imaging the counted objects; and annotating the objects in the image on the basis of comparison between the one or more candidates and the image pattern.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to an information processing method.

Background Art

[0002] Conventionally, there is known a technique for counting the number of components arranged on a sheet by converting an image obtained by imaging the sheet on which the components are arranged into a binary image (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above background art, binarization is essential for counting components. Therefore, there is room for improvement from the viewpoint of efficiency.

[0005] In view of such circumstances, an object of the present disclosure is to reduce the work load of annotation for counting.

Means for Solving the Problems

[0006] An information processing method according to an embodiment of the present disclosure is an information processing method by an information processing apparatus, receiving a selection of an image pattern for each unit of an object to be counted, detecting one or more candidates of the object to be counted in an image in which the object to be counted is imaged, annotating the object in the image based on a comparison between the one or more candidates and the image pattern, and including.

Effects of the Invention

[0007] According to one embodiment of the present disclosure, the work load of annotation for counting can be reduced.

Brief Description of Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0009] FIG. 1 is a schematic diagram of the information processing apparatus 1 of the present embodiment. The information processing apparatus 1 can communicate with one or more other terminals via a network. The network includes, for example, a mobile communication network, the Internet, or a fixed communication network.

[0010] In FIG. 1, for simplicity of explanation, only one information processing apparatus 1 is shown. However, the number of information processing apparatuses 1 is not limited to this. For example, the processing executed by the information processing apparatus 1 may be executed by a plurality of information processing apparatuses 1 arranged in a distributed manner.

[0011] The information processing apparatus 1 is a computer such as a server belonging to a cloud computing system or other computing systems. The information processing apparatus 1 may be installed, for example, in a facility dedicated to a business operator or a shared facility including a data center.

[0012] The internal configuration of the information processing apparatus 1 will be described in detail. The information processing apparatus 1 includes a control unit 11, a communication unit 12, and a storage unit 13. Each component of the information processing apparatus 1 is connected to be communicable with each other.

[0013] The control unit 11 includes, for example, one or more general-purpose processors including a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 11 may include one or more dedicated processors specialized for specific processing. Instead of including a processor, the control unit 11 may include one or more dedicated circuits. The dedicated circuit may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 11 may include an ECU (Electronic Control Unit). The control unit 11 controls the communication unit 12 to transmit and receive any information.

[0014] The communication unit 12 includes a communication module corresponding to one or more wired or wireless LAN (Local Area Network) standards for connecting to a network. The communication unit 12 may include a module corresponding to one or more mobile communication standards including LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation). The communication unit 12 may include a communication module or the like corresponding to one or more short-range communication standards or specifications including Bluetooth (registered trademark), AirDrop (registered trademark), IrDA, ZigBee (registered trademark), Felica (registered trademark), or RFID. The communication unit 12 transmits and receives any information via the network.

[0015] The storage unit 13 includes, for example, a semiconductor memory, a magnetic memory, an optical memory, or a combination of at least two of these, but is not limited thereto. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 13 may function as, for example, a main memory device, an auxiliary memory device, or a cache memory. The storage unit 13 may store the information of the result analyzed or processed by the control unit 11. The storage unit 13 may store various information related to the operation or control of the information processing apparatus 1. The storage unit 13 may store a system program, an application program, embedded software, and the like. The storage unit 13 may be provided outside the information processing apparatus 1 and accessed from the information processing apparatus 1.

[0016] Hereinafter, the details of the information processing method in this embodiment will be described. In this embodiment, annotation is performed on the object to be counted in the image.

[0017] As shown in FIG. 2, the control unit 11 receives the input of the image 21 in which the object to be counted is captured.

[0018] The control unit 11 receives from the user the selection of the image pattern GP for each unit of the object to be counted in the image 21. The image pattern GP is also referred to as a gallery image. One or more image patterns GP may be stored in the storage unit 13. Here, the object to be counted is, as an example, a component. One image pattern GP surrounded by one region (for example, one rectangle) corresponds to one unit of the object (here, one component). In the image pattern GP, a plurality of objects may overlap or be in contact with each other. As an alternative example, the image pattern GP may be selected in an image different from the image 21.

[0019] As an additional example or an alternative example, the control unit 11 may receive the selection of the image pattern GP via the touch panel of the terminal operated by the user. The control unit 11 may receive the selection of the image pattern GP in a plurality of regions via the touch panel. The region corresponding to the image pattern GP may be a region in dot units. For example, the image pattern GP may be selected by a single click.

[0020] The control unit 11 detects one or more candidates of the object to be counted in the entire image 21. Specifically, the control unit 11 determines one or more regions in the image 21 that respectively contain the detected candidates. Any algorithm can be used for candidate detection. The algorithm may be, for example, an artificial intelligence algorithm that performs object detection based on the contour of the object. As shown in FIG. 2, there may be a situation where the entire image 21 contains non-target objects 22 such as wire meshes. In this case, it is not preferable for the control unit 11 to detect the non-target object 22 as a candidate. Under such circumstances, as an alternative example, as shown in FIG. 3, the control unit 11 may perform candidate detection only in the region R1 specified by the user. Thereby, since the probability of detecting a non-target object as a candidate can be reduced, the processing related to candidate detection can be reduced. In other words, since the control unit 11 detects candidates in the region R1 specified so as not to include the non-target object 22 in the image 21, the processing related to candidate detection can be reduced. Note that one or more arbitrary regions R1 may be specified by the user.

[0021] The control unit 11 compares the one or more detected candidates with each image pattern GP. For each detected candidate, the control unit 11 identifies one image pattern GP having the maximum similarity with the candidate from among the one or more image patterns GP. When the similarity with the image pattern GP exceeds a predetermined threshold value, the control unit 11 extracts the image pattern GP from the storage unit 13. Any algorithm can be used for calculating the similarity. The algorithm may be, for example, an algorithm for calculating the cosine similarity between images.

[0022] A specific description will be given with reference to FIG. 6. The control unit 11 vectorizes each image pattern GPj to calculate a reference vector Rj. In FIG. 6, five reference vectors R1 to R5 corresponding to five image patterns GP1 to GP5 are calculated.

[0023] Subsequently, the control unit 11 vectorizes each region of the image 21 that contains the detected candidate to calculate a candidate vector Ti. In FIG. 6, n candidate vectors T1 to Tn corresponding to the n detected candidates are calculated.

[0024] Subsequently, the control unit 11 calculates a cosine similarity Sim-ij between the candidate vector Ti and each reference vector Rj for each candidate vector Ti. Note that the "cosine similarity" is an index that takes a value between -1 and +1, and the larger the value, the more similar the appearances of the two images are. In FIG. 6, for example, the cosine similarities Sim-11 to Sim-15 between the candidate vector T1 and each reference vector R1 to R5 are calculated. Similarly, the cosine similarities Sim-21 to Sim-25 between the candidate vector T2 and each reference vector R1 to R5 are calculated.

[0025] Subsequently, the control unit 11 identifies, for each candidate vector Ti, one image pattern GPj from among one or more image patterns GPj that has the maximum cosine similarity Sim-ij with the candidate vector Ti. In FIG. 6, for example, the maximum value of the cosine similarity calculated for the candidate vector T1 is Sim-13, and the image pattern GP3 corresponding to Sim-13 is identified. Similarly, the maximum value of the cosine similarity calculated for the candidate vector T2 is Sim-25, and the image pattern GP5 corresponding to Sim-25 is identified.

[0026] Then, for each candidate vector Ti, when the cosine similarity Sim-ij with the specified image pattern GPj exceeds a predetermined threshold, the control unit 11 extracts the specified image pattern GPj from the storage unit 13. The threshold is, for example, 0.5, but is not limited to this example and can be arbitrarily set. In FIG. 6, for example, the maximum value Sim-13 of the cosine similarity calculated for the candidate vector T1 exceeds the threshold, and the image pattern GP3 corresponding to Sim-13 is extracted. This suggests that the candidate corresponding to the candidate vector T1 is presumed to correspond to the object corresponding to the image pattern GP3. On the other hand, the maximum value Sim-25 of the cosine similarity calculated for the candidate vector T2 is below the threshold, and no image pattern GPj is extracted. This suggests that the candidate corresponding to the candidate vector T2 is presumed not to correspond to any object corresponding to the image pattern GPj (that is, is presumed to correspond to a non-target object).

[0027] In this way, the control unit 11 compares the detected one or more candidates with each image pattern GP, and for each detected candidate, identifies one image pattern GP having the maximum similarity with the candidate among the one or more image patterns GP, and when the similarity with the image pattern GP exceeds a predetermined threshold, extracts the image pattern GP from the storage unit 13.

[0028] As shown in FIG. 4, the control unit 11 attaches an annotation AN to each candidate from which the image pattern GP has been extracted.

[0029] In FIG. 5, a flowchart showing the operation of the information processing apparatus 1 is described.

[0030] In S1, the control unit 11 of the information processing apparatus 1 receives the input of the image in which the object to be counted is imaged. In S2, the control unit 11 receives the selection of the image pattern GP. In S3, the control unit 11 detects one or more candidates of the object to be counted in the input image in which the object to be counted is imaged.

[0031] In S4, the control unit 11 compares one or more candidates of the object with each image pattern GP, and for each detected candidate, identifies one image pattern GP having the maximum similarity with the candidate among the one or more image patterns GP. When the similarity with the image pattern GP exceeds a predetermined threshold, the control unit extracts the image pattern GP from the storage unit 13. In S5, the control unit 11 attaches an annotation to the candidate from which the image pattern GP has been extracted.

[0032] As described above, according to the present embodiment, the control unit 11 of the information processing apparatus 1 receives the selection of the image pattern GP for each unit of the object to be counted, detects one or more candidates of the object to be counted in the image in which the object to be counted is imaged, and attaches an annotation to the object in the image based on the comparison between the one or more candidates and the image pattern GP. With this configuration, the information processing apparatus 1 can attach an annotation to the object by comparing the selected image pattern GP with the one or more detected candidates, so that the work load of annotation can be reduced.

[0033] Also according to the present embodiment, the operation of the control unit 11 includes receiving from the user the selection of the image pattern GP in a plurality of regions via the touch panel. With this configuration, the information processing apparatus 1 can receive the selection of a plurality of image patterns GP, so that a sufficient number of image patterns GP can be prepared and the accuracy of annotation can be improved.

[0034] Also according to the present embodiment, each of the plurality of regions includes a region in dot units. With this configuration, the information processing apparatus 1 can receive the selection of the image pattern GP in dot units, so that the method of selecting the image pattern GP can be made flexible.

[0035] Also according to the present embodiment, the image pattern GP includes an image pattern in which a plurality of objects overlap. With this configuration, the information processing apparatus 1 can improve the accuracy of annotation for each unit of the object even when the objects overlap.

[0036] Moreover, according to the present embodiment, annotating an object based on comparison includes annotating the object based on whether the similarity between one or more candidates and the image pattern GP exceeds a threshold value. With this configuration, since the information processing apparatus 1 can attach an annotation when the similarity exceeds the threshold value, the accuracy of annotation can be improved.

[0037] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. In addition, changes can be made without departing from the spirit of the present disclosure. For example, the functions included in each means or each step can be rearranged so as not to be logically contradictory, and a plurality of means or steps can be combined into one or divided.

[0038] The drawings for explaining the embodiments according to the present disclosure are schematic. The dimensional ratios and the like on the drawings do not necessarily match the actual ones.

[0039] For example, in the above-described embodiment, a program that executes all or part of the functions or processes of the information processing apparatus 1 can be recorded on a computer-readable recording medium. The computer-readable recording medium includes a non-transitory computer-readable medium, and for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory. The distribution of the program is performed, for example, by selling, transferring, or lending a portable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. The distribution of the program may also be performed by storing the program in the storage of an arbitrary server and transmitting the program from the arbitrary server to another computer. The program may also be provided as a program product. As an embodiment of the present disclosure, it is also possible to take an embodiment as a system, a program, and a storage medium (for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card, etc.) on which the program is recorded.

[0040] The implementation form of the program is not limited to application programs such as object code compiled by a compiler and program code executed by an interpreter, and may be in the form of program modules incorporated in an operating system. Further, the program may or may not be configured such that all processing is performed only on the CPU on the control board. The program may be configured such that part or all of it is performed by another processing unit mounted on an expansion board or an expansion unit added to the board as necessary.

[0041] In the above-described embodiment, an example has been described in which the control unit 11 of the information processing apparatus 1 compares one or more detected candidates with each image pattern GP, and for each detected candidate, identifies one image pattern GP having the maximum similarity to the candidate from among the one or more image patterns GP. In another embodiment, the control unit 11 may compare one or more detected candidates with each image pattern GP, and for each detected candidate, identify each image pattern GP whose similarity to the candidate exceeds a predetermined threshold from among the one or more image patterns GP.

Description of Reference Numerals

[0042] 1 Information processing apparatus 11 Control unit 12 Communication unit 13 Storage unit

Claims

1. An information processing method by an information processing apparatus, comprising: receiving a selection of an image pattern for each unit of an object to be counted; detecting one or more candidates of the object to be counted in an image in which the object to be counted is imaged; annotating the object in the image based on a comparison between the one or more candidates and the image pattern; An information processing method including the above.

2. The information processing method according to Claim 1, further comprising: receiving a selection of an image pattern for each unit of the object in each of a plurality of regions via a touch panel.

3. The information processing method according to Claim 2, wherein: each of the plurality of regions includes a dot unit region.

4. The information processing method according to Claim 1, wherein: the image pattern includes an image pattern in which a plurality of objects overlap.

5. The information processing method according to Claim 1, wherein: annotating the object based on the comparison includes annotating the object based on whether a similarity between the one or more candidates and the image pattern exceeds a threshold value.

Images