Information processing device

Abstract

Perform proper annotation on image data. [Solution] The information processing device comprises: a first acquisition means for acquiring a sample image of an object; an extraction means for extracting object features related to the object from the sample image; a second acquisition means for acquiring an image of the object to be annotated; a detection means for detecting a region containing the object in the image using the object features; and an annotation means for performing annotation related to the object on the region containing the object.

Description

Technical Field

[0001] This disclosure relates to the technical field of information processing apparatuses.

Background Art

[0002] As this type of apparatus, for example, a system has been proposed in which a large language model (LLM) is caused to generate query data based on a document, and a pair of the document and the query data is used for learning a search model for a chatbot (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When performing machine learning, a large amount of pre-annotated learning data may be required. For example, when learning a model for detecting an object included in an image, it is required to perform annotation work on a large amount of image data. However, for an object having a special structure (for example, a dedicated part used only in a very small number of products), it is difficult to perform automatic annotation. In such a case, there arises a technical problem that the annotation work becomes manual, resulting in an increase in labor and cost.

[0005] This disclosure has been made in view of the above problems, and an object thereof is to provide an information processing apparatus capable of appropriately performing annotation on image data.

Means for Solving the Problems

[0006] <00000An information processing device according to one aspect of this disclosure comprises: a first acquisition means for acquiring a sample image of an object; an extraction means for extracting object features relating to the object from the sample image; a second acquisition means for acquiring an image of an object to be annotated; a detection means for detecting a region in the image containing the object using the object features; and an annotation means for performing annotation relating to the object on the region containing the object. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram showing the hardware configuration of an information processing device according to an embodiment. [Figure 2] This is a block diagram showing the functional configuration of an information processing apparatus according to an embodiment. [Figure 3] This is a flowchart showing the flow of the extraction operation by the information processing device according to the embodiment. [Figure 4] This is a flowchart showing the flow of annotation operations by the information processing device according to the embodiment. [Modes for carrying out the invention]

[0008] The following describes an embodiment of the information processing device with reference to the drawings.

[0009] (Hardware configuration) First, the hardware configuration of the information processing device according to the embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing the hardware configuration of the information processing device according to the embodiment.

[0010] In Figure 1, the information processing device 10 according to this embodiment comprises an arithmetic unit 110, a storage device 120, a communication device 130, an input device 140, and an output device 150. The arithmetic unit 110, the storage device 120, the communication device 130, the input device 140, and the output device 150 are connected to each other via a data bus.

[0011] The arithmetic unit 110 is configured to perform various arithmetic operations in the information processing device 10. The arithmetic unit 110 may have a processor. The arithmetic unit 110 may have a single processor or may have multiple processors. In other words, the arithmetic unit 110 may have one or more processors. The processor may be a multi-core processor. If the arithmetic unit 110 has a single processor that is a multi-core processor, then logically, the arithmetic unit 110 can be said to have multiple processors.

[0012] The processor in the arithmetic unit 110 may be at least one of the following: CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), and TPU (Tensor Processing Unit).

[0013] The storage device 120 may be at least one of the following: RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, magneto-optical disk drive, SSD (Solid State Drive), and optical disk array. In other words, the storage device 120 may be implemented by a single device or by multiple devices.

[0014] The storage device 120 is capable of storing desired data. The storage device 120 may store the computer program CP that the arithmetic unit 110 will execute. The storage device 120 may temporarily store data that the arithmetic unit 110 will use temporarily when the arithmetic unit 110 is executing the computer program CP.

[0015] The computer program CP may be recorded on a non-temporary recording medium that is readable by a computer. In this case, the computer program CP may be stored in the storage device 120 by reading the recording medium using a recording medium reading device (not shown) provided by the information processing device 10. At least one of the following may be used as the recording medium: an optical disc, a magnetic medium, a magneto-optical disc, a semiconductor memory, and any other medium capable of storing a program. The computer program CP may also be obtained from an external device (not shown) of the information processing device 10 via a communication device 130. In other words, the computer program CP may be downloaded from an external device to the storage device 120 of the information processing device 10.

[0016] The arithmetic unit 110 (for example, a processor) may execute the processing that the information processing device 10 should perform together with the memory device 120 in which the computer program CP is stored (in other words, together with the memory device 120 and the computer program CP stored in the memory device 120). For example, by the arithmetic unit 110 executing the computer program CP, a logical functional block for executing the processing that the information processing device 10 should perform may be realized within the arithmetic unit 110 (for example, within the processor).

[0017] The communication device 130 is configured to communicate with devices outside the information processing device 10. The communication device 130 may use wired communication or wireless communication.

[0018] The input device 140 is a device capable of receiving input of information to the information processing device 10 from the outside. The input device 140 may include an operating device (for example, a keyboard, a mouse, a touch panel, etc.) that can be operated by a user of the information processing device 10. The input device 140 may include a recording medium reader that can read information recorded on a recording medium detachable from the information processing device 10, such as a USB (Universal Serial Bus) memory. When information is input to the information processing device 10 via the communication device 130 (in other words, when the information processing device 10 acquires information via the communication device 130), the communication device 130 may function as an input device.

[0019] The output device 150 is a device capable of outputting information to the outside of the information processing device 10. The output device 150 may have a display device capable of outputting visual information such as characters and images as the above information. The output device 150 may also have a speaker capable of outputting auditory information such as voices as the above information. The output device 150 may be configured to output the above information (for example, control information of other devices, etc.) to other devices. The output device 150 may be capable of outputting information to a recording medium detachable from the information processing device 10, such as a USB memory. When the information processing device 10 outputs information via the communication device 130, the communication device 130 may function as an output device.

[0020] <Functional Configuration> Next, the functional configuration of the information processing device 10 according to the embodiment will be described while referring to FIG. 2. FIG. 2 is a block diagram showing the functional configuration of the information processing device according to the embodiment.

[0021] In FIG. 2, the information processing apparatus 10 is configured as an apparatus that performs annotation on image data. The information processing apparatus 10 includes, as components for realizing its functions, a sample image acquisition unit 210, a feature quantity extraction unit 220, a feature quantity database (DB) 230, a target image acquisition unit 240, an object region detection unit 250, and an annotation execution unit 260. Each of the sample image acquisition unit 210, the feature quantity extraction unit 220, the target image acquisition unit 240, the object region detection unit 250, and the annotation execution unit 260 may be a processing block realized by the arithmetic unit 110 described above. Further, the feature quantity DB 230 may be a database realized by the storage device 120 described above.

[0022] The sample image acquisition unit 210 is configured to be able to acquire a sample image. The sample image is an image including an object. The object is an object to be annotated. The object may be an object detected from an image in image recognition processing. For example, the object may be a part used when manufacturing a product. The sample image acquisition unit 210 may be configured to be able to acquire a plurality of sample images collectively.

[0023] The feature quantity extraction unit 220 is configured to be able to extract a feature quantity related to the object (hereinafter, appropriately referred to as "target feature quantity") from the sample image acquired by the sample image acquisition unit 210. Note that the method for extracting the target feature quantity from the sample image is not particularly limited. The feature quantity extraction unit 220 may extract the target feature quantity from the sample image using, for example, a machine-learned model.

[0024] The feature quantity DB 230 is configured to be able to store the target feature quantity extracted by the feature quantity extraction unit 220. The feature quantity DB 230 may be configured to store, for example, by associating the name or ID of the object with the target feature quantity. The target feature quantity stored in the feature quantity DB 2 is appropriately readable by the object region detection unit 250.

[0025] The target image acquisition unit 240 is configured to acquire target images. The target images are images that will be annotated. The target image acquisition unit 240 may be configured to acquire multiple target images at once.

[0026] The object region detection unit 250 is configured to detect regions containing an object (hereinafter referred to as "object region" as appropriate) from the target image acquired by the target image acquisition unit 240. The object region detection unit 250 detects object regions in the target image using target features stored in the feature database 230. More specifically, the object region detection unit 250 detects as object regions areas that have features that match the target features in the target image (for example, areas where the similarity is above a predetermined threshold). The object region detection unit 250 may detect multiple object regions from a single target image.

[0027] The annotation execution unit 260 performs annotation on the object region detected by the object region detection unit 250. Specifically, the annotation execution unit 260 assigns correct data to the object region in the target image, indicating that the object is in that region. For example, if the object region of part A is detected in the target image, the annotation execution unit 260 should assign correct data indicating that the detected object region is the region in which part A is reflected.

[0028] (Extraction operation) Next, with reference to Figure 3, the flow of the extraction operation by the information processing device 10 according to the embodiment (specifically, the operation when extracting feature quantities of an object from a sample image) will be described. Figure 3 is a flowchart showing the flow of the extraction operation by the information processing device according to the embodiment.

[0029] As shown in Figure 3, when the extraction operation by the information processing device 10 according to the embodiment is started, first the sample image acquisition unit 210 acquires a sample image (step S101). Then, the feature extraction unit 220 extracts the target features from the sample image acquired by the sample image acquisition unit 210 (step S102).

[0030] Next, the feature database DB230 stores the target features extracted by the feature extraction unit 220 (step S103). After that, the information processing device 10 determines whether or not to terminate the extraction operation (step S104). The information processing device 10 may determine to terminate the extraction operation if, for example, it has extracted the target features from all the sample images acquired by the sample image acquisition unit 210.

[0031] If it is determined that the extraction operation should not be terminated (step S104: NO), the processing target is moved to the next sample image (step S105), and processing starts from step S102. By repeating this process, target features are extracted from all sample images acquired by the sample image acquisition unit 210. On the other hand, if it is determined that the extraction operation should be terminated (step S104: YES), the series of operations will end.

[0032] (Annotation behavior) Next, with reference to Figure 4, the flow of annotation operations by the information processing device 10 according to the embodiment (specifically, the operations when performing annotation on a target image) will be described. Figure 4 is a flowchart showing the flow of annotation operations by the information processing device according to the embodiment.

[0033] As shown in Figure 4, when the annotation operation by the information processing device 10 according to the embodiment is started, the target image acquisition unit 240 first acquires the target image (step S201). Then, the object region detection unit 250 uses the target feature quantities stored in the feature quantity DB 230 (i.e., the feature quantities of the object extracted in the extraction operation described above) to detect the object region from the target image acquired by the target image acquisition unit 240 (step S202).

[0034] Next, the annotation execution unit 260 performs annotation on the object region detected by the object region detection unit 250 (step S203). After that, the information processing device 10 determines whether or not to terminate the annotation operation (step S204). The information processing device 10 may determine to terminate the annotation operation if, for example, annotation has been performed on all target images acquired by the target image acquisition unit 240.

[0035] If it is determined that the annotation operation should not be terminated (step S204: NO), the processing target is moved to the next target image (step S205), and processing starts from step S202. By repeating this process, annotation is performed on all target images acquired by the target image acquisition unit 240. On the other hand, if it is determined that the annotation operation should be terminated (step S204: YES), the annotation execution unit 260 outputs the annotated target images (specifically, pairs of target images and ground truth data) as training data (step S206).

[0036] (Technical effects) Next, the technical effects obtained by the information processing device 10 according to this embodiment will be described.

[0037] As explained in Figures 1 to 4, in the information processing device 10 according to this embodiment, annotation is performed on the target image based on the target features extracted from the sample image. In this way, annotation of image data can be performed appropriately.

[0038] For example, it eliminates the need for manual (e.g., visual) identification of objects to be annotated. Furthermore, the accuracy of object detection within target images improves. Specifically, because target features extracted from sample images are stored in advance, objects that cannot be detected by general-purpose models (e.g., specialized parts) can be accurately detected and annotated. As a result, it becomes possible to properly train image recognition models (e.g., models that recognize objects in images).

[0039] The embodiments of the invention derived from the above-described embodiments are described below.

[0040] An information processing device according to one aspect of this disclosure includes: a first acquisition means for acquiring a sample image of an object; an extraction means for extracting object features related to the object from the sample image; a second acquisition means for acquiring an image to be annotated; a detection means for detecting a region in the image containing the object using the object features; and an annotation means for performing annotation related to the object on the region containing the object. In the above embodiment, the "sample image acquisition unit 210" corresponds to an example of the "first acquisition means," the "feature extraction unit 220" corresponds to an example of the "extraction means," the "target image acquisition unit 240" corresponds to an example of the "second acquisition means," the "object region detection unit 250" corresponds to an example of the "detection means," and the "annotation execution unit 260" corresponds to an example of the "annotation means."

[0041] In the information processing apparatus according to the above embodiment, the first acquisition means may acquire a plurality of sample images, the extraction means may extract a plurality of target feature quantities from the plurality of sample images, and the detection means may detect a region containing the target object using the plurality of target feature quantities. In this way, it becomes possible to detect multiple types of objects contained in the target image.

[0042] This disclosure is not limited to the embodiments described above and can be modified as appropriate without contradicting the gist or idea of ​​the invention as can be inferred from the claims and the specification as a whole. Information processing devices with such modifications are also included within the technical scope of the present invention. [Explanation of Symbols]

[0043] 10...Information processing device, 110...Calculation unit, 120...Storage device, 130...Communication device, 140...Input device, 150...Output device, 210...Sample image acquisition unit, 220...Feature extraction unit, 230...Feature database, 240...Target image acquisition unit, 250...Target object region detection unit, 260...Annotation execution unit

Claims

1. A first acquisition means for acquiring a sample image of the target object, An extraction means for extracting target features related to the object from the sample image, A second acquisition means for acquiring the target image to be annotated, A detection means for detecting a region in the target image that includes the target object using the target feature quantity, An annotation means for performing annotations relating to the object on the region including the object, An information processing device equipped with the following features.

2. The first acquisition means acquires a plurality of the sample images, The extraction means extracts a plurality of target features from a plurality of sample images, The detection means detects a region containing the object using a plurality of the target feature quantities. The information processing apparatus according to claim 1.

Images