Suspected image generation device

Abstract

A pseudo image generation device (10) for generating a pseudo image from foreground and background image data comprises: a transparency rendering unit (11) for rendering a region in the foreground image, except the object to be synthesized, transparent; a synthesis unit (12) for synthesizing the background image data with the foreground image data after the transparency rendering so that the foreground image after the transparency rendering is superimposed at a prescribed position in the background image; a deletion unit (13) for deleting some background annotation data after the synthesis that overlaps the object being synthesized, from the background annotation data included in the background image data and having been affixed within the background image; and a data generation unit (14) for generating the annotation data of the pseudo image on the basis of the background annotation data after the deletion.

Description

Technical Field

[0001] The present disclosure relates to a pseudo-image generation device that generates pseudo-images with annotation data used for artificial intelligence learning and the like. The term "annotation data" as used herein means related information (also referred to as "annotation", "tag", or "label") of an object shown in an image, and is generally synthesized at a position near the object in the aforementioned image.

Background Art

[0002] In artificial intelligence learning, for example, by synthesizing the name of a plant as annotation data at a position near the plant in an image capturing the plant, image data with annotation data for various plants is generated, and the obtained large number of image data is input to artificial intelligence as training data for learning, thereby performing processing such as making the artificial intelligence learn the names of various plants. Generation of image data with annotation data is described in, for example, Patent Document 1. In addition, it is desirable that the training data input to artificial intelligence for learning be a large number of image data of various variations. Therefore, a composite image (an image in which a composite target object is shown in a background) obtained by compositing a background image capturing a background and a foreground image capturing a certain composite target object is often used as training data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Incidentally, in generating the annotated image data described above, it was necessary to repeatedly apply annotation data to images obtained by photographing only a part of an image under different conditions, or images obtained by slightly shifting the position or angle of an image, even if the parts of the image looked the same as the original. As a result, the processing load was extremely high, including the time and manpower required to apply the annotation data, the time and manpower required to collect various variations of data, and the processing load on the computer.

[0005] This disclosure aims to reduce the processing load described above in generating annotated image data and to generate numerous variations of pseudo-image data with minimal processing load. [Means for solving the problem]

[0006] The pseudo-image generating apparatus according to this disclosure generates a pseudo-image from foreground image data relating to a foreground image of an object to be synthesized and background image data relating to a background, comprising: a transparency processing unit that processes areas other than the object to be synthesized in the foreground image to make transparent; a synthesis unit that synthesizes the background image data and the foreground image data after transparency processing so that the foreground image after transparency processing by the transparency processing unit is superimposed on a predetermined position in the background image; a deletion unit that deletes background annotation data that overlaps with the object to be synthesized after synthesis by the synthesis unit in the background annotation data included in the background image data and attached to the background image; and a data generation unit that generates annotation data for the pseudo-image based on the background annotation data after deletion by the deletion unit.

[0007] In the above pseudo-image generation device, the transparency processing unit makes transparent the areas of the foreground image other than the object to be composited. Here, "transparency processing" refers to the process of processing the area other than the object to be composited so that it is visible through the image. Then, the compositing unit combines the background image data and the transparentized foreground image data so that the transparentized foreground image is superimposed on a predetermined position in the background image. In the resulting composite image, the object to be composited will be visible, and the background will be visible in the areas other than the object to be composited. Furthermore, the deletion unit deletes the background annotation data that overlaps with the object to be composited after the above compositing, and the data generation unit generates annotation data for the pseudo-image based on the deleted background annotation data.

[0008] As described above, by performing transparency processing on areas other than the object to be composited in the foreground image, and by deleting background annotation data that overlaps with the object to be composited after compositing, and then generating annotation data for pseudo-images, it becomes unnecessary to repeatedly apply annotation data to areas that look the same as the original image, such as images obtained by taking pictures of only a part of the whole image under different conditions, or images obtained by taking pictures of the same image at a slightly different position or angle. Therefore, the processing load in a broad sense, including the time and manpower required to apply annotation data, the time and manpower required to collect data of various variations, and the processing load on the computer, is reduced, and a large number of pseudo-image data variations can be generated with little load. [Effects of the Invention]

[0009] According to this disclosure, the processing load in generating annotated image data is reduced, and a large number of pseudo-image data variations can be generated with minimal overhead. [Brief explanation of the drawing]

[0010] [Figure 1] This is a functional block diagram of a pseudo-image generation device according to an embodiment of the invention. [Figure 2] This is a flowchart illustrating the process performed by the pseudo-image generation device. [Figure 3] This diagram illustrates the process of combining background and foreground image data. [Figure 4] This diagram illustrates the removal of background annotation data. [Figure 5] This figure shows an example of the hardware configuration of a pseudo-image generation device. [Modes for carrying out the invention]

[0011] Hereinafter, an embodiment of the pseudo-image generation device according to this disclosure will be described with reference to the drawings. The pseudo-image generation device according to this disclosure is a pseudo-image generation device that generates a pseudo-image from foreground image data relating to a foreground image of an object to be synthesized and background image data relating to a background image of the background, and an example of its configuration is shown in Figure 1.

[0012] As shown in Figure 1, the pseudo-image generation device 10 comprises a transparency processing unit 11, a synthesis unit 12, a deletion unit 13, and a data generation unit 14. The functions of each unit will be outlined below, but further details will be described later using Figures 2 to 4.

[0013] The transparency processing unit 11 is a functional unit that performs transparency processing on areas in the foreground image other than the object to be composited (frame F, described later). More specifically, the transparency processing unit 11 cuts out the target area containing the object to be composited from the foreground image and performs transparency processing on areas in the cut-out target area other than the object to be composited.

[0014] The synthesis unit 12 is a functional unit that synthesizes background image data and foreground image data after transparency processing so that the foreground image after transparency processing by the transparency processing unit 11 is superimposed on a predetermined position in the background image.

[0015] The deletion unit 13 is a functional unit that deletes background annotation data included in the background image data and attached to the background image that overlaps with the object to be synthesized after synthesis by the synthesis unit 12. Furthermore, it also has the function of transforming foreground annotation data included in the foreground image data and attached to the foreground image according to the position in which the foreground image is superimposed on the background image. For example, the deletion unit 13 transforms the coordinate information of the object to be synthesized included in the foreground annotation data from coordinates in the coordinate system of the foreground image to coordinates in the coordinate system of the background image.

[0016] The data generation unit 14 is a functional unit that generates pseudo-image annotation data based on the converted foreground annotation data and the deleted background annotation data. It also has the function of outputting pseudo-image image data including the generated annotation data. "Output" includes various forms of output, such as display output to a display, print output to a printer, and data transmission to an external device.

[0017] Next, the processes executed in the pseudo-image generation device 10 will be explained according to the flowchart in Figure 2. For example, the processes in Figure 2 are started when a predetermined execution start instruction is given by the operator of the pseudo-image generation device 10.

[0018] First, the transparency processing unit 11 acquires foreground image data including foreground annotation data and background image data including background annotation data (step S1), cuts out the target area including the frame from the foreground image as shown in Figure 3 (step S2), and then performs transparency processing on the area other than the frame in the cut-out target area as shown in Figure 3 (step S3). Note that the data acquisition in step S1 may be obtained by receiving the above data from an external device, or by reading it from a database (not shown) stored in the pseudo-image generation device 10, and the acquisition method is not limited to a specific method.

[0019] Next, the synthesizing unit 12 synthesizes the background image data and the transparency-processed foreground image data so that the transparency-processed foreground image is superimposed at a predetermined position in the background image, to generate image data of a pseudo image (step S4). In the synthesized image here, for example, as shown in Figure 3, the upper right portion of the frame is captured, and the area outside the frame captures the background wheat cultivation field.

[0020] Next, the deleting unit 13 converts the foreground annotation data according to the position where the foreground image is superimposed on the background image (step S5). Specifically, the coordinate information of the frame F included in the foreground annotation data is converted from coordinates in the coordinate system of the foreground image to coordinates in the coordinate system of the background image. For example, Figure 4 shows an example of obtaining a synthesized image M by synthesizing the foreground image P1 and the background image P2, where the coordinate information of the frame F included in the foreground annotation data a1 is converted from coordinates in the coordinate system of the foreground image to coordinates in the coordinate system of the background image, to obtain foreground annotation data A1 including the converted coordinate information. Note that in Figure 4, for convenience, the annotation data is represented by a rectangle surrounding part or all of the assigned object. Then, the deleting unit 13 deletes the background annotation data that overlaps with the frame in the synthesized image (step S6). For example, as shown in Figure 4, the background annotation data overlapping with the frame F in the synthesized image M (that is, the background annotation data A2 and A3 in the background image P2 attached to the wheat ears overlapping with the frame F) is deleted.

[0021] Further, the data generation unit 14 generates annotation data of the pseudo image from the converted foreground annotation data and the deleted background annotation data (step S7). In the synthesized image M of Figure 4, the converted foreground annotation data A1 and the deleted background annotation data coexist. Thereafter, the data generation unit 14 outputs the image data of the pseudo image including the annotation data of the pseudo image (step S8).

[0022] As described above, by performing transparency processing on areas other than the object to be composited in the foreground image, and by deleting background annotation data that overlaps with the object to be composited after the above-mentioned compositing, and then generating annotation data for the pseudo-image, it becomes unnecessary to repeatedly apply annotation data to areas that appear the same as the original image, such as images obtained by photographing only a part of an image under different conditions, or images obtained by photographing an image with a slightly shifted position or angle. Therefore, the processing load in a broad sense, including the time and manpower required to apply annotation data, the time and manpower required to collect data of various variations, and the processing load on the computer, is reduced, and a large number of pseudo-image data variations can be generated with a light load.

[0023] Furthermore, with the above-described embodiment, foreground images containing various objects to be synthesized and background images containing various backgrounds can be freely synthesized with minimal overhead, along with their annotation data. This allows for the generation of numerous variations of pseudo-image data, including synthesized images representing states that are impossible or rarely occur in the real world, with minimal overhead. This enables both reduced processing load and enhanced image diversity in the generation of training data, which is a prerequisite for artificial intelligence learning. For example, when synthesizing background image data with six types of annotation data and foreground image data with six types of annotation data, it becomes unnecessary to repeatedly apply background and foreground annotation data to each of the 36 possible synthesized images. By simply performing minimal overhead, such as deleting some background annotation data that overlaps with the object to be synthesized (frame F) and converting the foreground annotation data, 36 possible synthesized images can be obtained, thus achieving both reduced processing load and enhanced image diversity in the generation of training data.

[0024] Furthermore, in step S2 of Figure 2, an example is shown where the target area including the object to be composited (frame F) is cut out from the foreground image. However, such a cutting process is not mandatory. For example, if the target area including the object to be composited (frame F) corresponds to the entire foreground image, the cutting process is unnecessary.

[0025] Furthermore, it is not mandatory for the foreground image to have annotation data. For example, this disclosure is applicable and produces a similar effect when compositing a foreground image without annotation data onto a background image with background annotation data.

[0026] (Explanation of terms, explanation of hardware configuration (Figure 5), etc.) The block diagrams used in the above descriptions of embodiments and modifications show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining software with the one or more devices described above.

[0027] Functions include, but are not limited to, judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. As mentioned above, the method of implementation is not particularly limited.

[0028] For example, the pseudo-image generation device in one embodiment of the present disclosure may function as a computer that performs the processing in this embodiment. Figure 5 is a diagram showing an example of the hardware configuration of the pseudo-image generation device 10 according to one embodiment of the present disclosure. The pseudo-image generation device 10 described above may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0029] In the following description, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the pseudo-image generation device 10 may include one or more of the devices shown in the figure, or it may be configured to omit some of the devices.

[0030] Each function in the pseudo-image generation device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which causes the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of data reading and writing in the memory 1002 and storage 1003.

[0031] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, and so on.

[0032] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. Although the above processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0033] Memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 1002 may also be called a register, cache, main memory, etc. Memory 1002 can store executable programs (program code), software modules, etc., for carrying out a wireless communication method according to one embodiment of the present disclosure.

[0034] Storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. Storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database including at least one of memory 1002 and storage 1003, or other suitable medium.

[0035] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also called a network device, network controller, network card, communication module, etc.

[0036] The input device 1005 is an input device that accepts input from the outside (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to the outside (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel). Furthermore, each device such as the processor 1001 and memory 1002 are connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0037] Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0038] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0039] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described herein may be reordered, provided they are consistent with each other. For example, the methods described herein present various step elements in an exemplary order and are not limited to that specific order.

[0040] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.

[0041] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0042] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0043] In this disclosure, if articles are added through translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0044] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different." [Explanation of Symbols]

[0045] 10... Pseudo-image generation device, 11... Transparency processing unit, 12... Synthesis unit, 13... Deletion unit, 14... Data generation unit, 1001... Processor, 1002... Memory, 1003... Storage, 1004... Communication device, 1005... Input device, 1006... Output device, 1007... Bus.

Claims

1. A pseudo-image generation device that generates a pseudo-image from foreground image data relating to a foreground image of an object to be synthesized and background image data relating to a background, A transparency processing unit that performs a transparency process on areas in the foreground image other than the object to be synthesized, A combining unit combines the background image data and the foreground image data after transparency processing so that the foreground image after transparency processing by the transparency processing unit is superimposed on a predetermined position in the background image. A deletion unit deletes background annotation data that overlaps with the object to be synthesized after synthesis by the synthesis unit, in the background annotation data attached to the background image data included in the background image data. A data generation unit generates annotation data for the pseudo-image based on the background annotation data after deletion by the deletion unit, A pseudo-image generation device equipped with [a specific feature].

2. The aforementioned deletion section further, The foreground annotation data included in the foreground image data and attached to the foreground image is transformed according to the position in which the foreground image is superimposed on the background image. The data generation unit, Based on the converted foreground annotation data and the deleted background annotation data, the annotation data for the pseudo-image is generated. The pseudo-image generation apparatus according to claim 1.

3. The transparency processing unit is From the foreground image, the target region containing the object to be synthesized is cut out of the foreground image, and the region other than the object to be synthesized in the cut-out target region is made transparent. The pseudo-image generation apparatus according to claim 1.

Images