Pseudo image generation device
Patent Information
- Application Number
- JP2024507571
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2043-02-07
AI Technical Summary
The existing methods for generating image data with annotation in artificial intelligence learning require a high processing load due to the need for repeated annotation of images captured under different conditions or with slight positional or angular variations, leading to increased time, manpower, and computational requirements.
A pseudo image generation device that performs transparency processing on areas other than the synthesis target in the foreground image, combines it with the background image, deletes overlapping background annotation data, and generates pseudo-image annotation data, thereby reducing the need for repeated annotation across variations.
This approach significantly reduces the processing load by allowing the generation of multiple pseudo image variations with minimal effort, enabling efficient data collection and processing, and increasing the diversity of training data for artificial intelligence learning.
Abstract
Description
Pseudo image generation device
[0001] The present disclosure relates to a pseudo-image generating device that generates pseudo-images with annotation data used for AI learning, etc. Note that "annotation data" here refers to information related to an object in an image (also called "annotation," "tag," or "label"), and is generally synthesized in the vicinity of the object in the image.
[0002] In AI learning, for example, image data with annotation data for various plants is generated by superimposing the name of the plant as annotation data on an image of the plant near the plant in the image. The resulting large number of image data are then input as training data into an AI to learn the names of various plants. The generation of image data with annotation data is described, for example, in Patent Literature 1. Furthermore, it is desirable that the training data input to the AI for learning be a large number of image data with a wide variety of variations. Therefore, a composite image (an image showing a composite object in a background) obtained by combining a background image with a foreground image showing a certain composite object is often used as training data.
[0003] Japanese Patent Application Laid-Open No. 2006-108813
[0004] Incidentally, in generating the image data with annotation data described above, it was necessary to repeatedly add annotation data to images obtained by photographing only a portion of an entire image under different conditions, or to images obtained by photographing an image at a slightly different position or angle, even if the portions looked the same as in the original image. This resulted in a very high processing load in the broad sense, including the time and manpower required to add annotation data, the time and manpower required to collect various types of data, and the processing load on the computer.
[0005] The present disclosure aims to reduce the processing load as described above when generating image data with annotation data, and to generate a large number of variations of pseudo image data with a light load.
[0006] The pseudo image generating device of the present disclosure is a pseudo image generating device that generates a pseudo image from foreground image data relating to a foreground image that depicts a compositing object and background image data relating to a background image that depicts a background, and includes a transparency processing unit that performs a transparency process on areas in the foreground image other than the compositing object; a synthesis unit that synthesizes the background image data and the foreground image data after the transparency process so that the foreground image after the transparency process by the transparency processing unit is superimposed at a predetermined position on the background image; a deletion unit that deletes background annotation data that overlaps with the compositing object after compositing by the synthesis unit from background annotation data that is included in the background image data and added to the background image; and a data generation unit that generates annotation data of the pseudo image based on the background annotation data after deletion by the deletion unit.
[0007] In the pseudo image generating device, a transparency processing unit performs transparency processing on areas of the foreground image other than the compositing object. Here, "transparency processing" refers to a process for processing the foreground image so that areas other than the compositing object are visible through the image. Then, a compositing unit combines background image data and the foreground image data after the transparency processing so that the foreground image after the transparency processing is superimposed at a predetermined position on the background image. The composite image shows the compositing object, and the background appears in the areas other than the compositing object. Furthermore, a deletion unit deletes background annotation data that overlaps with the compositing object after the compositing process from background annotation data included in the background image data and added to the background image, and a data generation unit generates annotation data for the pseudo image based on the background annotation data after the deletion.
[0008] As described above, by generating annotation data for a pseudo image after performing transparency processing on areas in the foreground image other than the compositing object and deleting background annotation data that overlaps with the compositing object after compositing, it becomes unnecessary to repeatedly add annotation data to areas that appear the same as the original image, such as an image obtained by photographing only a portion of an entire image under different conditions, or an image obtained by photographing an image with a slightly shifted position or angle. This reduces the processing load in a broad sense, including the time and manpower required to add annotation data, the time and manpower required to collect data on various variations, and the processing load on the computer, and makes it possible to generate a large number of variations of pseudo image data with a light load.
[0009] According to the present disclosure, the processing load as described above in generating image data with annotation data can be reduced, and a large number of variations of pseudo image data can be generated with a light load.
[0010] It is a functional block diagram of a pseudo image generating device in an embodiment of the present invention. It is a flow diagram showing processing executed by the pseudo image generating device. It is a diagram for explaining processing up to synthesis of background image data and foreground image data. It is a diagram for explaining deletion of background annotation data. It is a diagram showing an example of the hardware configuration of the pseudo image generating device.
[0011] An embodiment of a pseudo image generating device according to the present disclosure will be described below with reference to the drawings. The pseudo image generating device according to the present disclosure is a pseudo image generating device that generates a pseudo image from foreground image data relating to a foreground image that captures a composite object and background image data relating to a background image that captures a background. An example configuration of the pseudo image generating device is shown in FIG.
[0012] 1, the pseudo image generating device 10 includes a transparency processing unit 11, a synthesis unit 12, a deletion unit 13, and a data generating unit 14. The functions of each unit will be outlined below, and details will be described later using FIGS.
[0013] The transparency processing unit 11 is a functional unit that performs transparency processing on the area other than the compositing object (here, a frame F described later) in the foreground image. More specifically, the transparency processing unit 11 cuts out a target area including the compositing object from the foreground image, and performs transparency processing on the area other than the compositing object in the target area obtained by cutting out the target area.
[0014] The composition unit 12 is a functional unit that composes background image data and foreground image data after transparency processing so that the foreground image after transparency processing by the transparency processing unit 11 is superimposed at a predetermined position on the background image.
[0015] The deletion unit 13 is a functional unit that deletes background annotation data that is included in the background image data and added to the background image, and that overlaps with a compositing object after compositing by the compositing unit 12, and further has a function of converting foreground annotation data that is included in the foreground image data and added to the foreground image according to the position where the foreground image is superimposed on the background image. For example, the deletion unit 13 converts coordinate information of the compositing object included in the foreground annotation data from coordinates in the coordinate system of the foreground image to coordinates in the coordinate system of the background image.
[0016] The data generation unit 14 is a functional unit that generates annotation data of a pseudo image based on the converted foreground annotation data and the deleted background annotation data, and also has a function of outputting image data of the pseudo image including the generated annotation data. Note that "output" includes various forms of output, such as display output to a display, print output to a printer, and data transmission to an external device.
[0017] Next, the processing executed in the pseudo image generating device 10 will be described with reference to the flow diagram of Fig. 2. For example, the processing in Fig. 2 is started when a predetermined execution start instruction is issued by the operator of the pseudo image generating device 10 as a trigger.
[0018] First, the transparency processing unit 11 acquires foreground image data including foreground annotation data and background image data including background annotation data (step S1), cuts out a target area including a frame from the foreground image (step S2) as shown in Fig. 3, and then performs transparency processing on the area other than the frame in the target area obtained by cutting out (step S3) as shown in Fig. 3. Note that the data acquisition in step S1 may be performed by receiving the data from an external device or by reading it from a database (not shown) stored in the pseudo image generating device 10, and the acquisition method is not limited to a specific method.
[0019] Next, the composition unit 12 generates image data of a pseudo image by combining the background image data and the foreground image data after the transparency processing so that the foreground image after the transparency processing is superimposed at a predetermined position on the background image (step S4). In this composite image, for example, as shown in Figure 3, the upper right part of the frame is reflected, and the wheat cultivation field in the background is reflected in the area outside the frame.
[0020] Next, the deletion unit 13 converts the foreground annotation data according to the position where the foreground image is superimposed on the background image (step S5). Specifically, the deletion unit 13 converts the coordinate information of the frame F included in the foreground annotation data from coordinates in the coordinate system of the foreground image to coordinates in the coordinate system of the background image. For example, FIG. 4 shows an example in which a foreground image P1 and a background image P2 are combined to obtain a composite image M. The coordinate information of the frame F included in the foreground annotation data a1 is converted from coordinates in the coordinate system of the foreground image to coordinates in the coordinate system of the background image, and foreground annotation data A1 including the converted coordinate information is generated. Note that, for convenience, in FIG. 4, annotation data is represented by a rectangle that surrounds part or all of the object to which the annotation data is added. Then, the deletion unit 13 deletes the background annotation data that overlaps with the frame in the combined image (step S6). For example, as shown in FIG. 4, background annotation data that overlaps with frame F in composite image M (that is, background annotation data A2 and A3 in background image P2 that are attached to the ears of wheat that overlap with frame F) is deleted.
[0021] Furthermore, the data generating unit 14 generates annotation data for a pseudo image from the converted foreground annotation data and the deleted background annotation data (step S7). In the composite image M of Fig. 4, the converted foreground annotation data A1 and the deleted background annotation data coexist. Thereafter, the data generating unit 14 outputs image data of the pseudo image including the annotation data of the pseudo image (step S8).
[0022] According to the embodiment described above, annotation data for a pseudo image is generated after performing transparency processing on areas in the foreground image other than the compositing object and deleting background annotation data that overlaps with the compositing object after the compositing. This eliminates the need to repeatedly add annotation data to areas that appear the same as the original image, such as an image obtained by photographing only a portion of an entire image under different conditions, or an image obtained by photographing an image with a slightly shifted position or angle. This reduces the processing load in a broad sense, including the time and manpower required to add annotation data, the time and manpower required to collect data on various variations, and the processing load on the computer, and makes it possible to generate a large number of variations of pseudo image data with a light load.
[0023] Furthermore, according to the above-described embodiment, foreground images showing various compositing objects and background images showing various backgrounds can be freely composited together with the annotation data with a low load. Therefore, a large number of variations of pseudo image data, including composite images that represent situations that are impossible or rarely occur in the real world, can be generated with a low load. This makes it possible to both reduce the processing load and increase the diversity of images in the teacher data generation that is a prerequisite for learning artificial intelligence. For example, when combining six types of background image data with annotation data and six types of foreground image data with annotation data, it is not necessary to repeatedly add background and foreground annotation data to each of the 36 composite images. Instead, a total of 36 composite images can be obtained by simply performing low-load processing, such as deleting some of the background annotation data that overlaps with the compositing object (frame F) and converting the foreground annotation data. This makes it possible to both reduce the processing load and increase the diversity of images in the teacher data generation.
[0024] In addition, in step S2 of Figure 2, an example is shown in which a target area including the compositing object (frame F) is cut out from the foreground image, but such cut-out processing is not essential, and for example, if the target area including the compositing object (frame F) corresponds to the entire foreground image, cut-out processing is not necessary.
[0025] Furthermore, it is not essential that the foreground image has annotation data. For example, the present disclosure can be applied to a case where a foreground image without annotation data is combined with a background image with background annotation data, and similar effects can be achieved.
[0026] (Explanation of terms, explanation of hardware configuration (Figure 5), etc.) Note that the block diagrams used in explaining the above embodiments and variant examples show functional blocks. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are connected directly or indirectly (for example, using wires, wirelessly, etc.) and these multiple devices. A functional block may be realized by combining software with the single device or multiple devices.
[0027] Functions include, but are not limited to, judgment, determination, assessment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs transmission is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on how these functions are implemented.
[0028] For example, the pseudo image generating device according to an embodiment of the present disclosure may function as a computer that performs the processing of this embodiment. Fig. 5 is a diagram showing an example of the hardware configuration of the pseudo image generating device 10 according to an embodiment of the present disclosure. The pseudo image generating device 10 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, etc.
[0029] In the following description, the term "apparatus" can be interpreted as a circuit, a device, a unit, etc. The hardware configuration of the pseudo image generating apparatus 10 may be configured to include one or more of the devices shown in the drawings, or may be configured to exclude some of the devices.
[0030] Each function of the pseudo-image generating device 10 is realized by loading specified software (programs) onto hardware such as the processor 1001 and memory 1002, causing the processor 1001 to perform calculations, control communication via the communication device 1004, and control at least one of reading and writing data in the memory 1002 and storage 1003.
[0031] The processor 1001 controls the entire computer by running, for example, an operating system, and may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc.
[0032] The processor 1001 also reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described in the above-described embodiments. While the above-described various processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may be transmitted from a network via a telecommunications line.
[0033] The memory 1002 is a computer-readable recording medium and may be configured by, for example, at least one of a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store executable programs (program codes), software modules, etc. for implementing a wireless communication method according to an embodiment of the present disclosure.
[0034] Storage 1003 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 1003 may also be referred to as an auxiliary storage device. The above-mentioned storage medium may be, for example, a database including at least one of memory 1002 and storage 1003, or other appropriate medium.
[0035] The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module.
[0036] The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 1005 and the output device 1006 may be integrated (e.g., a touch panel). Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses for each device.
[0037] The aspects / embodiments described in this disclosure may be used alone, in combination, or switched depending on the implementation. Notification of predetermined information (e.g., notification that "X is true") is not limited to explicit notification, but may be implicit (e.g., not notifying the predetermined information).
[0038] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.
[0039] The order of the procedures, sequences, flowcharts, etc. of each aspect / embodiment described in this disclosure may be changed unless it is consistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.
[0040] Input and output information may be stored in a specific location (for example, memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be transmitted to another device.
[0041] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."
[0042] When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, when the term "or" is used in this disclosure, it is not intended to be an exclusive or.
[0043] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.
[0044] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different."
[0045] 10...pseudo image generation device, 11...transparency processing unit, 12...combining unit, 13...deleting unit, 14...data generation unit, 1001...processor, 1002...memory, 1003...storage, 1004...communication device, 1005...input device, 1006...output device, 1007...bus.
Claims
1. A pseudo image generating device that generates a pseudo image from foreground image data relating to a foreground image showing a synthesis object and background image data relating to a background image showing a background, comprising: a transparency processing unit that performs a transparency process on an area other than the object to be synthesized in the foreground image; a synthesis unit that synthesizes the background image data and the foreground image data after the transparency processing by the transparency processing unit such that the foreground image after the transparency processing is superimposed at a predetermined position on the background image; a deletion unit that deletes background annotation data that is included in the background image data and added to the background image and that overlaps with the object to be synthesized after synthesis by the synthesis unit; a data generating unit that generates annotation data of the pseudo image based on the background annotation data after the deletion by the deleting unit; A pseudo image generating device comprising:
2. The deletion unit further converting foreground annotation data included in the foreground image data and added to the foreground image in accordance with a position where the foreground image is superimposed on the background image; The data generation unit generating annotation data of the pseudo image based on the converted foreground annotation data and the deleted background annotation data; 2. The pseudo image generating device according to claim 1.
3. The transparency processing unit includes: cutting out a target area including the object to be synthesized in the foreground image from the foreground image, and performing a process of making transparent an area other than the object to be synthesized in the target area obtained by cutting out the target area; 2. The pseudo image generating device according to claim 1.
