Image generation device and selection system

Abstract

To provide an image generation device and a selection system for rapidly and finely generating various different types of generation images regarding a removal target object including a new product.SOLUTION: The image generation device includes: a name acquisition unit for receiving an input name of a removal target; a base image acquisition unit for acquiring existing image data related to the removal target by searching the Internet using the name as a search keyword; and an image generation unit for generating learning data including a plurality of types of generated images of the removal target having at least different colors, postures, and states, by performing machine learning based on the image data.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an image generating device and a sorting system. [Background technology]

[0002] At waste disposal facilities, non-combustible materials are stored in containers called platforms before being crushed. The stored non-combustible materials may include items (objects to be removed) that may ignite or generate heat when crushed, such as devices with built-in lithium-ion batteries and gas cylinders. A device described in Patent Document 1 below is known as a device for removing these objects to be removed.

[0003] The device disclosed in Patent Document 1 below separates and stores waste into those to be supplied to a waste treatment device and those not, and includes a storage section for storing the waste, a sorting device for sorting the waste stored in the storage section by type, and a sorted waste storage compartment for storing the waste sorted according to type. The sorting device includes a waste characteristic detection means for detecting waste characteristic information, a robot arm for moving waste for which waste characteristic information has been detected to a compartment in the sorted waste storage compartment corresponding to the type of waste, and a control unit for identifying the type of waste from the waste characteristic information and controlling the robot arm. The control unit is said to perform machine learning based on waste identification information input in advance to generate learning data related to waste sorting. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2020-203249 Summary of the Invention [Problem to be solved by the invention]

[0005] However, non-combustible waste contains a wide range of new products and derivative products that are released into the market every day, which are different from existing product groups. For this reason, it is necessary to quickly generate or strengthen learning data every time a new product appears.

[0006] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an image generation device and a sorting system that are capable of quickly and precisely generating various generated images of objects to be removed, including new products. [Means for solving the problem]

[0007] In order to solve the above problem, the image generation device disclosed herein is an image generation device that generates a generated image of a specific object to be removed in a non-combustible waste treatment facility, and includes: a name acquisition unit that accepts an input name of the object to be removed; a basic image acquisition unit that acquires existing image data of the object to be removed by searching the Internet using the name as a search keyword; and an image generation unit that generates learning data including multiple types of generated images of the object to be removed that differ in at least color, posture, and state by performing machine learning based on the image data.

[0008] The sorting system according to the present disclosure includes an image generating device and a notification device that notifies an operator of the presence or absence of the objects to be removed in an actual non-combustible waste treatment facility based on the learning data generated by the image generating device. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide an image generating device and a sorting system that are capable of quickly and precisely generating various generated images of objects to be removed, including new products. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram showing a configuration of a sorting system according to a first embodiment of the present disclosure. [Figure 2]FIG. 1 is a functional block diagram of an image generating device according to a first embodiment of the present disclosure. [Figure 3] 4 is a flowchart showing a processing flow of the image generating device according to the first embodiment of the present disclosure. [Figure 4] FIG. 2 is an explanatory diagram showing a processing flow of the sorting system according to the first embodiment of the present disclosure. [Figure 5] FIG. 1 is a functional block diagram of a sorting system according to a first embodiment of the present disclosure. [Figure 6] 3 is a flowchart showing a processing flow of the sorting system according to the first embodiment of the present disclosure. [Figure 7] FIG. 10 is a schematic diagram showing the configuration of a reporting device according to a second embodiment of the present disclosure. [Figure 8] FIG. 2 is a hardware configuration diagram illustrating the configuration of a computer according to each embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] First Embodiment A sorting system 1 and an image generating device 21 according to a first embodiment of the present disclosure will be described below with reference to FIGS. 1 to 6. The sorting system 1 and the image generating device 21 are installed, for example, in a facility that crushes non-combustible materials (non-combustible material processing facility). They are primarily used to sort / remove specific removal targets 100 contained in non-combustible materials. The "specific removal targets 100" referred to here are items that may ignite / generate heat when subjected to physical impact, such as electronic devices with built-in lithium-ion batteries or gas cylinders used on a table. In addition, items that are unsuitable for processing as non-combustible materials are treated as specific removal targets 100.

[0012] (Configuration of sorting system 1) 1, the sorting system 1 includes a sorting device 10 and a control device 20. The sorting system 1 also includes a storage unit 11, a transport unit 12, a removal unit 13, a first sorting unit 14, a second sorting unit 15, and a camera unit 16.

[0013] The storage unit 11 is a container for storing non-combustible materials. The transport unit 12 is, for example, a belt conveyor, and sequentially sends the non-combustible materials stored in the storage unit 11 to the outside. The removal unit 13 is another belt conveyor connected downstream of the transport unit 12. The removal unit 13 is supported on the floor surface so as to be rotatable horizontally.

[0014] Under normal circumstances (i.e., when the non-combustible materials do not contain the removal targets 100), the removal target 13 is connected in series to the conveying unit 12, and its downstream end is connected to a first sorting unit 14 that stores the sorted non-combustible materials. On the other hand, if the control device 20, which will be described later, determines that the non-combustible materials being conveyed contain the removal targets 100, the removal unit 13 rotates. This causes the removal unit 13 to throw the removal targets 100 into a second sorting unit 15 adjacent to the first sorting unit 14. In this way, the removal targets 100 and the remaining non-combustible materials are separated.

[0015] The camera unit 16 captures images of the group of non-combustible materials being transported on the transport unit 12 using visible light, X-rays, or infrared rays. The images or videos captured by the camera unit 16 are sent as electrical signals to the control device 20. The control device 20 determines the presence or absence of the object to be removed 100 based on the images or videos, and controls the operation of the removal unit 13.

[0016] (Configuration of control device 20) 1, the control device 20 has an image generation device 21 and a detection device 22. The image generation device 21 generates a generated image (described later) for identifying the removal target 100. The detection device 22 detects the presence or absence of the removal target 100 based on the result of comparing the generated image with an image or video obtained from a camera.

[0017] (Configuration of image generating device 21) As shown in FIG. 2, the image generating device 21 includes a name acquiring unit 31, a basic image acquiring unit 32, an image generating unit 33, and a storage unit .

[0018] The name acquisition unit 31 receives the name of the removal target 100 as text input by the worker. The "name" here refers to a phrase that serves as a keyword for identifying the removal target 100, such as the model name, manufacturer name, model number, nickname, abbreviation, etc. of an electronic device, including a smartphone or smartwatch currently on the market.

[0019] The basic image acquisition unit 32 uses the name acquired by the name acquisition unit 31 as a search keyword and acquires image data (called basic images) of the item corresponding to the name using an internet search engine. The more basic images there are, the better, and for example, images introducing the product from a manufacturer, or images posted on review sites, mail order sites, etc. are preferably used. In other words, the basic images can be said to be images showing the appearance of the item in its normal or unused state.

[0020] The image generation unit 33 generates generated images using AI machine learning based on the multiple basic images. The term "generated images" here refers to multiple types of images of the item to be removed 100, with different colors, postures, angles, or conditions. More specifically, it is desirable to have a wide variety of generated images, such as images of models of the item in different colors, images of the item in a deteriorated and partially damaged state, images of the item buried under other items so that only a portion is visible, and even images that have been subjected to computational processing to enlarge, reduce, transform, and rotate. For example, if image data of an item photographed from the front and diagonally from the front are obtained as basic images, the image generation unit 33 generates generated images of the item viewed from all angles so as to complement the differences between these two images.

[0021] Examples of techniques for obtaining such a generated image include the following. The image generation unit 33 generates a predetermined image by, for example, executing a generative adversarial network (GAN). As a specific example, the image generation unit 33 generates a first image in a raster format using the generative adversarial network (GAN). For example, the GAN can use any of the following techniques: (1)pix2pix By learning the relationship between images from pairs of condition images and images, it is possible to create pairs learned from a single image. An image that complements the image relationship of A is generated. (2) CycleGAN Using two sets of images, one image is used to generate the other image, and the other image is used to return the first image. It is trained so that accuracy increases when the process is repeated (cycled). (3) CGAN (Conditional GAN) This is called a conditional GAN, where the generator and discriminator are given additional information in addition to the image data. Then we train them to be able to condition them. (4)DCGAN (Deep Convolutional GAN) DCGAN is a type of CGAN in a broad sense, and the main difference from the original GAN ​​is that it The generator and discriminator networks each have a fully connected layer. Instead, images are generated using convolutional layers and transposed convolutional layers. (5)PGGAN(Progressive Growing GAN) Unlike PGGAN and DCGAN, the resolution of the training data is gradually increased. At the same time, the generator and discriminator networks also add layers while maintaining their target structure to increase the resolution. By increasing the value, an image is generated. (6) BigGAN The generator uses orthogonal normalization and can conditionally generate high-resolution images up to 512x512 pixels. The generated model is used to generate an image. (7) StyleGAN Starting with low-resolution learning, the model is trained by gradually adding layers corresponding to higher resolutions. The image is generated using progressive growing. (8) StackGAN By configuring GAN in multiple stages, the first stage GAN can capture the overall image at low resolution. A high-resolution image is generated, and then a higher resolution image is generated in the subsequent stage of the GAN. (9) AttnGAN By paying attention to the individual words in text descriptions, such as image captions, The image is generated by combining details in different sub-regions of the image. The above-mentioned method is an example of GANs, and other GANs methods may also be used.

[0022] The generated image obtained in this manner is stored in the storage unit 34.

[0023] (Processing flow of image generation device 21) 3, in image generation device 21, name acquisition unit 31 accepts a name input by an operator (step S1). Next, basic image acquisition unit 32 acquires a basic image by an Internet search based on the name (steps S2 and S3). Finally, image generation unit 33 generates a generated image (step S4).

[0024] (Configuration of the detection device 22) As shown in FIG. 5, the detection device 22 includes a camera image acquisition unit 41, a generated image acquisition unit 42, a comparison unit 43, a drive unit 44, and a second storage unit 45.

[0025] The camera image acquisition unit 41 acquires images or videos of non-combustible materials as subjects from the camera unit 16 described above. The generated image acquisition unit 42 acquires generated images generated by the image generation device 21 described above. The comparison unit 43 compares the camera image with the generated image to determine whether the object to be removed 100 is included in the camera image. For example, this determination is made based on whether feature points of the object to be removed 100 included in the generated image are present in the camera image. It is desirable that this determination be realized by AI, as with image generation. The drive unit 44 controls the operation of the removal unit 13 based on the determination result of the comparison unit 43.

[0026] (Processing flow of the detection device 22) In the detection device 22, first, the camera image acquisition unit 41 acquires a camera image (step S11). Next, the generated image acquisition unit 42 acquires a generated image (step S12). In step S13, the comparison unit 43 compares the camera image with the generated image (step S13). Next, in step S14, if it is determined that the removal target 100 is included in the camera image (step S14: Yes), the drive unit 44 rotates the removal unit 13 to transport the removal target 100 to the second sorting unit 15. Once the transport of the removal target 100 is complete, the drive unit 44 returns the removal unit 13 to its initial position and posture (step S15). If it is determined No in step S14, the process returns to step S11 and continues.

[0027] The relationship between the data and each component in the sorting system 1 described above is as shown in Figure 4. In the figure, the image judger is a device that judges the appropriateness of a generated image, and is provided as part of the image generation unit 33. That is, the image judger evaluates the similarity with existing data stored in the memory unit 34 and determines whether the generated image is appropriate for use by the comparison unit 43. In addition, the detection AI in the figure is provided as a function of the comparison unit 43.

[0028] (Action and effect) The non-combustible materials stored in the storage unit 11 may include items (removal targets 100) that may ignite or generate heat when crushed, such as devices with built-in lithium-ion batteries and gas cylinders. Furthermore, such removal targets 100 include a wide range of items that differ from existing product lines, such as new products and derivative products that are released on the market every day. Therefore, it is necessary to quickly generate or strengthen machine learning learning data every time a new product is released. However, in the past, once learning data was generated, there was a problem that subsequent autonomous reinforcement learning or the like could not be performed, and the generated images became outdated. To solve this problem, the present embodiment employs the above-described configurations.

[0029] According to the above configuration, existing image data of a specific removal target 100 is obtained by searching the Internet using the name of the removal target 100 as a search keyword. The image generation unit 33 performs machine learning based on the image data to generate all kinds of generated images of the removal target 100 with different colors, postures, and states. As a result, even when a new product is released onto the market, for example, generated images of the product (removal target 100) in various states, angles, postures, or colors, as well as images of the product buried under other waste, can be immediately obtained simply by inputting the product's name. As a result, the removal target 100 can be identified and removed quickly and with high accuracy at the non-combustible waste treatment facility. This allows the non-combustible waste treatment facility to continue operating more stably and smoothly than before.

[0030] According to the above configuration, image data of new removal targets 100 can be acquired from the Internet each time, and existing learning data can be updated and strengthened each time. This makes it possible to keep the learning data up to date. Therefore, when a new product is released onto the market and the product is disposed of as non-combustible waste, if the product corresponds to the removal target 100, a generated image of the product can be immediately generated. As a result, the removal target 100 can be quickly identified and removed at the non-combustible waste treatment facility with high accuracy. Therefore, the non-combustible waste treatment facility can continue to operate more stably and smoothly than before.

[0031] According to the above configuration, a specific object to be removed 100 can be identified from a group of non-combustible objects by comparing the image generated by the image generating device 21 with an image of the actual non-combustible objects. Furthermore, by operating the removal unit 13 as needed, only the object to be removed 100 can be immediately removed from the group. This allows the object to be identified and removed quickly with high accuracy at the non-combustible object treatment facility. Therefore, the non-combustible object treatment facility can continue to operate more stably and smoothly than before.

[0032] The first embodiment of the present disclosure has been described above. Note that various changes and modifications can be made to the above configuration without departing from the gist of the present disclosure.

[0033] For example, the configuration of the removal unit 13 described above is just one example, and a robot arm other than a conveyor can also be used as the removal unit 13. In this case, it is advantageous in that it can capture / remove the removal target 100 more precisely and accurately than a conveyor.

[0034] Second Embodiment Next, a second embodiment of the present disclosure will be described with reference to Fig. 7. Note that the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0035] In this embodiment, in addition to the image generating device 21 and the detection device 22, a notification device 50 is further provided. The notification device 50 is a device for notifying a worker working in the actual storage section 11 of the presence or absence of the object to be removed 100 and its location based on the generated image.

[0036] As shown in FIG. 7 , the notification device 50 has a display unit 51 and an augmented reality generation unit 52. The display unit 51 is a head-mounted display or a glasses-like display that covers the worker's field of vision and is capable of transparently displaying the actual work site scene. The augmented reality generation unit 52 displays the presence of the removal target 100 identified by the detection device 22 by using light to overlay the actual scene transparently displayed on the display unit 51. Specifically, the outline of the removal target 100 can be highlighted with light or displayed in a color different from the surroundings, allowing the worker to perceive the presence of the removal target 100 as distinct from the surroundings. In addition, sound notification may be used in combination.

[0037] (Action and effect) According to the above configuration, the notification device 50 notifies the worker of the location of the object to be removed 100 in the actual treatment facility based on the learning data generated by the image generation device 21. This enables the worker to easily identify the location of the object to be removed 100 and perform the work of removing it.

[0038] According to the above configuration, the location of the object to be removed 100 can be displayed as augmented reality in the worker's field of vision using light while the worker is wearing the display unit 51. This allows the worker to continue working in a hands-free state, further improving the efficiency and safety of the work.

[0039] (Other embodiments) The above describes in detail the embodiments of the present disclosure with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present disclosure.

[0040] It should be noted that the control device 20 in the embodiment of the present disclosure may change the order of processing as long as appropriate processing is performed.

[0041] The storage unit 34, the second storage unit 45, and other storage devices in the embodiments of the present disclosure may be provided anywhere within a range where appropriate information can be transmitted and received. Furthermore, there may be multiple storage units 34 and other storage devices, and data may be stored in a distributed manner within a range where appropriate information can be transmitted and received.

[0042] The above-described processing steps performed by the control device 20 are stored in the form of a program on a recording medium that can be read by the computer 200, and the above processing is performed by the computer 200 reading and executing this program. A specific example of the computer 200 is shown below.

[0043] As shown in FIG. 8, the computer 200 includes a CPU 201, a main memory 202, a storage 203, and an interface 204. For example, the above-described control device 20 is implemented in a computer 200. The operations of the above-described processing units are stored in the form of a program in a storage 203. The CPU 201 reads the program from the storage 203, loads it into the main memory 202, and executes the above-described processing in accordance with the program. The CPU 201 also allocates a storage area in the main memory 202 corresponding to the above-described storage unit 34 in accordance with the program.

[0044] Examples of storage 203 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), and a semiconductor memory. Storage 203 may be an internal medium directly connected to the bus of computer 200, or an external medium connected to computer 200 via interface 204 or a communication line. Furthermore, when this program is distributed to computer 200 via a communication line, computer 200 that receives the program may load the program into main memory 202 and execute the above-mentioned processing. Storage 203 is a non-transitory tangible storage medium.

[0045] The program may also implement some of the functions described above. Furthermore, the program may be a file that can implement the functions described above in combination with a program already recorded in computer 200, a so-called differential file (differential program).

[0046] In addition to or instead of the above configuration, a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), or similar processing devices may be provided. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions realized by the processor may be realized by the integrated circuit.

[0047] <Additional Notes> The image generating device 21 and the sorting system 1 described in each embodiment can be understood, for example, as follows.

[0048] (1) The image generating device 21 according to the first aspect is an image generating device 21 that generates a generated image of a specific object to be removed 100 in a non-combustible waste treatment facility, and includes a name acquisition unit 31 that accepts an input name of the object to be removed 100, a basic image acquisition unit 32 that acquires existing image data of the object to be removed 100 by searching the Internet using the name as a search keyword, and an image generating unit 33 that generates learning data including multiple types of generated images of the object to be removed 100 that differ in at least color, posture, and condition by performing machine learning based on the image data.

[0049] According to the above configuration, for example, when a new product is released on the market, by simply inputting the name, it is possible to immediately obtain generated images of the product (object to be removed 100) in various states, angles, postures, or colors, as well as images of the product buried in other waste.

[0050] (2) The image generating device 21 according to the second aspect is the image generating device 21 of (1), in which the image generating unit 33 is capable of strengthening the learning data by machine learning the new generated image based on the image data relating to the new object 100 to be removed using the learning data that has already been generated.

[0051] According to the above configuration, it is possible to keep the learning data always up to date.

[0052] (3) The sorting system 1 according to the third aspect is a sorting system 1 equipped with an image generating device 21 of (1) or (2), wherein the non-combustible material treatment facility has a storage section 11 for storing non-combustible material, a transport section 12 for transporting the non-combustible material from the storage section 11, and a removal section 13 for removing the object to be removed 100 from the non-combustible material being transported, and the sorting system 1 further includes a drive section 44 for controlling the operation of the removal section 13 based on the learning data generated by the image generating device 21.

[0053] According to the above configuration, by operating the removal unit 13 as needed, it is possible to immediately remove only the object to be removed 100 from the group.

[0054] (4) The sorting system 1 according to the fourth aspect includes an image generating device 21 of (1) or (2) and a notification device 50 that notifies an operator of the location of the object to be removed 100 in the non-combustible waste treatment facility based on the learning data generated by the image generating device 21.

[0055] According to the above configuration, the worker can easily identify the position of the object 100 to be removed and perform the work of removing it.

[0056] (5) The sorting system 1 according to the fifth aspect is the sorting system 1 of (4), wherein the notification device 50 has a display unit 51 that covers the worker's field of vision and transparently displays the actual work site scene, and an augmented reality generation unit 52 that displays the presence of the object 100 to be removed by superimposing light on the scene.

[0057] According to the above configuration, it is possible to further improve the efficiency and safety of the work. [Explanation of symbols]

[0058] 1...Sorting system 10...Sorting device 11...Storage section 12...Transport section 13...Removal section 14...First sorting section 15...Second sorting section 16...Camera section 20...Control device 21...Image generation device 22...Detection device 31...Name acquisition section 32...Basic image acquisition section 33...Image generation section 34...Memory section 41...Camera image acquisition section 42...Generated image acquisition section 43...Comparing section 44...Drive section 45...Second memory section 50...Reporting device 51...Display section 52...Augmented reality generation section 100...Object to be removed 200...Computer 201...CPU 202...Main memory 203...Storage 204...Interface

Claims

1. An image generating device that generates an image of a specific object to be removed in a non-combustible waste treatment facility, a name acquisition unit that receives an input name of the object to be removed; a basic image acquisition unit that acquires existing image data related to the object to be removed by searching the Internet using the name as a search keyword; an image generation unit that performs machine learning based on the image data to generate learning data including a plurality of types of generated images of the removal target object that differ in at least color, posture, and state; An image generating device comprising:

2. The image generating device according to claim 1, wherein the image generating unit is capable of strengthening the learning data by performing machine learning to generate new generated images based on the image data relating to new objects to be removed using the learning data that has already been generated.

3. A sorting system comprising the image generating device according to claim 1 or 2, The non-combustible waste treatment facility includes: a storage section for storing non-combustible materials; a transport unit that transports the non-combustible material from the storage unit; a removal unit that removes the removal target material from the non-combustible material being transported; and The sorting system comprises: The sorting system further includes a drive unit that controls the operation of the removal unit based on the learning data generated by the image generation device.

4. The image generating device according to claim 1 or 2; a notification device that notifies an operator of the location of the object to be removed in the non-combustible waste treatment facility based on the learning data generated by the image generation device; A sorting system comprising:

5. The notification device a display unit that covers the worker's field of vision and displays the actual work site scene through a transparent display; an augmented reality generating unit that displays the presence of the object to be removed by superimposing light on the scene; The sorting system of claim 4, comprising:

Images