Determination device, determination method, program and collection device

Abstract

To provide a determination device or the like capable of suitably supporting collection of a PET bottle.SOLUTION: A determination device 2 includes: an acquisition part for acquiring an image having the imaged PET bottle: and a determination part which determines whether or not the PET bottle is in a recyclable state by inputting the acquired image into a learned model so as to be determined whether or not the PET bottle is in the recyclable state when the image is inputted; and an output part which outputs a recovery instruction of the PET bottle according to the determination result.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a determination device, a determination method, a program, and a recovery device.

Background Art

[0002] PET bottle collection for recycling is widely carried out. However, in order to recycle PET bottles, it is desirable to collect PET bottles in a certain recyclable state, such as removing the cap from the PET bottle and removing the label printed with the product name, etc. However, PET bottles are often put into the collection box with the cap still attached or the label not removed.

[0003] From the above circumstances, a method for collecting PET bottles in a recyclable state has been proposed. For example, in Patent Document 1, when the cap of a PET bottle is put into the cap inlet of a PET bottle collection box, a PET bottle / cap separation device that opens the inlet of the PET bottle is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the invention according to Patent Document 1 only separates the cap of the PET bottle, and for example, it cannot distinguish the label attached to the PET bottle or the contents of the PET bottle such as the remaining drink.

[0006] In one aspect, an object is to provide a determination device or the like that can suitably support the collection of PET bottles.

Means for Solving the Problems

[0007] The determination device according to one aspect includes an acquisition unit that acquires an image of a PET bottle, a determination unit that inputs the acquired image into a learned model that determines whether the PET bottle is in a recyclable state when the image is input, and determines whether the PET bottle is in a recyclable state, and an output unit that outputs a collection instruction for the PET bottle according to the determination result.

Effect of the Invention

[0008] In one aspect, it is possible to suitably support the collection of PET bottles.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0010] Hereinafter, the present invention will be described in detail based on the drawings showing its embodiments. (Embodiment 1) FIG. 1 is an explanatory diagram showing a configuration example of a PET bottle collection system. In the present embodiment, a PET bottle collection system that determines whether a PET bottle is in a recyclable state from an image of the PET bottle taken will be described. The PET bottle collection system includes a server 1, a collection device (determination device) 2, and a terminal 3. Each device is communicatively connected to a network N such as the Internet.

[0011] The server 1 is a server computer capable of various information processing and information transmission and reception. Note that the computer corresponding to the server 1 is not limited to a server computer and may be a personal computer or the like. In the present embodiment, the server 1 performs machine learning for learning predetermined training data and functions as a generation device that generates a determination model 50 for determining whether a PET bottle is in a recyclable state when an image of the PET bottle taken is input. The server 1 transmits the data of the generated determination model 50 to the collection device 2 and causes it to be installed.

[0012] The recycling device 2 is a recycling bin for recycling PET bottles. It uses the above-described determination model 50 to determine whether a PET bottle is in a recyclable state, and recycles the PET bottle according to the determination result. For example, the recycling device 2 is installed beside a beverage vending machine or at a predetermined facility (such as a retail store that sells beverages).

[0013] In addition, although only one recycling device 2 is shown in FIG. 1, in this embodiment, it will be described assuming that a plurality of recycling devices 2 are installed at multiple locations.

[0014] For example, the recycling device 2 includes a recycling bin main body 201, a PET bottle inlet 202, a lid 203, a camera 204, and a background plate 205. The recycling bin main body 201 is a housing into which PET bottles are inserted and accommodates the PET bottles inserted through the PET bottle inlet 202. The PET bottle inlet 202 is an opening provided in the top plate of the recycling bin main body 201 through which PET bottles are inserted. The lid 203 is a lid that covers the PET bottle inlet 202 and is a plate-like member on which PET bottles can be placed. As will be described later, the lid 203 is configured to be operable to open and close and is opened and closed according to the determination result of whether a PET bottle can be recycled.

[0015] The camera 204 is an imaging device that images the PET bottles placed on the lid 203. In this embodiment, it will be described assuming that the recycling device 2 (recycling bin) and the camera 204 are integrated, but the camera 204 may be installed at a position physically separated from the recycling device 2. The background plate 205 is a member provided so that the background of the image obtained by imaging the PET bottles is constant, and is provided in a form protruding upward from one side of the top plate of the recycling bin main body 201 located on the opposite side of the camera 204 when viewed from the PET bottle inlet 202 and the lid 203. to the recycling bin main body 201.

[0016] As described below, when a PET bottle is placed on the lid 203 which is normally closed, the recycling device 2 images the PET bottle with the camera 204. Then, the recycling device 2 inputs the captured image into the determination model 50 to determine whether the PET bottle is in a recyclable state. If it is determined that the PET bottle is in a recyclable state, the recycling device 2 opens the lid 203 and drops and moves the PET bottle inside the recycling box body 201.

[0017] In the present embodiment, the recycling device 2 (recycling box) for recycling PET bottles and the device for determining whether a PET bottle is in a recyclable state are described as being integrated. However, the present embodiment is not limited to this. For example, the determination of recyclability may be performed by a local computer (such as a personal computer) connected to the recycling device 2 or a server 1 on the cloud, and the recycling device 2 may only open and close the lid 203 according to the determination result. That is, the recycling device 2 for recycling PET bottles and the determination device for determining recyclability do not need to be integrated, and the two may be separate devices.

[0018] The terminal 3 is an information processing terminal possessed by a collector (such as a beverage manufacturer that installs vending machines, a staff member of a facility where the recycling device 2 is installed, etc.) who collects the PET bottles recycled by the recycling device 2, and is, for example, a smartphone, a personal computer, a tablet terminal, etc. As described below, the server 1 manages the recycling status of PET bottles in the recycling device 2, and may notify the terminal 3 of a collection instruction to collect the PET bottles from the recycling device 2 when the recycling box body 201 is full.

[0019] FIG. 2 is a block diagram showing a configuration example of the server 1. The server 1 includes a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary storage unit 14. The control unit 11 is an arithmetic processing device such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), etc. By reading and executing the program P1 stored in the auxiliary storage unit 14, it performs various information processing, control processing, etc. The main memory unit 12 is a temporary storage area such as SRAM (Static Random Access Memory), D RAM (Dynamic Random Access Memory), flash memory, etc., and temporarily stores the data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside.

[0020] The auxiliary storage unit 14 is a non-volatile storage area such as a large-capacity memory and a hard disk, and stores the program P1 and other data necessary for the control unit 11 to execute processing. In addition, the auxiliary storage unit 14 stores the recovery device DB141. The recovery device DB141 is a database that stores information on each recovery device 2 installed at a plurality of locations.

[0021] Note that the auxiliary storage unit 14 may be an external storage device connected to the server 1. Also, the server 1 may be a multi-computer composed of a plurality of computers, or may be a virtual machine virtually constructed by software.

[0022] Also, in this embodiment, the server 1 is not limited to the above configuration. For example, it may include an input unit for receiving operation inputs, a display unit for displaying images, etc. Also, the server 1 may be provided with a reading unit for reading a portable storage medium 1a such as a CD (Compact Disk)-ROM, DVD (Digital Versatile Disc)-ROM, etc., and reading and executing the program P1 from the portable storage medium 1a. Alternatively, the server 1 may read the program P1 from the semiconductor memory 1b.

[0023] FIG. 3 is a block diagram showing a configuration example of the recovery device 2. The recovery device 2 includes a control unit 21, a main storage unit 22, a communication unit 23, a display unit 24, an opening / closing drive unit 25, and an auxiliary storage unit 26. The control unit 21 is an arithmetic processing device such as one or more CPUs or MPUs, and performs various information processing, control processing, etc. by reading and executing the program P2 stored in the auxiliary storage unit 26. The main storage unit 22 is a temporary storage area such as a RAM, and temporarily stores data necessary for the control unit 21 to execute arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside. The display unit 24 is a display means such as a display or a predetermined indicator (such as an LED (Light Emitting Diode) lamp) (see FIGS. 7 and 8), and displays the determination result of whether recycling is possible. The opening / closing drive unit 25 is a drive mechanism that drives the opening / closing operation (for example, a slide operation) of the lid 203, and opens the lid 203 according to an opening instruction (recovery instruction) from the control unit 21.

[0024] The auxiliary storage unit 26 is a non-volatile storage area such as a hard disk or a large-capacity memory, and stores the program P2 and other data necessary for the control unit 21 to execute processing. Further, the auxiliary storage unit 26 stores the determination model 50. The determination model 50 is a machine learning model that has learned predetermined training data, and is a model that outputs a determination result of whether a PET bottle is in a recyclable state when an image of the PET bottle is input. The determination model 50 is assumed to be used as a program module that constitutes a part of artificial intelligence software.

[0025] Note that the recovery device 2 may include a reading unit that reads a portable storage medium 2a such as a CD-ROM, and reads and executes the program P2 from the portable storage medium 2a. Alternatively, the recovery device 2 may read the program P2 from the semiconductor memory 2b.

[0026] FIG. 4 is an explanatory diagram showing an example of the record layout of the collection device DB141. The collection device DB141 includes a device ID column, an installation location column, a maximum collection quantity column, and a collection status column. The device ID column stores a device ID for identifying each collection device 2 installed at a plurality of locations. The installation location column, the maximum collection quantity column, and the collection status column respectively store, in association with the device ID, the installation location of the collection device 2, the maximum number of PET bottles that can be accommodated in the collection box main body 201, and the number of collected bottles currently accommodated in the collection box main body 201.

[0027] FIG. 5 is an explanatory diagram regarding the determination process of recyclability. FIG. 5A shows a PET bottle in a recyclable state, and FIGS. 5B to 5D show PET bottles in a non-recyclable state. Based on FIGS. 5A to 5D, the determination process of recyclability based on the determination model 50 will be described.

[0028] As described above, the server 1 performs machine learning to learn predetermined training data, and generates a determination model 50 that takes an image obtained by imaging a PET bottle as an input and outputs a determination result of recyclability. The determination model 50 is, for example, a neural network generated by deep learning, and is a CNN (Convolution Neural Network; convolutional neural network) having a plurality of convolutional layers.

[0029] The determination model 50 includes an input layer that receives an input of an image, an intermediate layer that extracts feature amounts of the input image, and an output layer that outputs a determination result indicating whether or not it is in a recyclable state. The input layer receives an input of an image obtained by imaging a PET bottle, and transfers the input image data to the intermediate layer. The intermediate layer includes a plurality of convolutional layers that perform convolution on the image data, extracts feature amounts by convolving the image data, and transfers them to the output layer. The output layer outputs a determination result indicating whether or not it is in a recyclable state based on the feature amounts extracted by the intermediate layer.

[0030] The determination model 50 according to the present embodiment outputs a determination result of determining recyclability for each of a plurality of items used as determination criteria. The plurality of items are, for example, the key attached to the PET bottle These are three items: the presence or absence of a cap, the presence or absence of a label, and the presence or absence of the contents in the PET bottle.

[0031] Figures 5B to 5D illustrate, respectively, the state where the cap is attached to the PET bottle, the state where the label is pasted, and the state where the contents (beverage) remain in the PET bottle. In Figures 5B to 5D, the parts corresponding to the cap, the label, and the contents are illustrated with thick lines. When collecting the PET bottles in the states of Figures 5B to 5D, operations such as removing the cap or removing the label are required for recycling. Therefore, in this embodiment, the determination model 50 is used to detect these states, and as shown in Figure 5A, the PET bottle without a cap, a label, and contents is collected.

[0032] The server 1 generates the determination model 50 using training data in which correct data (correct label) indicating whether the PET bottle in the captured image for training is in a recyclable state is given to the captured image of the PET bottle for training. The correct data is data indicating the correct values of the determination results for each of the above three items. In this embodiment, as labels (determination results) that the determination model 50 can output, eight labels are prepared according to the presence or absence of a cap (two cases), the presence or absence of a label (two cases), and the presence or absence of contents (two cases), i.e., 2×2×2 = 8 cases. In the training data, one of the above eight labels is given as correct data for the training image.

[0033] The server 1 inputs the captured image of the PET bottle for training into the determination model 50 and obtains, as an output, the determination result of determining whether it can be recycled for each of the above items. The server 1 compares the determination result output from the determination model 50 with the correct data and optimizes parameters such as the weights between neurons so that the two are approximated. The server 1 performs learning on a plurality of captured images included in the training data and generates the determination model 50.

[0034] Note that the determination model 50 according to the present embodiment only determines whether there is content in the PET bottle, but it may also be configured to determine what the content in the PET bottle is. Specifically, the determination model 50 may be configured to determine whether the content in the PET bottle is a beverage or a foreign object (such as a cigarette butt) other than a beverage. In this case, for example, as the correct values of the determination items related to the presence or absence of the content, three correct values may be prepared including the state where the PET bottle is empty, the state where the beverage remains, and the case where a foreign object exists. Thereby, it is possible to separately detect a PET bottle in a state where a beverage remains and a PET bottle in which a foreign object other than a beverage exists.

[0035] For example, the recycling device 2 acquires the data of the determination model 50 from the server 1 via the network N and stores it in the auxiliary storage unit 26. The recycling device 2 determines whether the PET bottle is in a recyclable state using the determination model 50, and if it is determined that the PET bottle is in a recyclable state, the lid 203 is opened to recycle the PET bottle.

[0036] FIG. 6 is an explanatory diagram regarding the recycling process of the PET bottle. FIG. 7 is an explanatory diagram showing an example of the display of the determination result of recyclability. Based on FIGS. 6 and 7, the processing contents of the recycling device 2 will be described.

[0037] On the upper side of FIG. 6, a state where the camera 204 images the PET bottle placed on the lid 203 is illustrated. When the PET bottle is placed on the lid 203, the recycling device 2 images the PET bottle with the camera 204. Then, the recycling device 2 inputs the image of the PET bottle into the determination model 50 to determine whether the PET bottle is in a recyclable state. Specifically, the recycling device 2 makes determinations for each item of the presence or absence of a cap, the presence or absence of a label, and the presence or absence of content. Further, the recycling device 2 comprehensively determines whether all items are in a recyclable state, that is, whether the PET bottle has no cap, label, and content and is recyclable for recycling. In the following description, for convenience, the determination result obtained by combining the determination results of the above items is referred to as the "comprehensive determination result".

[0038] Note that before determining whether recycling is possible based on the determination model 50, the recycling device 2 may recognize the object shown in the image and determine whether the recognized object is a PET bottle. That is, before inputting the image into the determination model 50, the recycling device 2 determines whether the object shown in the image is a PET bottle. If it is determined that the object is a PET bottle, the image is input into the determination model 50 to determine whether it is in a recyclable state. This can prevent a situation where an object other than a PET bottle (e.g., an empty can) is discarded.

[0039] FIG. 7 illustrates a display screen when presenting the determination result of whether recycling is possible to the user (the person who discards the PET bottle). The recycling device 2 includes a display means such as a liquid crystal display as a display unit 24 (not shown in FIGS. 1 and 6), and the display unit 24 displays the screen. The screen includes a PET bottle image 71, item-by-item determination results 72, and a comprehensive determination result 73. The recycling device 2 displays the PET bottle image 71 captured by the camera 204 and displays the determination result of whether recycling is possible by the determination model 50 as the item-by-item determination results 72 and the comprehensive determination result 73.

[0040] The item-by-item determination results 72 are display columns that display the determination results of each item used as the determination criteria. For example, the recycling device 2 displays the determination results of each item with a tick or a cross, and emphasizes the item (cross) determined to be in a non-recyclable state by a method such as color-coding. In FIG. 7, for the sake of convenience, the color-coded state is illustrated by hatching.

[0041] Note that the recycling device 2 may not only emphasize the item determined to be in a non-recyclable state, but also present the location of the PET bottle corresponding to the item to the user. For example, the recycling device 2 recognizes the object (label in FIG. 7) corresponding to the item determined to be in a non-recyclable state from the PET bottle image 71, and may superimpose an object such as a bounding box on the PET bottle image 71 in the image area corresponding to the object. Thus, the method of presenting the item determined to be in a non-recyclable state to the user is not particularly limited.

[0042] The comprehensive determination result 73 is a display column that displays the comprehensive determination result of whether the state is recyclable in all items. For example, as shown in FIG. 7, the collection device 2 displays the comprehensive determination result with an X mark.

[0043] In this way, the collection device 2 displays the determination result of whether recycling is possible and presents it to the user. As shown in FIG. 7, when it is determined that the state is not recyclable, the item corresponding to the cause is displayed in the item-by-item determination result 72. In this case, the user removes the object (label in FIG. 7) related to the corresponding item, re-images the PET bottle, and makes a determination. Thereby, the collection of PET bottles can be suitably carried out.

[0044] By the way, in the example of FIG. 5, since it targets a product in which only a partial area is covered with a label and the bottom of the PET bottle is exposed in a side view (front view), there is no problem. However, depending on the product, there are also products in which most of the side surface of the PET bottle is covered with a label and the bottom is not exposed. In this case, it is assumed that the inside of the PET bottle is hidden by the label and cannot be seen, and the presence or absence of the contents cannot be determined simultaneously with the presence or absence of the label.

[0045] In this case, the collection device 2 can determine whether recycling is possible by processing as follows. For example, the collection device 2 separately provides a camera for bottom surface imaging on the lid 203 on which the PET bottle is placed, and acquires a bottom surface image obtained by imaging the PET bottle from the bottom surface. Then, when the collection device 2 determines that there is a label as a result of the determination by the above determination model 50, it determines the presence or absence of the contents from the bottom surface image. The determination based on the bottom surface image may use a machine learning model such as the determination model 50, or may be performed based on a rule (image pattern matching). In this way, by combining the image obtained by imaging the PET bottle from the side and the image obtained by imaging the PET bottle from the bottom surface, it is possible to determine the presence or absence of the contents regardless of the size of the label.

[0046] ​Note that the above process is just an example, and the present embodiment is not limited thereto. For example, when the recycling device 2 determines that there is a label on the PET bottle, it may display on the screen of FIG. 7 a guidance to remove the label and re-image. For example, the recycling device 2 may display "×" as the determination result of "label" in the item-by-item determination result 72, and display "△" (indeterminable) as the determination result of "contents", and display a guidance text to re-image after removing the label. Thereby, it is possible to suitably determine the presence or absence of the contents without imaging the PET bottle from the bottom surface.

[0047] FIG. 8 is an explanatory diagram showing another example of the display of the determination result. Although the case of displaying the determination result on the display screen has been described in FIG. 7, the display means for the determination result is not limited thereto. For example, as shown in FIG. 8, the recycling device 2 may include a dedicated display (indicator such as an LED lamp) for displaying the determination result of recyclability as the display unit 24, and display the determination result on the display. The display includes a plurality of item-specific lamps 81 for displaying the determination results of each item and a comprehensive lamp 82 for displaying the comprehensive determination result, and changes the display color of each lamp according to the determination result. Thereby, the display means for the determination result can be configured more simply than the display screen of FIG. 7.

[0048] Returning to FIG. 6 to continue the explanation. When it is determined that the PET bottle is in a recyclable state, the recycling device 2 collects the PET bottle. Specifically, as shown in the lower part of FIG. 6, the recycling device 2 outputs an opening instruction to open the lid 203 to the opening and closing drive unit 25 to open the lid 203, and drops and moves the PET bottle placed on the lid 203 into the collection box body 201.

[0049] Through the above process, the recycling device 2 collects PET bottles in a recyclable state. Here, it is preferable to store the collection status of the PET bottles collected in the collection box body 201, and when the collection box body 201 is full, let a predetermined collector collect the PET bottles.

[0050] For example, each recycling device 2 counts the number of times the lid 203 is opened and closed, that is, the number of PET bottles recycled, and transmits the number of recycled bottles to the server 1 via the network N. The server 1 stores the number of PET bottles recycled in each recycling device 2 in the recycling device DB 141. The server 1 compares the current number of recycled bottles with the maximum number of recycled bottles, and determines, for example, whether the number of recycled bottles is equal to or greater than the maximum number of recycled bottles. When it is determined that the number of recycled bottles is equal to or greater than the maximum number of recycled bottles, the server 1 notifies the terminal 3 of a collection instruction indicating the recycling device 2 for which the number of recycled bottles has become equal to or greater than the maximum number of recycled bottles. PET bottles can be suitably collected.

[0051] In the above, it is determined whether the recycling box main body 201 is full according to the number of PET bottles. However, the determination may be made based on the capacity (volume) of the PET bottles. In this case, for example, the recycling device 2 recognizes the capacity from the size of the PET bottles shown in the image, and may total the capacities of the PET bottles put into the recycling box main body 201 and compare it with the maximum storage capacity. Thus, it is sufficient that the recycling device 2 can compare the total amount of PET bottles stored in the recycling box main body 201 with a predetermined upper limit value.

[0052] Also, in the above, the collection instruction is notified to the terminal 3 (collector) via the server 1. However, a series of processes may be completed in the recycling device 2, and the collection instruction may be directly notified from the recycling device 2 to the terminal 3.

[0053] From the above, according to the first embodiment, recyclable PET bottles can be suitably discriminated, and PET bottle recycling can be supported.

[0054] FIG. 9 is a flowchart showing the procedure of the generation process of the determination model 50. Based on FIG. 9, the processing content when generating the determination model 50 by machine learning will be described. The control unit 11 of server 1 acquires training data for generating the determination model 50 (step S11). The training data is data to which correct data indicating whether or not the PET bottle is in a recyclable state is attached to the captured image of the training PET bottle. As described above, the correct data is data indicating the correct value of the determination result for each of a plurality of items such as the presence or absence of a cap, the presence or absence of a label, and the presence or absence of contents.

[0055] Based on the training data, the control unit 11 generates a determination model 50 for determining whether or not the PET bottle is in a recyclable state when an image of the PET bottle is input (step S12). Specifically, as described above, the control unit 11 generates a neural network such as a CNN as the determination model 50. The control unit 11 inputs the captured image of the training PET bottle into the determination model 50 and obtains, as an output, the determination result for each item such as the presence or absence of a cap and the presence or absence of a label. The control unit 11 compares the determination result output from the determination model 50 with the correct data, and optimizes parameters such as the weights between neurons so that the two are approximated, thereby generating the determination model 50. The control unit 11 ends a series of processes.

[0056] FIG. 10 is a flowchart showing the procedure of the PET bottle collection process. Based on FIG. 10, the processing content executed by the collection device 2 will be described. The control unit 21 of the collection device 2 captures an image of the PET bottle placed on the lid 203 covering the PET bottle inlet 202 with the camera 204 (step S31). The control unit 21 inputs the captured image into the determination model 50 to determine whether or not the PET bottle is in a recyclable state (step S32). Specifically, the control unit 21 makes a determination for each of a plurality of items used as the determination criteria for recyclability. The items are, for example, the presence or absence of a cap, the presence or absence of a label, and the presence or absence of contents. Note that the control unit 21 may determine whether or not the object shown in the image is a PET bottle before performing the determination process of step S32.

[0057] The control unit 21 displays the determination result of recyclability on the display unit 24 (step S33). Specifically, the control unit 21 displays the determination result of each of the above items, and also displays the comprehensive determination result of determining whether or not it is in a recyclable state for all items.

[0058] As described above, when the control unit 21 determines in step S32 that there is a label on the PET bottle, it may acquire a bottom image obtained by imaging the PET bottle from the bottom and determine the presence or absence of contents. Also, when it is determined that there is a label on the PET bottle, the control unit 21 may display a guidance to remove the label and re-image in step S33, and may re-image the PET bottle with the label removed and determine the presence or absence of contents.

[0059] Based on the determination result in step S32, the control unit 21 determines whether or not the PET bottle is in a recyclable state (step S34). If it is determined that it is not in a recyclable state (S34: NO), the control unit 21 ends the series of processes. If it is determined that it is in a recyclable state (S34: YES), the control unit 21 outputs an opening instruction (collection instruction) to open the lid 203 to the opening / closing drive unit 25 to open the lid 203 (step S35), and ends the series of processes.

[0060] In the above, the collection device 2 has been described as including a PET bottle collection means (a lid 203 on which a PET bottle can be placed and an opening / closing drive unit 25 for opening and closing the lid 203), but the present embodiment is not limited to this. For example, the collection device 2 may only display the determination result by the determination model 50 and display a collection instruction to the user to put (collect) the PET bottle into the PET bottle inlet 202 when it is in a recyclable state. That is, the collection device 2 (determination device) only needs to be able to output a PET bottle collection instruction according to at least the determination result of recyclability, and a configuration for operating the PET bottle collection means according to the collection instruction is not essential.

[0061] ​As described above, according to the first embodiment, it is determined by the determination model 50 whether or not the PET bottle imaged is in a recyclable state. The lid 203 is opened and closed according to the determination result, and the PET bottle in a recyclable state is collected. Further, when it is in a non-recyclable state, the determination items that are the causes are presented to the user, and the removal of the corresponding things (such as caps and labels) is promoted. Thereby, the collection of PET bottles can be suitably supported.

[0062] (Modification Example 1) The collection device 2 according to the first embodiment was a collection box having only the PET bottle inlet 202 as an inlet for throwing in waste. On the other hand, an inlet (second inlet) for accessory items such as the cap and label of the PET bottle may be provided in the collection device 2. In this modification example, the collection device 2 provided with a cap inlet 206 and a label inlet 207 in addition to the PET bottle inlet 202 will be described.

[0063] FIG. 11 is a schematic diagram showing a configuration example of the collection device 2 according to the first modification example. The collection box body 201 of the collection device 2 according to this modification example includes a cap inlet 206 and a label inlet 207 in addition to the PET bottle inlet 202. The cap inlet 206 and the label inlet 207 are openings for throwing the cap and label (articles) attached to the PET bottle into the collection box body 201, respectively, and are normally covered with lids 208 and 209. The lids 208 and 209 of the cap inlet 206 and the label inlet 207 are configured to be operable to open and close according to an instruction from the control unit 21, similar to the PET bottle inlet 202.

[0064] In addition to the cap inlet 206 and the label inlet 207, an inlet for discarding the contents (beverage) of the PET bottle may be provided.

[0065] The recycling device 2 opens and closes the PET bottle inlet 202, the cap inlet 206, and the label inlet 207 according to the determination result of recyclability based on the determination model 50. That is, similar to the first embodiment, when the recycling device 2 determines that the overall determination result is a recyclable state, the PET bottle inlet 202 is opened to collect the PET bottle. On the other hand, when it is determined that there is a cap and / or a label, the recycling device 2 outputs an opening instruction to an opening and closing drive unit (not shown) that opens and closes the lids 208 and 209, and opens the cap inlet 206 and / or the label inlet 207. Thereby, the separation of caps and labels can be promoted, and the convenience of the user can be improved.

[0066] Since other aspects are the same as those in the first embodiment, detailed illustrations and descriptions such as flowcharts are omitted in this embodiment.

[0067] (Modification Example 2) In the first embodiment, the form in which the recycling device 2 determines the recyclability from the captured image of the PET bottle and collects the PET bottle has been described. Here, the image of the PET bottle captured by the recycling device 2 may be used as training data for re-learning to update (re-learn) the determination model 50. In this modification example, the update process of the determination model 50 will be described.

[0068] FIG. 12 is a flowchart showing the procedure of the update process of the determination model 50. Based on FIG. 12 Next, the processing contents of the update process of the determination model 50 will be described. The control unit 11 of the server 1 acquires the image of the PET bottle captured by the camera 204 from each recycling device 2 (step S101). For example, the recycling device 2 stores the captured image of the PET bottle to be determined in the auxiliary storage unit 26 and periodically transmits the captured image to the server 1.

[0069] The server 1 may obtain images of all the PET bottles to be determined from the collection device 2, or may obtain only images of some PET bottles and use them for re-learning. For example, the server 1 may obtain an image determined to be in a non-recyclable state among the PET bottles to be determined. Also, for example, the server 1 may obtain an image of a PET bottle in which the probability value indicating the certainty of the determination result, which is output together with the determination result of recyclability from the determination model 50, belongs to a predetermined numerical range (for example, a numerical range around 50%). Thereby, it is possible to select images suitable as training data for re-learning, such as images of PET bottles in a non-recyclable state or images of PET bottles with uncertain determination results.

[0070] The control unit 11 receives a setting input for setting correct data indicating whether or not the obtained PET bottle image is in a recyclable state from the administrator of this system (step S102). The correct data set in step S102 is data representing the correct values of recyclability for each item such as the cap and label, similar to the first embodiment.

[0071] The control unit 11 updates the determination model 50 using the image of the PET bottle for which the correct data has been set (assigned) in step S102 as training data for re-learning (step S103). That is, the control unit 11 inputs the image of the PET bottle for re-learning into the determination model 50 and optimizes parameters such as the weights between neurons so that the determination result output from the determination model 50 approximates the correct data. The control unit 11 transmits the data of the updated determination model 50 to the collection device 2 (step S104) and ends a series of processes.

[0072] In the flowchart of FIG. 12, the process subject for updating the determination model 50 has been described as the server 1, but the collection device 2 may locally update the determination model 50.

[0073] As described above, according to this Modification Example 2, the determination model 50 can be updated through the operation of this system, and the determination accuracy of recyclability can be improved.

[0074] (Embodiment 2) In Embodiment 1, a system for end-users mainly using PET bottles was described. In this embodiment, a system for a business operator who sorts a large number of PET bottles discarded by end-users will be described.

[0075] FIG. 13 is a schematic diagram showing a configuration example of a PET bottle collection system according to Embodiment 2. The PET bottle collection system according to this embodiment has a determination device 4, a belt conveyor 5, and a camera 6 instead of the collection device 2. For example, the determination device 4 is communicatively connected to the server 1 via the network N, and acquires the data of the determination model 50 from the server 1 and stores it in the local storage.

[0076] The determination device 4 is a device that determines the recyclability of PET bottles using the determination model 50, and is, for example, a personal computer. Note that the server 1 and the determination device 4 may be configured as an integrated computer. Similar to the collection device 2 of Embodiment 1, the determination device 4 may be configured as an integrated device with the belt conveyor 5, the camera 6, etc.

[0077] The belt conveyor 5 is a belt conveyor for sorting waste (PET bottles), and is a device that conveys PET bottles in one direction. The belt conveyor 5 conveys a PET bottle (not shown in FIG. 13) placed on the conveyor belt in one direction, and an operator sorts the PET bottles on the conveyor belt into recyclable ones and non-recyclable ones. Note that a PET bottle collection means (collection device) such as a robot arm may be provided in the system of FIG. 13 so that PET bottles can be sorted mechanically without relying on an operator.

[0078] The camera 6 is an imaging device that images the PET bottles conveyed by the belt conveyor 5. For example, as shown in FIG. 13, the camera 6 is disposed directly above the belt conveyor 5 and images a plurality of PET bottles conveyed by the belt conveyor 5.

[0079] FIG. 14 is a block diagram showing a configuration example of the determination device 4. The determination device 4 includes a control unit 41, a main storage unit 42, a communication unit 43, a display unit 44, and an auxiliary storage unit 45. The control unit 41 is an arithmetic processing device such as one or more CPUs or MPUs, and performs various information processes, control processes, etc. by reading and executing the program P4 stored in the auxiliary storage unit 45. The main storage unit 42 is a temporary storage area such as a RAM, and temporarily stores data necessary for the control unit 41 to execute arithmetic processing. The communication unit 43 is a communication module for performing communication-related processes, and transmits and receives information to and from the outside. The display unit 44 is a display screen such as a liquid crystal display, and displays an image given from the control unit 41. The auxiliary storage unit 45 is a non-volatile storage area such as a hard disk or a large-capacity memory, and stores the program P4 and other data necessary for the control unit 41 to execute processing. Further, the auxiliary storage unit 45 stores the determination model 50.

[0080] Note that the determination device 4 may include a reading unit that reads a portable storage medium 4a such as a CD-ROM, and reads and executes the program P4 from the portable storage medium 4a. Alternatively, the determination device 4 may read the program P4 from the semiconductor memory 4b.

[0081] FIG. 15 is an explanatory diagram regarding the determination process of recyclability according to the second embodiment. Based on FIG. 15, the outline of this embodiment will be described. As described above, the camera 6 images a plurality of PET bottles conveyed by the belt conveyor 5. In this embodiment, it is assumed that a plurality of PET bottles are included in one captured image (frame).

[0082] The determination device 4 inputs the image captured by the camera 6 into the determination model 50 to determine whether the PET bottle is in a recyclable state. Here, the determination device 4 determines whether each of the plurality of PET bottles included in the captured image is in a recyclable state.

[0083] For example, before inputting the image into the determination model 50, the determination device 4 recognizes individual PET bottles from the image. The determination device 4 extracts the image region corresponding to each PET bottle from the image and inputs each image region into the determination model 50 individually. Thereby, the determination device 4 determines whether each PET bottle is in a recyclable state or not. Note that, similar to the first embodiment, the determination device 4 may perform determination for each item such as the presence or absence of a cap and the presence or absence of a label, or may perform only the comprehensive determination of whether it is in a recyclable state or not.

[0084] In the above, the image region of each individual PET bottle is extracted from the image captured by the camera 6 for determination, but the present embodiment is not limited to this. For example, the determination device 4 prepares a determination model 50 that has learned an image captured image in which recyclable PET bottles and non-recyclable PET bottles are mixed as training data, and may be able to directly determine the recyclability of each PET bottle from an image in which a plurality of PET bottles are shown. In this way, the determination device 4 only needs to be able to determine the recyclability of each PET bottle from an image in which a plurality of PET bottles are shown, and the specific processing method is not particularly limited. The determination device 4 displays the determination result of whether each PET bottle is recyclable or not. For example, as shown in FIG. 15, the determination device 4 displays the image captured by the camera 6 and superimposes a display label (such as a bounding box) indicating the image region of each PET bottle. Then, the determination device 4 indicates whether each PET bottle is in a recyclable state or not, for example, by changing the display color of the display label. In FIG. 15, for the sake of illustration, the bounding box of the PET bottle in a non-recyclable state is shown with a thick line.

[0085] The determination device 4 displays the determination result of whether each PET bottle is recyclable or not. For example, as shown in FIG. 15, the determination device 4 displays the image captured by the camera 6 and superimposes a display label (such as a bounding box) indicating the image region of each PET bottle. Then, the determination device 4 indicates whether each PET bottle is in a recyclable state or not, for example, by changing the display color of the display label. In FIG. 15, for the sake of illustration, the bounding box of the PET bottle in a non-recyclable state is shown with a thick line.

[0086] The determination device 4 outputs a collection instruction for the PET bottle according to the above determination result of recyclability. For example, the determination device 4 outputs a collection instruction for the PET bottle determined to be in a non-recyclable state among the plurality of PET bottles included in the image. Note that a collection instruction for a recyclable PET bottle instead of a non-recyclable PET bottle may be output.

[0087] The determination device 4 displays, for example, an alert indicating that a non-recyclable PET bottle has been detected. Alternatively, the determination device 4 may notify the collection instruction to a mobile terminal (such as a smartphone) carried by an operator who performs the sorting operation of the PET bottles. Alternatively, when a collection means (collection device) such as a robot arm is provided in the present system, the determination device 4 may output a collection instruction to the collection means to collect the corresponding PET bottle.

[0088] FIG. 16 is a flowchart showing the procedure of the PET bottle collection process according to Embodiment 2. Based on FIG. 16, the processing content executed by the determination device 4 will be described. The control unit 41 of the determination device 4 acquires an image obtained by imaging a plurality of PET bottles conveyed by the belt conveyor 5 from the camera 6 (step S201). The control unit 41 inputs the acquired image into the determination model 50 and determines whether each of the plurality of PET bottles included in the image is in a recyclable state (step S202). The control unit 41 displays the determination result of whether each PET bottle is in a recyclable state (step S203).

[0089] The control unit 41 determines whether there is a PET bottle in a non-recyclable state among a plurality of PET bottles to be determined (step S204). If it is determined that there is a PET bottle in a non-recyclable state (S204: YES), the control unit 41 outputs a collection instruction for the corresponding PET bottle (step S205). For example, the control unit 41 may perform an alert display on the display unit 44 of its own device, may directly notify the operator who performs the sorting operation, or may control the operation of a collection means such as a robotic arm.

[0090] After executing the process of step S205, or when the result in step S204 is NO, the control unit 41 returns the process to step S201. Thereby, the control unit 41 continuously determines whether the PET bottles conveyed by the belt conveyor 5 can be recycled and outputs a collection instruction.

[0091] As described above, according to the second embodiment, it is determined whether each PET bottle is in a recyclable state from the images of a plurality of PET bottles, and a collection instruction is output. Thereby, it is possible to cope even when sorting a large number of PET bottles.

[0092] (Embodiment 3) FIG. 17 is a functional block diagram showing the operation of the collection device 2 or the determination device 4 of the above-described embodiment. When the control unit 21 or the control unit 41 executes the program P2 or P4, the collection device 2 or the determination device 4 operates as follows. The acquisition unit 171 acquires an image of a PET bottle. The determination unit 172 inputs the acquired image into a learned model that determines whether the PET bottle is in a recyclable state when the image is input, and determines whether the PET bottle is in a recyclable state. The output unit 173 outputs a collection instruction for the PET bottle according to the determination result.

[0093] The third embodiment is as described above, and the other parts are the same as those in the first and second embodiments. Therefore, the corresponding parts are denoted by the same reference numerals and the detailed description thereof is omitted.

[0094] The embodiments disclosed this time should be considered as illustrative in all aspects and not restrictive. The scope of the present invention is shown not by the above meaning but by the claims, and it is intended that all modifications within the meaning and scope equivalent to the claims are included.

Explanation of Signs

[0095] 1 Server 11 Control Unit 12 Main Memory Unit 13 Communication Unit 14 Auxiliary Storage Unit P1 Program 2 Recycling Device (Judgment Device) 21 Control Unit 22 Main Memory Unit 23 Communication Unit 24 Display Unit 25 Opening / Closing Driving Unit 26 Auxiliary Storage Unit P2 Program 50 Judgment Model 201 Recycling Box Body 202 PET Bottle Inlet 203, 208, 209 Lids 204 Camera 205 Background Board 206 Cap Inlet (Second Inlet) 207 Label Inlet (Second Inlet) 3 Terminal 4 Judgment Device 41 Control Unit 42 Main Memory Unit 43 Communication Unit 44 Display Unit 45 Auxiliary Storage Unit P2 Program 5 Belt Conveyor 6 Camera

Claims

1. An acquisition unit that acquires an image of a PET bottle; A determination unit that inputs the acquired image into a pre-trained model that determines whether the PET bottle is in a recyclable state for a plurality of items including whether the content of the PET bottle is a beverage or a foreign object, and determines whether the PET bottle is in a recyclable state for each of the plurality of items; An output unit that outputs a collection instruction for the PET bottle according to the determination result; A display unit that displays a determination result indicating whether the PET bottle is in a recyclable state for each of the plurality of items A determination device comprising the above.

2. The plurality of items further includes the presence or absence of a cap of the PET bottle and the presence or absence of a label The determination device according to claim 1.

3. When it is determined that the PET bottle has a label, a second acquisition unit that acquires a bottom image of the PET bottle imaged from the bottom is provided, The determination unit determines whether the content of the PET bottle is a beverage or a foreign object based on the bottom image The determination device according to claim 2.

4. When it is determined that the PET bottle has a label, the display unit displays a guidance to remove the label, The acquisition unit acquires an image obtained by re-imaging the PET bottle, The determination unit inputs the re-imaged image into the model and determines whether the content of the PET bottle is a beverage or a foreign object The determination device according to claim 2.

5. The acquisition unit acquires the images of a plurality of the PET bottles, The determination unit inputs the acquired images into the model and determines whether each PET bottle is in a recyclable state The determination device according to any one of claims 1 to 4.

6. The output unit outputs a collection instruction for the PET bottle determined to be in a non-recyclable state among the plurality of PET bottles included in the image The determination device according to claim 5.

7. A second determination unit that determines whether the object shown in the image is a PET bottle is provided, When the determination unit determines that the object is a PET bottle, the determination unit inputs the image into the model and determines whether the PET bottle is recyclable The determination device according to any one of claims 1 to 6.

8. A reception unit that receives a setting input for setting correct answer data indicating whether or not the PET bottle is in a recyclable state with respect to the image to be determined; An update unit that updates the model based on the image and the correct answer data The determination device according to any one of claims 1 to 7, comprising:

9. Obtain an image of a PET bottle, Input the obtained image into a learned model that determines whether or not the PET bottle is in a recyclable state for each of a plurality of items including whether the contents of the PET bottle are a beverage or a foreign object when the image is input, and determine whether or not the PET bottle is in a recyclable state for each of the plurality of items, Output a collection instruction for the PET bottle according to the determination result, Display a determination result indicating whether or not the PET bottle is in a recyclable state for each item for each of the plurality of items A determination method in which a computer executes a process.

10. Obtain an image of a PET bottle, Input the obtained image into a learned model that determines whether or not the PET bottle is in a recyclable state for each of a plurality of items including whether the contents of the PET bottle are a beverage or a foreign object when the image is input, and determine whether or not the PET bottle is in a recyclable state for each of the plurality of items, Output a collection instruction for the PET bottle according to the determination result, Display a determination result indicating whether or not the PET bottle is in a recyclable state for each item for each of the plurality of items A program for causing a computer to execute a process.

11. A collection box body having an inlet for a PET bottle, A lid that covers the inlet and is configured to be able to place a PET bottle and move the placed PET bottle into the collection box body in accordance with an opening operation, An opening / closing drive unit that opens and closes the lid, An imaging unit that images the PET bottle placed on the lid, A determination unit that inputs the imaged image into a learned model that determines whether or not the PET bottle is in a recyclable state for each of a plurality of items including whether the contents of the PET bottle are a beverage or a foreign object when the image of the PET bottle is input, and determines whether or not the PET bottle is in a recyclable state for each of the plurality of items A display unit that displays, for each of the plurality of items, a determination result indicating whether or not the PET bottle is recyclable for each item. When it is determined that the PET bottle is in a recyclable state, the opening / closing drive unit opens the lid. Recycling device.

Images