Determination device, determination method, program, and recovery device

Abstract

To provide a determination device or the like capable of suitably supporting collection of a PET bottle.SOLUTION: A determination device 2 includes: an acquisition part for acquiring an image having the imaged PET bottle: and a determination part which determines whether or not the PET bottle is in a recyclable state by inputting the acquired image into a learned model so as to be determined whether or not the PET bottle is in the recyclable state when the image is inputted; and an output part which outputs a recovery instruction of the PET bottle according to the determination result.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a determination device, a determination method, a program, and a recovery device.

Background Art

[0002] PET bottle collection for recycling is widely carried out. However, in order to recycle PET bottles, it is desirable to collect the PET bottles in a certain recyclable state, such as removing the cap from the PET bottle and removing the label printed with the product name, etc. However, PET bottles are often put into the collection box with the cap still attached or the label not removed.

[0003] From the above circumstances, a method for collecting PET bottles in a recyclable state has been proposed. For example, in Patent Document 1, when the cap of a PET bottle is put into the cap inlet of a PET bottle collection box, a PET bottle / cap separation device that opens the inlet of the PET bottle is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the invention according to Patent Document 1 only separates the cap of the PET bottle, and for example, it cannot discriminate about the label attached to the PET bottle or the contents of the PET bottle such as the remaining drink.

[0006] One aspect aims to provide a determination device or the like that can suitably support the collection of PET bottles.

Means for Solving the Problems

[0007] The determination device for one aspect includes an acquisition unit that acquires an image of a PET bottle, a determination unit that inputs the acquired image to a model that has been trained to determine whether or not the PET bottle is in a recyclable state when the image is input and determines whether or not the PET bottle is in a recyclable state, and an output unit that outputs a request to collect the PET bottle according to the determination result. [Effects of the Invention]

[0008] In one respect, it can effectively support the collection of plastic bottles. [Brief explanation of the drawing]

[0009] [Figure 1] This is an explanatory diagram showing an example configuration of a PET bottle recycling system. [Figure 2] This is a block diagram showing an example server configuration. [Figure 3] This is a block diagram showing an example of the configuration of a recovery device. [Figure 4] This is an explanatory diagram showing an example of the record layout of the recovery device database. [Figure 5] This is an explanatory diagram regarding the process for determining whether or not an item is recyclable. [Figure 6] This is an explanatory diagram regarding the collection and processing of plastic bottles. [Figure 7] This is an explanatory diagram showing an example of how the results of the recycling eligibility determination are displayed. [Figure 8] This is an explanatory diagram showing other examples of how the judgment results are displayed. [Figure 9] This is a flowchart showing the steps involved in generating the judgment model. [Figure 10] This is a flowchart showing the procedure for recycling and processing plastic bottles. [Figure 11] This is a schematic diagram showing an example of the configuration of the recovery device 2 according to Modification 1. [Figure 12] This is a flowchart showing the procedure for updating the judgment model. [Figure 13] It is a schematic diagram showing a configuration example of a PET bottle recycling system according to Embodiment 2. [Figure 14] It is a block diagram showing a configuration example of a determination device. [Figure 15] It is an explanatory diagram regarding the determination process of recyclability according to Embodiment 2. [Figure 16] It is a flowchart showing the procedure of the PET bottle collection process according to Embodiment 2. [Figure 17] It is a functional block diagram showing the operation of the collection device in the above-described embodiment.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, the present invention will be described in detail based on the drawings showing its embodiments. (Embodiment 1) FIG. 1 is an explanatory diagram showing a configuration example of a PET bottle recycling system. In the present embodiment, a PET bottle recycling system that determines whether a PET bottle imaged is in a recyclable state from an image of the PET bottle will be described. The PET bottle recycling system includes a server 1, a collection device (determination device) 2, and a terminal 3. Each device is communicatively connected to a network N such as the Internet.

[0011] The server 1 is a server computer capable of various information processing and information transmission and reception. Note that the computer corresponding to the server 1 is not limited to a server computer and may be a personal computer or the like. In the present embodiment, the server 1 performs machine learning for learning predetermined training data and functions as a generation device that generates a determination model 50 for determining whether a PET bottle imaged is in a recyclable state when an image of the PET bottle is input. The server 1 transmits the data of the generated determination model 50 to the collection device 2 for installation.

[0012] The recycling device 2 is a recycling bin for recycling PET bottles. It is a device that uses the above determination model 50 to determine whether a PET bottle is in a recyclable state and recycles the PET bottle according to the determination result. For example, the recycling device 2 is installed beside a vending machine for beverages or in a predetermined facility (such as a retail store that sells beverages).

[0013] In addition, although only one recycling device 2 is illustrated in FIG. 1, in the present embodiment, it will be described assuming that a plurality of recycling devices 2 are installed at a plurality of locations.

[0014] For example, the recycling device 2 includes a recycling bin main body 201, a PET bottle inlet 202, a lid 203, a camera 204, and a background plate 205. The recycling bin main body 201 is a housing into which PET bottles are inserted and accommodates the PET bottles inserted through the PET bottle inlet 202. The PET bottle inlet 202 is an opening provided in the top plate of the recycling bin main body 201 through which PET bottles are inserted. The lid 203 is a lid that covers the PET bottle inlet 202 and is a plate-like member on which a PET bottle can be placed. As will be described later, the lid 203 is configured to be operable to open and close and is opened and closed according to the determination result of whether a PET bottle can be recycled.

[0015] The camera 204 is an imaging device that images the PET bottle placed on the lid 203. In the present embodiment, it will be described assuming that the recycling device 2 (recycling bin) and the camera 204 are integrated, but the camera 204 may be installed at a position physically separated from the recycling device 2. The background plate 205 is a member provided so that the background of the image of the PET bottle being imaged is constant, and is provided in a shape protruding upward from one side of the top plate of the recycling bin main body 201 located on the opposite side of the camera 204 when viewed from the PET bottle inlet 202 and the lid 203. to be located.

[0016] As described later, when a PET bottle is placed on the lid 203, which is normally closed, the collection device 2 takes an image of the PET bottle with the camera 204. The collection device 2 then inputs the image into the judgment model 50 to determine whether or not the PET bottle is in a recyclable state. If it is determined that the PET bottle is in a recyclable state, the collection device 2 opens the lid 203 and drops the PET bottle into the collection box body 201.

[0017] In this embodiment, the collection device 2 (collection box) for collecting PET bottles and the device for determining whether or not the PET bottles are recyclable are described as being integrated, but this embodiment is not limited to this. For example, the determination of whether or not the bottles are recyclable may be performed by a local computer (personal computer, etc.) connected to the collection device 2, or by a server 1 on the cloud, and the collection device 2 may only open or close the lid 203 according to the determination result. In other words, the collection device 2 for collecting PET bottles and the determination device for determining whether or not the bottles are recyclable do not need to be integrated, and the two may be separate devices.

[0018] Terminal 3 is an information processing terminal held by a collector (for example, a beverage manufacturer that installed a vending machine, or an employee of a facility where the collection device 2 is installed) that collects the PET bottles collected by the collection device 2, and can be a smartphone, personal computer, tablet, etc. As will be described later, Server 1 manages the status of PET bottle collection in the collection device 2, and may notify Terminal 3 of a collection instruction to collect PET bottles from the collection device 2 if the collection box body 201 is full.

[0019] Figure 2 is a block diagram showing an example configuration of Server 1. Server 1 comprises a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit 14. The control unit 11 is a processing unit such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), and GPUs (Graphics Processing Units), and performs various information processing, control processing, etc. by reading and executing the program P1 stored in the auxiliary storage unit 14. The main memory unit 12 is an SRAM (Static Random Access Memory), D The RAM (Dynamic Random Access Memory) and flash memory are temporary storage areas that temporarily store data necessary for the control unit 11 to perform calculations. The communication unit 13 is a communication module for performing communication-related processing and for sending and receiving information with the outside world.

[0020] The auxiliary storage unit 14 is a non-volatile storage area such as a large-capacity memory or hard disk, and stores the program P1 and other data necessary for the control unit 11 to execute processing. The auxiliary storage unit 14 also stores the recovery device DB 141. The recovery device DB 141 is a database that stores information on each of the recovery devices 2 installed at multiple locations.

[0021] The auxiliary storage unit 14 may be an external storage device connected to the server 1. Furthermore, the server 1 may be a multi-computer system consisting of multiple computers, or it may be a virtual machine virtually constructed by software.

[0022] Furthermore, in this embodiment, the server 1 is not limited to the above configuration and may include, for example, an input unit for receiving operation input, a display unit for displaying images, etc. The server 1 may also include a read unit for reading portable storage media 1a such as a CD (Compact Disk)-ROM or DVD (Digital Versatile Disc)-ROM, and may read and execute the program P1 from the portable storage media 1a. Alternatively, the server 1 may read the program P1 from a semiconductor memory 1b.

[0023] Figure 3 is a block diagram showing an example configuration of the recovery device 2. The recovery device 2 consists of a control unit 21 and a main It comprises a storage unit 22, a communication unit 23, a display unit 24, an opening / closing drive unit 25, and an auxiliary storage unit 26. The control unit 21 is one or more arithmetic processing units such as CPUs and MPUs, and performs various information processing and control processing by reading and executing the program P2 stored in the auxiliary storage unit 26. The main memory unit 22 is a temporary storage area such as RAM, and temporarily stores data necessary for the control unit 21 to perform arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information with the outside. The display unit 24 is a display means such as a display or a predetermined indicator (such as an LED (Light Emitting Diode) lamp) (see Figures 7 and 8), and displays the result of the determination of whether or not it can be recycled. The opening / closing drive unit 25 is a drive mechanism that drives the opening and closing operation (e.g., sliding operation) of the lid 203, and opens the lid 203 according to the opening instruction (collection instruction) from the control unit 21.

[0024] The auxiliary storage unit 26 is a non-volatile storage area such as a hard disk or large-capacity memory, and stores the program P2 and other data necessary for the control unit 21 to execute processing. The auxiliary storage unit 26 also stores the judgment model 50. The judgment model 50 is a machine learning model that has been trained on predetermined training data, and when an image of a plastic bottle is taken as input, it outputs a judgment result determining whether or not the plastic bottle is in a recyclable state. The judgment model 50 is intended to be used as a program module that constitutes part of the artificial intelligence software.

[0025] The recovery device 2 may also include a reading unit for reading a portable storage medium 2a such as a CD-ROM, and may read and execute the program P2 from the portable storage medium 2a. Alternatively, the recovery device 2 may read the program P2 from the semiconductor memory 2b.

[0026] Figure 4 is an explanatory diagram showing an example of the record layout of the collection device DB141. The collection device DB141 includes a device ID column, an installation location column, a maximum number of bottles to be collected column, and a collection status column. The device ID column stores the device ID for identifying each collection device 2 installed at multiple locations. The installation location column, the maximum number of bottles to be collected column, and the collection status column, respectively, are associated with the device ID and store the installation location of the collection device 2, the maximum number of PET bottles that can be stored in the collection box body 201, and the number of bottles currently stored in the collection box body 201.

[0027] Figure 5 is an explanatory diagram of the process for determining whether or not a product is recyclable. Figure 5A shows a PET bottle that is recyclable, and Figures 5B to 5D show PET bottles that are not recyclable. Based on Figures 5A to 5D, the process for determining whether or not a product is recyclable based on the determination model 50 will be explained.

[0028] As described above, Server 1 performs machine learning to learn from predetermined training data and generates a judgment model 50 that takes an image of a plastic bottle as input and outputs a result determining whether or not it is recyclable. The judgment model 50 is a neural network generated by deep learning, for example, and is a CNN (Convolutional Neural Network) having multiple convolutional layers.

[0029] The judgment model 50 has an input layer that accepts an image input, an intermediate layer that extracts features from the input image, and an output layer that outputs a judgment result on whether or not it is recyclable. The input layer accepts an image of a PET bottle as input and passes the input image data to the intermediate layer. The intermediate layer has multiple convolutional layers that convolve the image data, extract features by convolving the image data, and pass them to the output layer. Based on the features extracted in the intermediate layer, the output layer outputs a judgment result indicating whether or not it is in a recyclable state.

[0030] The judgment model 50 according to this embodiment outputs a judgment result that determines whether or not recycling is possible for each of the multiple items used as judgment criteria. These multiple items are, for example, the key attached to the PET bottle. The three items are: the presence or absence of a cap, the presence or absence of a label, and the presence or absence of contents in the plastic bottle.

[0031] Figures 5B to 5D illustrate the states of a PET bottle with the cap still attached, with the label still attached, and with the contents (beverage) still inside, respectively. In Figures 5B to 5D, the cap, label, and contents are shown with thick lines. When PET bottles in the states shown in Figures 5B to 5D are collected, it is necessary to remove the cap or label in order to recycle them. Therefore, in this embodiment, the judgment model 50 is used to detect these states, and as shown in Figure 5A, PET bottles without caps, labels, and contents are collected.

[0032] Server 1 generates a judgment model 50 using training data in which ground truth data (ground truth labels) indicating whether or not a plastic bottle is recyclable are attached to images of plastic bottles used for training. The ground truth data is data that shows the correct values ​​for the judgment results of each of the three items mentioned above. In this embodiment, there are 2 × 2 × 2 = 8 possible labels (judgment results) that the judgment model 50 can output, depending on whether or not there is a cap (2 types), whether or not there is a label (2 types), and whether or not there is contents (2 types). In the training data, one of the above 8 types of labels is attached as ground truth data to the training images.

[0033] Server 1 inputs images of training plastic bottles into the judgment model 50 and obtains judgment results as output, determining whether each bottle is recyclable based on the above criteria. Server 1 compares the judgment results output from the judgment model 50 with the ground truth data and optimizes parameters such as the weights between neurons so that the two approximate each other. Server 1 learns from multiple images included in the training data and generates the judgment model 50.

[0034] In this embodiment, the determination model 50 only determines whether or not there is anything inside the PET bottle, but it may also determine what the contents of the PET bottle are. Specifically, the determination model 50 may determine whether the contents of the PET bottle are a beverage or a foreign object other than a beverage (such as a cigarette butt). In this case, for example, three correct answers could be prepared for the determination item related to the presence or absence of contents: an empty PET bottle, a bottle with beverage remaining, and a bottle with a foreign object present. This makes it possible to separately detect PET bottles with beverage remaining and PET bottles with foreign objects other than beverages.

[0035] For example, the collection device 2 obtains data for the determination model 50 from the server 1 via the network N and stores it in the auxiliary storage unit 26. The collection device 2 uses the determination model 50 to determine whether or not the PET bottle is in a recyclable state, and if it determines that it is recyclable, it opens the lid 203 and collects the PET bottle.

[0036] Figure 6 is an explanatory diagram regarding the collection and processing of PET bottles. Figure 7 is an explanatory diagram showing an example of the display of the determination result for whether or not a bottle is recyclable. Based on Figures 6 and 7, the processing contents of the collection device 2 will be explained.

[0037] The upper part of Figure 6 illustrates how the camera 204 images a PET bottle placed on the lid 203. When a PET bottle is placed on the lid 203, the collection device 2 images the PET bottle with the camera 204. The collection device 2 then inputs the image of the PET bottle into the judgment model 50 to determine whether the PET bottle is in a recyclable state. Specifically, the collection device 2 makes judgments for each item: presence or absence of a cap, presence or absence of a label, and presence or absence of contents. Furthermore, the collection device 2 makes a comprehensive judgment for all items, that is, whether the PET bottle is free of a cap, label, and contents, and is collectible for recycling. For convenience, in the following explanation, the judgment results for each of the above items will be combined to make a comprehensive judgment. The final result is called the "overall judgment result."

[0038] Furthermore, the collection device 2 may recognize objects in the image and determine whether the recognized objects are PET bottles before making a determination of recyclability based on the determination model 50. That is, before inputting the image to the determination model 50, the collection device 2 determines whether the objects in the image are PET bottles, and if it determines that they are PET bottles, it inputs the image to the determination model 50 to determine whether or not they are in a recyclable state. This prevents objects other than PET bottles (for example, empty cans) from being dumped.

[0039] Figure 7 illustrates the display screen used to show the user (the person disposing of the plastic bottle) the result of the recycling feasibility determination. The collection device 2 is equipped with a display means such as a liquid crystal display as a display unit 24 (not shown in Figures 1 and 6), and displays the screen on the display unit 24. The screen includes a plastic bottle image 71, item-specific determination results 72, and overall determination results 73. The collection device 2 displays the plastic bottle image 71 captured by the camera 204, and also displays the recycling feasibility determination results by the determination model 50 as item-specific determination results 72 and overall determination results 73.

[0040] The item-specific judgment result 72 is a display area that shows the judgment result for each item used as a judgment criterion. For example, the collection device 2 displays the judgment result for each item as a checkmark or cross, and highlights items that have been determined to be unrecyclable (cross) using color coding or other methods. In Figure 7, for convenience, the color coding is illustrated with hatching.

[0041] Furthermore, the collection device 2 may not only highlight items that it has determined to be unrecyclable, but may also show the user the corresponding location on the PET bottle. For example, the collection device 2 may recognize an object (label in Figure 7) corresponding to an item that it has determined to be unrecyclable from the PET bottle image 71, and superimpose an object such as a bounding box onto the PET bottle image 71 in the image area corresponding to that object. Thus, the method of showing the user items that it has determined to be unrecyclable is not particularly limited.

[0042] The overall judgment result 73 is a display field that shows the overall judgment result, which determines whether or not all items are in a recyclable state. For example, as shown in Figure 7, the collection device 2 displays the overall judgment result as a checkmark or a cross.

[0043] In this manner, the collection device 2 displays and presents the result of the determination of whether or not the item is recyclable to the user. As shown in Figure 7, if it is determined that the item is not recyclable, the item corresponding to the cause is displayed in the item-specific determination result 72. In this case, the user removes the object related to the item (label in Figure 7) and re-images the PET bottle to perform the determination. This allows for efficient collection of PET bottles.

[0044] By the way, in the example in Figure 5, the label covers only a portion of the bottle when viewed from the side (or front), leaving the bottom of the PET bottle exposed, so there is no problem. However, depending on the product, the label may cover almost the entire side of the PET bottle, leaving the bottom completely hidden. In this case, it is conceivable that the contents of the PET bottle may be hidden by the label and it may not be possible to determine whether there is content or not simultaneously with the presence or absence of the label.

[0045] In this case, the collection device 2 can determine whether or not a bottle is recyclable by processing it as follows. For example, the collection device 2 is equipped with a separate camera for bottom imaging on the lid 203 on which the PET bottle is placed, and acquires a bottom image of the PET bottle taken from the bottom. Then, if the collection device 2 determines that there is a label based on the judgment model 50 described above, it determines whether or not there is any contents from the bottom image. Note that the judgment based on the bottom image is performed by machine learning models such as the judgment model 50. You can use Dell, or you can use a rule-based approach (image pattern matching). By combining images of the PET bottle taken from the side and images taken from the bottom in this way, you can determine whether or not there is contents inside, regardless of the size of the label.

[0046] The above process is merely an example, and this embodiment is not limited thereto. For example, if the collection device 2 determines that there is a label on the PET bottle, it may display instructions on the screen in Figure 7 to remove the label and re-image. For example, the collection device 2 may display "×" as the result of the "label" determination in the item-specific determination result 72, and "△" (undeterminable) as the result of the "contents" determination, and display instructions to remove the label and re-image. This makes it possible to suitably determine whether or not there is content inside the PET bottle without imaging the bottom of the bottle.

[0047] Figure 8 is an explanatory diagram showing another example of displaying the judgment result. Figure 7 describes the case where the judgment result is displayed on a display screen, but the means of displaying the judgment result is not limited to this. For example, as shown in Figure 8, the collection device 2 may be equipped with a dedicated indicator (such as an LED lamp) as a display unit 24 for displaying the judgment result of whether or not it is recyclable, and the judgment result may be displayed on this indicator. The indicator is equipped with a plurality of item-specific lamps 81 that display the judgment result for each item and a total lamp 82 that displays the overall judgment result, and the display color of each lamp is changed according to the judgment result. This makes it possible to configure the means of displaying the judgment result more simply than the display screen in Figure 7.

[0048] Returning to Figure 6, let's continue the explanation. If the collection device 2 determines that the PET bottle is in a recyclable condition, it collects the PET bottle. Specifically, as shown in the lower part of Figure 6, the collection device 2 outputs an opening command to the opening / closing drive unit 25 to open the lid 203, and the PET bottle placed on the lid 203 falls into the collection box body 201.

[0049] Through the above process, the collection device 2 collects PET bottles that are in a recyclable state. At this point, it is preferable to store the collection status of the PET bottles collected in the collection box body 201, and to have a designated collector collect the PET bottles when the collection box body 201 is full.

[0050] For example, each collection device 2 counts the number of times the lid 203 is opened and closed, i.e., the number of PET bottles collected, and transmits the number of collected bottles to the server 1 via the network N. The server 1 stores the number of PET bottles collected by each collection device 2 in the collection device DB 141. The server 1 compares the current number of collected bottles with the maximum number of collected bottles and determines, for example, whether the number of collected bottles is equal to or greater than the maximum number of collected bottles. If it determines that the number of collected bottles is equal to or greater than the maximum number of collected bottles, the server 1 notifies the terminal 3 of a collection instruction indicating the collection device 2 whose number of collected bottles is equal to or greater than the maximum number of collected bottles. PET bottles can be collected efficiently.

[0051] In the above, the determination of whether the collection box body 201 is full is made based on the number of PET bottles, but the determination may also be made based on the capacity (volume) of the PET bottles. In this case, for example, the collection device 2 can recognize the capacity from the size of the PET bottles shown in the image, and then total the capacity of each PET bottle placed in the collection box body 201 and compare it with the maximum capacity. Thus, the collection device 2 only needs to be able to compare the total amount of PET bottles contained in the collection box body 201 with a predetermined upper limit.

[0052] Furthermore, although the above describes notifying terminal 3 (collector) of collection instructions via server 1, the entire process could also be completed in the collection device 2, and collection instructions could be notified directly from the collection device 2 to terminal 3.

[0053] Based on the above, according to this embodiment 1, it is possible to suitably identify PET bottles that are in a recyclable state and to support PET bottle collection.

[0054] Figure 9 is a flowchart showing the procedure for generating the judgment model 50. Based on Figure 9, the process for generating the judgment model 50 using machine learning will be explained. The control unit 11 of server 1 acquires training data to generate the judgment model 50 (step S11). The training data is data in which ground truth data indicating whether or not the PET bottle is in a recyclable state is attached to an image of a PET bottle used for training. As described above, the ground truth data is data that shows the correct values ​​of the judgment result for multiple items, such as the presence or absence of a cap, the presence or absence of a label, and the presence or absence of contents.

[0055] Based on the training data, the control unit 11 generates a judgment model 50 that determines whether or not an image of a plastic bottle is recyclable when it is input (step S12). Specifically, as described above, the control unit 11 generates a neural network such as a CNN as the judgment model 50. The control unit 11 inputs the training images of plastic bottles into the judgment model 50 and obtains the judgment results as output for each item, such as the presence or absence of a cap and the presence or absence of a label. The control unit 11 compares the judgment results output from the judgment model 50 with the ground truth data and generates the judgment model 50 by optimizing parameters such as the weights between neurons so that the two approximate each other. The control unit 11 then completes the series of processes.

[0056] Figure 10 is a flowchart showing the procedure for collecting and processing plastic bottles. Based on Figure 10, the processing performed by the collection device 2 will be explained. The control unit 21 of the collection device 2 uses the camera 204 to image the PET bottle placed on the lid 203 covering the PET bottle input opening 202 (step S31). The control unit 21 inputs the imaged image into the judgment model 50 to determine whether the PET bottle is in a recyclable state (step S32). Specifically, the control unit 21 makes a judgment for each of several items that serve as the criteria for determining whether or not the bottle is recyclable. These items include, for example, the presence or absence of a cap, the presence or absence of a label, and the presence or absence of contents. The control unit 21 may also determine whether or not the object in the image is a PET bottle before performing the judgment process in step S32.

[0057] The control unit 21 displays the result of the recycling feasibility determination on the display unit 24 (step S33). Specifically, the control unit 21 displays the determination result for each of the above items, as well as an overall determination result that determines whether or not the item is in a recyclable state for all items.

[0058] As described above, if the control unit 21 determines in step S32 that there is a label on the PET bottle, it may acquire a bottom image of the PET bottle taken from the bottom and determine whether there is any contents inside. Alternatively, if the control unit 21 determines that there is a label on the PET bottle, it may display a message in step S33 indicating that the label should be removed and the image re-taken, and then re-image the PET bottle with the label removed to determine whether there is any contents inside.

[0059] Based on the determination result in step S32, the control unit 21 determines whether or not the PET bottle is in a recyclable state (step S34). If it is determined that the bottle is not in a recyclable state (S34: NO), the control unit 21 terminates the series of processes. If it is determined that the bottle is in a recyclable state (S34: YES), the control unit 21 outputs an open command (collection command) to the open / close drive unit 25 to open the lid 203 (step S35), and terminates the series of processes.

[0060] In the above description, the collection device 2 was described as being equipped with means for collecting PET bottles (a lid 203 on which a PET bottle can be placed, and an opening / closing drive unit 25 for opening and closing the lid 203), but this embodiment is not limited to this. For example, the collection device 2 is equipped with means for collecting PET bottles (a lid 203 on which a PET bottle can be placed, and an opening / closing drive unit 25 for opening and closing the lid 203). The system may simply display the result and, if the bottle is recyclable, display a collection instruction to the user to insert (collect) the PET bottle into the PET bottle input port 202. In other words, the collection device 2 (determination device) only needs to be able to output a collection instruction for the PET bottle according to the determination result of whether or not it is recyclable, and it is not necessary for the device to operate the PET bottle collection means according to that collection instruction.

[0061] As described above, according to this embodiment 1, the determination model 50 determines from the image of the PET bottle whether or not it is in a recyclable state, and the lid 203 is opened and closed according to the determination result to collect the PET bottle in a recyclable state. If the bottle is in a non-recyclable state, the determination item that caused this is presented to the user, and the user is prompted to remove the corresponding item (cap, label, etc.). This effectively supports the collection of PET bottles.

[0062] (Variation 1) The collection device 2 according to Embodiment 1 was a collection box having only a PET bottle input port 202 as an input port for introducing waste. On the other hand, the collection device 2 may also be provided with an input port (second input port) for accessories such as PET bottle caps and labels. In this modified example, a collection device 2 that has a cap input port 206 and a label input port 207 in addition to the PET bottle input port 202 will be described.

[0063] Figure 11 is a schematic diagram showing an example of the configuration of the collection device 2 according to Modification 1. The collection box body 201 of the collection device 2 according to this modification includes a cap input port 206 and a label input port 207, in addition to the PET bottle input port 202. The cap input port 206 and the label input port 207 are openings for inserting caps and labels (items) attached to PET bottles into the collection box body 201, and are normally covered by lids 208 and 209. The lids 208 and 209 of the cap input port 206 and the label input port 207 are configured to open and close according to instructions from the control unit 21, similar to the PET bottle input port 202.

[0064] In addition to the cap slot 206 and the label slot 207, an opening for disposing of the contents (beverage) of the PET bottle may also be provided.

[0065] The collection device 2 opens and closes the PET bottle input port 202, the cap input port 206, and the label input port 207 according to the determination result of whether or not the bottle is recyclable based on the determination model 50. That is, similar to Embodiment 1, if the collection device 2 determines that the bottle is recyclable as a result of the overall determination, it opens the PET bottle input port 202 and collects the PET bottle. On the other hand, if it determines that there is a cap and / or label, the collection device 2 outputs an open command to the open / close drive unit (not shown) that opens and closes the lids 208 and 209, and opens the cap input port 206 and / or the label input port 207. This promotes the separation of caps and labels and improves user convenience.

[0066] Except for the points mentioned above, this embodiment is the same as Embodiment 1, therefore, detailed illustrations and explanations such as flowcharts are omitted in this embodiment.

[0067] (Modification 2) In Embodiment 1, a model was described in which the collection device 2 determines whether a PET bottle is recyclable from an image of the PET bottle and collects the PET bottle. Here, the images of the PET bottles captured by the collection device 2 may be used as training data for retraining to update (retrain) the judgment model 50. In this modified example, the process of updating the judgment model 50 will be described.

[0068] Figure 12 is a flowchart showing the procedure for updating the judgment model 50. Next, we will explain the processing details of the update process for the judgment model 50. The control unit 11 of server 1 acquires images of PET bottles captured by the camera 204 from each collection device 2 (step S101). For example, collection device 2 stores the captured images of the PET bottles to be judged in the auxiliary storage unit 26 and periodically transmits the captured images to server 1.

[0069] Server 1 may acquire images of all PET bottles targeted for judgment from the collection device 2, or it may acquire images of only some of the PET bottles to use for retraining. For example, Server 1 may acquire images of PET bottles that have been determined to be unrecyclable. Alternatively, Server 1 may acquire images of PET bottles whose probability value, which indicates the likelihood of the judgment result output from the judgment model 50 along with the judgment result of whether or not they are recyclable, falls within a predetermined numerical range (for example, a numerical range of around 50%). This allows for the selection of images suitable as training data for retraining, such as images of PET bottles that are unrecyclable or images of PET bottles for which the judgment result is uncertain.

[0070] The control unit 11 receives a setting input from the system administrator to set correct data indicating whether or not the acquired PET bottle is in a recyclable state for the image of the PET bottle (step S102). The correct data set in step S102 is data that represents the correct value for recyclability for each item such as the cap and label, similar to Embodiment 1.

[0071] In step S102, the control unit 11 updates the judgment model 50 using the images of the PET bottles to which the correct answer data was set (assigned) as training data for retraining (step S103). That is, the control unit 11 inputs the images of the PET bottles for retraining into the judgment model 50 and optimizes parameters such as the weights between neurons so that the judgment result output from the judgment model 50 approximates the correct answer data. The control unit 11 transmits the updated judgment model 50 data to the retrieval device 2 (step S104), and the series of processes ends.

[0072] Although the flowchart in Figure 12 explains the process assuming that Server 1 is the entity responsible for updating the judgment model 50, the retrieval device 2 may also perform the update of the judgment model 50 locally.

[0073] Based on the above, according to this modified example 2, the judgment model 50 can be updated through the operation of this system, thereby improving the accuracy of the determination of whether or not a material is recyclable.

[0074] (Embodiment 2) Embodiment 1 described a system for end users that primarily utilizes PET bottles. This embodiment describes a system for businesses that sort large quantities of PET bottles discarded by end users.

[0075] Figure 13 is a schematic diagram showing an example configuration of a PET bottle collection system according to Embodiment 2. The PET bottle collection system according to this embodiment has a determination device 4, a belt conveyor 5, and a camera 6 instead of a collection device 2. For example, the determination device 4 is connected to a server 1 via a network N and acquires data of the determination model 50 from the server 1 and stores it in local storage.

[0076] The determination device 4 is a device that determines whether a PET bottle is recyclable using the determination model 50, and is, for example, a personal computer. The server 1 and the determination device 4 may be configured as a single computer. Similar to the collection device 2 in Embodiment 1, the determination device 4 is... It may be configured as an integrated device with the belt conveyor 5, camera 6, etc.

[0077] Belt conveyor 5 is a belt conveyor for sorting waste (PET bottles) and is a device that transports PET bottles in one direction. Belt conveyor 5 transports PET bottles (not shown in Figure 13) placed on the conveyor belt in one direction, and an operator sorts the PET bottles on the conveyor belt into recyclable and non-recyclable bottles. In addition, the system in Figure 13 may be equipped with a means (collection device) for collecting PET bottles, such as a robotic arm, to allow for mechanical sorting of PET bottles without the need for an operator.

[0078] Camera 6 is an imaging device that captures images of PET bottles being transported on the conveyor belt 5. For example, as shown in Figure 13, camera 6 is positioned directly above the conveyor belt 5 and captures images of numerous PET bottles being transported on the conveyor belt 5.

[0079] Figure 14 is a block diagram showing an example configuration of the determination device 4. The determination device 4 comprises a control unit 41, a main memory unit 42, a communication unit 43, a display unit 44, and an auxiliary memory unit 45. The control unit 41 is one or more arithmetic processing units such as CPUs and MPUs, and performs various information processing and control processing by reading and executing the program P4 stored in the auxiliary storage unit 45. The main memory unit 42 is a temporary storage area such as RAM, and temporarily stores data necessary for the control unit 41 to perform arithmetic processing. The communication unit 43 is a communication module for performing communication-related processing, and sends and receives information with the outside. The display unit 44 is a display screen such as a liquid crystal display, and displays images provided by the control unit 41. The auxiliary storage unit 45 is a non-volatile storage area such as a hard disk or large-capacity memory, and stores the program P4 and other data necessary for the control unit 41 to perform processing. The auxiliary storage unit 45 also stores the judgment model 50.

[0080] The determination device 4 may also include a reading unit for reading a portable storage medium 4a such as a CD-ROM, and may read and execute the program P4 from the portable storage medium 4a. Alternatively, the determination device 4 may read the program P4 from the semiconductor memory 4b.

[0081] Figure 15 is an explanatory diagram relating to the process for determining whether or not a material can be recycled according to Embodiment 2. Based on Figure 15, the outline of this embodiment will be explained. As described above, camera 6 captures images of numerous PET bottles being transported on the conveyor belt 5. In this embodiment, multiple PET bottles are included in a single captured image (frame).

[0082] The determination device 4 inputs the image captured by the camera 6 into the determination model 50 to determine whether or not the PET bottle is in a recyclable state. Here, the determination device 4 determines whether or not each of the multiple PET bottles included in the captured image is in a recyclable state.

[0083] For example, the determination device 4 recognizes individual PET bottles from an image before inputting the image to the determination model 50. The determination device 4 extracts the image region corresponding to each PET bottle from the image and inputs each image region individually to the determination model 50. In this way, the determination device 4 determines whether or not each PET bottle is in a recyclable state. The determination device 4 may also perform determinations for each item, such as the presence or absence of a cap or label, as in Embodiment 1, or it may only perform an overall determination of whether or not it is in a recyclable state.

[0084] In the above description, the image region of each PET bottle is extracted from the image captured by camera 6 to perform the determination, but this embodiment is not limited to this. For example, the determination device 4 distinguishes between PET bottles that are recyclable and PET bottles that are not recyclable. A judgment model 50 trained on images containing both plastic and PET bottles as training data may be prepared, enabling the direct determination of whether each PET bottle is recyclable from an image showing multiple PET bottles. Thus, the judgment device 4 only needs to be able to determine whether each PET bottle is recyclable from an image showing multiple PET bottles, and the specific processing method is not particularly limited.

[0085] The determination device 4 displays the determination result for each PET bottle, indicating whether or not it is recyclable. For example, as shown in Figure 15, the determination device 4 displays the image captured by the camera 6 and overlays a display label (bounding box, etc.) indicating the image area of ​​each PET bottle. The determination device 4 then indicates whether or not each PET bottle is recyclable by, for example, changing the display color of the display label. In Figure 15, for illustrative purposes, the bounding box of a PET bottle that is not recyclable is shown with a thick line.

[0086] The determination device 4 outputs a collection instruction for the PET bottles according to the above determination result regarding whether or not they are recyclable. For example, the determination device 4 outputs a collection instruction for the PET bottles that are determined to be unrecyclable from among the multiple PET bottles included in the image. Note that it may also output a collection instruction for recyclable PET bottles instead of unrecyclable ones.

[0087] The determination device 4 displays an alert indicating, for example, that a PET bottle in an unrecyclable state has been detected. Alternatively, the determination device 4 may notify a collection instruction to a portable terminal (such as a smartphone) carried by the worker performing the PET bottle sorting work. Or, if the system is equipped with a collection means (collection device) such as a robotic arm, the determination device 4 may output a collection instruction to the collection means to collect the relevant PET bottles.

[0088] Figure 16 is a flowchart showing the procedure for collecting PET bottles according to Embodiment 2. Based on Figure 16, the processing content performed by the determination device 4 will be explained. The control unit 41 of the determination device 4 acquires images of multiple PET bottles being transported on the belt conveyor 5 from the camera 6 (step S201). The control unit 41 inputs the acquired images into the determination model 50 and determines whether each of the multiple PET bottles included in the image is in a recyclable state or not (step S202). The control unit 41 displays the determination result for each PET bottle, determining whether it is in a recyclable state or not (step S203).

[0089] The control unit 41 determines whether or not there are any PET bottles in an unrecyclable state among the multiple PET bottles to be judged (step S204). If it is determined that there are PET bottles in an unrecyclable state (S204: YES), the control unit 41 outputs a collection instruction for the PET bottles in question (step S205). For example, the control unit 41 may display an alert on the display unit 44 of the device, directly notify the worker performing the sorting work, or control the operation of a collection means such as a robotic arm.

[0090] After the process in step S205 is executed, or if the result in step S204 is NO, the control unit 41 returns the process to step S201. As a result, the control unit 41 continuously determines whether the PET bottles being transported on the belt conveyor 5 are recyclable and outputs a collection instruction.

[0091] Based on the above, according to this second embodiment, it is possible to determine whether or not each individual PET bottle is in a recyclable state from an image of multiple PET bottles and output a collection instruction. This makes it possible to handle the sorting of large quantities of PET bottles.

[0092] (Embodiment 3) Figure 17 is a functional block diagram showing the operation of the recovery device 2 or determination device 4 in the configuration described above. The control unit 21 or control unit 41 executes program P2 or P4, causing the recovery device 2 or determination device 4 to operate as follows. The acquisition unit 171 acquires an image of the PET bottle. The determination unit 172 inputs the acquired image into a model that has been trained to determine whether or not the PET bottle is in a recyclable state when the image is input, and determines whether or not the PET bottle is in a recyclable state. The output unit 173 outputs a collection instruction for the PET bottle according to the determination result.

[0093] This third embodiment is as described above, and is otherwise the same as embodiments 1 and 2. Therefore, the same reference numerals are used for corresponding parts, and their detailed descriptions are omitted.

[0094] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims, not in the sense described above, and all modifications within the sense and scope equivalent to the claims are intended. [Explanation of symbols]

[0095] 1 server 11 Control Unit 12 Main memory 13 Communications Department 14 Auxiliary storage P1 Program 2. Recovery device (determination device) 21 Control Unit 22 Main memory 23 Communications Department 24 Display 25 Opening / closing drive unit 26 Auxiliary storage P2 Program 50 Judgment Models 201 Collection box body 202 Plastic bottle slot 203, 208, 209 lid 204 Camera 205 Background board 206 Cap insertion slot (second insertion slot) 207 Label input slot (second input slot) 3 terminals 4 Judgment device 41 Control Unit 42 Main memory 43 Communications Department 44 Display section 45 Auxiliary storage P2 Program 5 Belt conveyor 6 cameras

Claims

1. An acquisition unit that acquires an image of a plastic bottle, A model that has been trained to determine whether the PET bottle is recyclable based on multiple items, including whether the contents of the PET bottle are a beverage or foreign matter other than a beverage, when the aforementioned image is input, is provided with a determination unit that inputs the acquired image and determines whether the PET bottle is recyclable based on the multiple items, An output unit that outputs a command to collect the PET bottle according to the judgment result, A display unit that displays a determination result for each of the aforementioned multiple items indicating whether or not the PET bottle is in a recyclable state, and an overall determination result indicating whether or not the PET bottle is in a recyclable state based on the determination results for each of the aforementioned items. A determination device equipped with the following features.

2. The aforementioned items further include whether or not the PET bottle has a cap and whether or not it has a label. The determination device according to claim 1.

3. If it is determined that the PET bottle has a label, the system includes a second acquisition unit that acquires a bottom image of the PET bottle taken from the bottom surface. The determination unit determines, based on the bottom image, whether the contents of the PET bottle are a beverage or a foreign object other than a beverage. The determination device according to claim 2.

4. If the display unit determines that the PET bottle has a label, it will display instructions that the label should be removed. The acquisition unit acquires an image of the PET bottle taken again, The determination unit inputs the re-captured image into the model to determine whether the contents of the PET bottle are a beverage or a foreign object other than a beverage. The determination device according to claim 2.

5. The acquisition unit acquires the images of the multiple PET bottles, The determination unit inputs the acquired image into the model and determines whether each PET bottle is in a recyclable state. A determination device according to any one of claims 1 to 4.

6. The output unit outputs a collection instruction for the PET bottles that are determined to be unrecyclable from among the multiple PET bottles included in the image. The determination device according to claim 5.

7. The system includes a second determination unit that determines whether or not the object in the aforementioned image is a plastic bottle. If the determination unit determines that the object is a plastic bottle, it inputs the image into the model to determine whether or not the plastic bottle is recyclable. A determination device according to any one of claims 1 to 6.

8. A receiving unit that accepts setting input to set correct data indicating whether or not the PET bottle is in a recyclable state for the image to be judged, An update unit updates the model based on the aforementioned image and correct data. A determination device according to any one of claims 1 to 7, comprising:

9. We acquire an image of a plastic bottle, When the aforementioned image is input to a model that has been trained to determine whether the PET bottle is recyclable for each of several items, including whether the contents of the PET bottle are a beverage or foreign matter other than a beverage, the acquired image is input and the model determines whether the PET bottle is recyclable for each of the several items, Depending on the judgment result, an instruction to collect the PET bottle will be output. The display shows a determination result for each of the aforementioned items indicating whether the PET bottle is in a recyclable state, and an overall determination result indicating whether the PET bottle is in a recyclable state based on the determination results for each item. A method for determining whether a computer should execute a process.

10. We acquire an image of a plastic bottle, When the aforementioned image is input to a model that has been trained to determine whether the PET bottle is recyclable for each of several items, including whether the contents of the PET bottle are a beverage or foreign matter other than a beverage, the acquired image is input and the model determines whether the PET bottle is recyclable for each of the several items, Depending on the judgment result, an instruction to collect the PET bottle will be output. The display shows a determination result for each of the aforementioned items indicating whether the PET bottle is in a recyclable state, and an overall determination result indicating whether the PET bottle is in a recyclable state based on the determination results for each item. A program that instructs a computer to perform a process.

11. A collection box body having an opening for inserting plastic bottles, A lid that covers the aforementioned opening, on which a PET bottle can be placed, and configured to move the placed PET bottle to the main body of the collection box in conjunction with the opening operation, The opening and closing drive unit for opening and closing the aforementioned lid, An imaging unit for imaging the PET bottle placed on the lid, A model that has been trained to determine whether the PET bottle is recyclable based on multiple items, including whether the contents of the PET bottle are a beverage or foreign matter other than a beverage, when an image of the PET bottle is taken as input, is provided with a determination unit that takes the captured image as input and determines whether the PET bottle is recyclable based on the multiple items, The system includes a display unit that displays a determination result for each of the aforementioned items indicating whether or not the PET bottle is in a recyclable state, and an overall determination result indicating whether or not the PET bottle is in a recyclable state based on the determination results for each of the aforementioned items. If the PET bottle is determined to be in a recyclable state, the opening / closing drive unit opens the lid. Recovery device.

Images