Determination device, determination method, program, and recovery device

Abstract

To provide a determination device or the like capable of suitably supporting collection of a PET bottle.SOLUTION: A determination device 2 includes: an acquisition part for acquiring an image having the imaged PET bottle: and a determination part which determines whether or not the PET bottle is in a recyclable state by inputting the acquired image into a learned model so as to be determined whether or not the PET bottle is in the recyclable state when the image is inputted; and an output part which outputs a recovery instruction of the PET bottle according to the determination result.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a determination device, a determination method, a program, and a recovery device. [Background technology]

[0002] PET bottles are widely collected for recycling, but in order to recycle them, it is desirable to collect them in a certain state that makes them recyclable, such as by removing the caps from the bottles and the labels with the product name printed on them. However, bottles are often thrown into collection bins with the caps still attached or the labels still intact.

[0003] In light of the above, methods have been proposed for collecting PET bottles in a recyclable state. For example, Patent Document 1 discloses a PET bottle / cap sorting device that opens the cap insertion opening of a PET bottle collection box when a PET bottle cap is placed in the cap insertion opening. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6130545 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the invention of Patent Document 1 only separates the caps of PET bottles, and is unable to distinguish the contents of the PET bottles, such as the labels attached to the bottles or any leftover drinks.

[0006] In one aspect, an object is to provide a determination device or the like that can suitably support the collection of PET bottles. [Means for solving the problem]

[0007] A determination device according to one aspect includes an acquisition unit that acquires an image of a PET bottle, a determination unit that inputs the acquired image into a model that has been trained to determine whether the PET bottle is in a recyclable state when the image is input, and determines whether the PET bottle is in a recyclable state, and an output unit that outputs instructions to collect the PET bottle depending on the determination result. [Effects of the Invention]

[0008] In one aspect, it is possible to favorably support the collection of PET bottles. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing an example of the configuration of a PET bottle collection system. [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of a server. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of a recovery device. [Figure 4] FIG. 10 is an explanatory diagram showing an example of a record layout of a collection device DB. [Figure 5] FIG. 10 is an explanatory diagram regarding a process for determining whether or not a product is recyclable. [Figure 6] FIG. 1 is an explanatory diagram regarding a recycling process for PET bottles. [Figure 7] FIG. 10 is an explanatory diagram showing an example of a display of a determination result of recyclability. [Figure 8] FIG. 10 is an explanatory diagram showing another example of displaying the determination result. [Figure 9] 10 is a flowchart showing a procedure for generating a determination model. [Figure 10] 10 is a flowchart showing the procedure for collecting PET bottles. [Figure 11] 10 is a schematic diagram showing a configuration example of a recovery device 2 according to a first modified example. FIG. [Figure 12] 10 is a flowchart illustrating a procedure for updating a determination model. [Figure 13] FIG. 10 is a schematic diagram showing a configuration example of a PET bottle collection system according to a second embodiment. [Figure 14] FIG. 2 is a block diagram showing an example of the configuration of a determination device. [Figure 15] FIG. 10 is an explanatory diagram regarding the process of determining whether or not a product is recyclable according to the second embodiment. [Figure 16] 10 is a flowchart showing the procedure of a PET bottle collection process according to the second embodiment. [Figure 17] FIG. 2 is a functional block diagram showing the operation of the recovery device of the above embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] The present invention will be described in detail below with reference to the drawings showing embodiments thereof. (Embodiment 1) FIG. 1 is an explanatory diagram showing an example of the configuration of a PET bottle collection system. In this embodiment, a PET bottle collection system that determines whether a PET bottle is in a recyclable state based on an image of the PET bottle will be described. The PET bottle collection system includes a server 1, a collection device (determination device) 2, and a terminal 3. Each device is communicatively connected to a network N such as the Internet.

[0011] The server 1 is a server computer capable of various information processing and sending and receiving information. Note that the computer corresponding to the server 1 is not limited to a server computer, but may be a personal computer or the like. In this embodiment, the server 1 functions as a generation device that performs machine learning to learn predetermined training data and, when an image of a PET bottle is input, generates a determination model 50 that determines whether the PET bottle is in a recyclable state. The server 1 transmits data of the generated determination model 50 to the collection device 2, which installs the data.

[0012] The collection device 2 is a collection box for collecting PET bottles, and is a device that determines whether or not a PET bottle is recyclable using the above-mentioned determination model 50, and collects the PET bottle according to the determination result. For example, the collection device 2 is installed next to a beverage vending machine or in a predetermined facility (such as a retail store that sells beverages).

[0013] Although only one recovery device 2 is shown in FIG. 1, the present embodiment will be described assuming that recovery devices 2 are installed at multiple locations.

[0014] For example, collection device 2 includes collection box main body 201, PET bottle insertion opening 202, lid 203, camera 204, and background board 205. Collection box main body 201 is a housing into which PET bottles are inserted, and stores the PET bottles inserted through PET bottle insertion opening 202. PET bottle insertion opening 202 is an opening provided in the top plate of collection box main body 201, and PET bottles are inserted through it. Lid 203 is a lid that covers PET bottle insertion opening 202, and is a plate-like member on which PET bottles can be placed. As will be described later, lid 203 is configured to be openable and closable, and is opened and closed depending on the result of a determination as to whether the PET bottle is recyclable.

[0015] Camera 204 is an imaging device that captures an image of a PET bottle placed on lid 203. In this embodiment, the description will be given assuming that collection device 2 (collection box) and camera 204 are integrated, but camera 204 may be installed in a location physically separated from collection device 2. Background board 205 is a member provided so that the background of the captured image of the PET bottle remains constant, and is provided in a manner that protrudes upward from one side of the top board of collection box main body 201 located on the opposite side from camera 204 when viewed from PET bottle insertion opening 202 and lid 203.

[0016] As will be described later, when a PET bottle is placed on the lid 203, which is normally closed, the collection device 2 captures an image of the PET bottle with the camera 204. The collection device 2 then inputs the captured image into the determination model 50 and determines whether the PET bottle is in a recyclable state. If it is determined that the PET bottle is in a recyclable state, the collection device 2 opens the lid 203 and drops the PET bottle into the collection box body 201.

[0017] In this embodiment, the collection device 2 (collection box) that collects PET bottles and the device that determines whether the PET bottles are recyclable are described as being integrated, but the embodiment is not limited to this. For example, the determination of recyclability may be made by a local computer (personal computer, etc.) connected to the collection device 2, or by a server 1 on the cloud, and the collection device 2 may simply open and close the lid 203 depending on the determination result. In other words, the collection device 2 that collects PET bottles and the determination device that determines whether the bottles are recyclable do not need to be integrated, and the two may be separate devices.

[0018] Terminal 3 is an information processing terminal carried by a collector (for example, a beverage manufacturer that installed a vending machine, or an employee of a facility where collection device 2 is installed) who collects the PET bottles collected by collection device 2, and is, for example, a smartphone, personal computer, or tablet terminal. As will be described later, server 1 manages the collection status of PET bottles in collection device 2, and may notify terminal 3 of a collection instruction to collect PET bottles from collection device 2 when collection box main body 201 is full.

[0019] 2 is a block diagram showing an example of the configuration of the server 1. The server 1 includes a control unit 11, a main memory unit 12, a communication unit 13, and an auxiliary memory unit . The control unit 11 is an arithmetic processing device such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), etc., and performs various information processing, control processing, etc. by reading and executing a program P1 stored in the auxiliary storage unit 14. The main storage unit 12 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc., and temporarily stores data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside.

[0020] The auxiliary storage unit 14 is a non-volatile storage area such as a large-capacity memory or a hard disk, and stores the program P1 and other data required for the control unit 11 to execute processing. The auxiliary storage unit 14 also stores a collection device DB 141. The collection device DB 141 is a database that stores information about each collection device 2 installed at multiple locations.

[0021] The auxiliary storage unit 14 may be an external storage device connected to the server 1. The server 1 may be a multi-computer consisting of multiple computers, or may be a virtual machine virtually constructed by software.

[0022] In this embodiment, the server 1 is not limited to the above configuration, and may include, for example, an input unit for accepting operation input, a display unit for displaying images, etc. The server 1 may also include a reading unit for reading a portable storage medium 1a such as a CD (Compact Disk)-ROM or a DVD (Digital Versatile Disc)-ROM, and may read and execute the program P1 from the portable storage medium 1a. Alternatively, the server 1 may read the program P1 from a semiconductor memory 1b.

[0023] 3 is a block diagram showing an example of the configuration of the collection device 2. The collection device 2 includes a control unit 21, a main memory unit 22, a communication unit 23, a display unit 24, an opening / closing drive unit 25, and an auxiliary memory unit . The control unit 21 is an arithmetic processing unit such as one or more CPUs, MPUs, etc., and performs various information processing, control processing, etc. by reading and executing a program P2 stored in the auxiliary storage unit 26. The main storage unit 22 is a temporary storage area such as RAM, and temporarily stores data necessary for the control unit 21 to execute arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside. The display unit 24 is a display means (see Figures 7 and 8) such as a display or a predetermined indicator (an indicator such as an LED (Light Emitting Diode) lamp), and displays the determination result of whether or not the product is recyclable. The opening / closing drive unit 25 is a drive mechanism that drives the opening and closing operation (e.g., sliding operation) of the lid 203, and opens the lid 203 in accordance with an opening instruction (recycling instruction) from the control unit 21.

[0024] The auxiliary storage unit 26 is a non-volatile storage area such as a hard disk or large-capacity memory, and stores the program P2 and other data necessary for the control unit 21 to execute processing. The auxiliary storage unit 26 also stores a determination model 50. The determination model 50 is a machine learning model that has learned predetermined training data, and when an image of a PET bottle is input, the determination model 50 outputs a determination result as to whether the PET bottle is in a recyclable state. The determination model 50 is expected to be used as a program module that constitutes part of artificial intelligence software.

[0025] The collection device 2 may be provided with a reading unit for reading a portable storage medium 2a such as a CD-ROM, and may read and execute the program P2 from the portable storage medium 2a. Alternatively, the collection device 2 may read the program P2 from the semiconductor memory 2b.

[0026] 4 is an explanatory diagram showing an example of a record layout of the collection device DB 141. The collection device DB 141 includes a device ID column, an installation location column, a maximum collection number column, and a collection status column. The device ID column stores a device ID for identifying each collection device 2 installed at multiple locations. The installation location column, the maximum collection number column, and the collection status column each store, in association with the device ID, the installation location of the collection device 2, the maximum number of PET bottles that can be collected in the collection box main body 201, and the number of collected bottles currently stored in the collection box main body 201.

[0027] Fig. 5 is an explanatory diagram regarding the process of determining whether or not a plastic bottle is recyclable. Fig. 5A illustrates a plastic bottle in a recyclable state, and Figs. 5B to 5D illustrate plastic bottles in an unrecyclable state. The process of determining whether or not a plastic bottle is recyclable based on determination model 50 will be described with reference to Figs. 5A to 5D.

[0028] As described above, the server 1 performs machine learning to learn predetermined training data, and generates a determination model 50 that receives an image of a PET bottle as an input and outputs a determination result as to whether the bottle is recyclable. The determination model 50 is, for example, a neural network generated by deep learning, such as a CNN (Convolution Neural Network) having multiple convolution layers.

[0029] The determination model 50 has an input layer that accepts image input, a middle layer that extracts features of the input image, and an output layer that outputs a determination result of whether the bottle is recyclable. The input layer accepts input of an image of a PET bottle and passes the input image data to the middle layer. The middle layer has multiple convolution layers that convolve the image data, convolves the image data to extract features, and passes them to the output layer. The output layer outputs a determination result indicating whether the bottle is recyclable based on the features extracted in the middle layer.

[0030] The determination model 50 according to the present embodiment outputs a determination result of whether a plastic bottle is recyclable or not based on a plurality of criteria, such as whether the plastic bottle has a cap, a label, and whether the plastic bottle has contents.

[0031] 5B to 5D respectively show a state in which the cap is still attached to the PET bottle, a state in which the label is still attached, and a state in which the contents (drink) remain in the PET bottle. Note that in FIGS. 5B to 5D, the parts corresponding to the cap, label, and contents are shown in bold lines. When a PET bottle in the state shown in FIGS. 5B to 5D is collected, the cap and label must be removed in order to be recycled. Therefore, in this embodiment, a determination model 50 is used to detect these states, and PET bottles without caps, labels, or contents are collected, as shown in FIG. 5A.

[0032] The server 1 generates a determination model 50 using training data in which correct answer data (correct answer labels) indicating whether or not a training PET bottle is in a recyclable state are assigned to captured images of the training PET bottle. The correct answer data is data indicating the correct answer values of the determination results for each of the above three items. In this embodiment, 2 x 2 x 2 = 8 labels are prepared as labels (determination results) that the determination model 50 can output, depending on whether or not there is a cap (2 ways), whether or not there is a label (2 ways), and whether or not there is content (2 ways). In the training data, one of the above 8 labels is assigned as correct answer data to the training image.

[0033] The server 1 inputs images of training PET bottles into the determination model 50 and obtains as output the determination results of whether the bottles are recyclable for each of the above items. The server 1 compares the determination results output from the determination model 50 with the correct answer data and optimizes parameters such as the weights between neurons so that the two results are similar. The server 1 learns from multiple images included in the training data and generates the determination model 50.

[0034] Although the determination model 50 according to this embodiment only determines whether or not there is anything in the PET bottle, it may also be configured to determine what the contents of the PET bottle are. Specifically, the determination model 50 may determine whether the contents of the PET bottle are a beverage or a foreign object other than a beverage (such as a cigarette butt). In this case, for example, three correct values may be prepared as correct values for the determination item relating to the presence or absence of contents: when the PET bottle is empty, when there is beverage remaining, and when there is a foreign object present. This makes it possible to separately detect a PET bottle with beverage remaining and a PET bottle with a foreign object other than a beverage present.

[0035] For example, the collection device 2 acquires data of the determination model 50 from the server 1 via the network N and stores it in the auxiliary storage unit 26. The collection device 2 uses the determination model 50 to determine whether the PET bottle is in a recyclable state, and if it determines that the PET bottle is in a recyclable state, opens the lid 203 and collects the PET bottle.

[0036] Fig. 6 is an explanatory diagram regarding the collection process of PET bottles. Fig. 7 is an explanatory diagram showing an example of the display of the determination result of recyclability. The processing contents of the collection device 2 will be described with reference to Figs. 6 and 7.

[0037] The upper part of FIG. 6 illustrates how camera 204 captures an image of a PET bottle placed on lid 203. When a PET bottle is placed on lid 203, collection device 2 captures an image of the PET bottle using camera 204. Then, collection device 2 inputs the image of the PET bottle into determination model 50 to determine whether the PET bottle is in a recyclable state. Specifically, collection device 2 makes a determination for each item: whether or not there is a cap, whether or not there is a label, and whether or not there is a content. Furthermore, collection device 2 makes a comprehensive determination as to whether or not all items are in a recyclable state, i.e., whether or not the PET bottle has no cap, label, or content and can be collected for recycling. For convenience in the following explanation, the determination result obtained by combining the determination results for each of the above items will be referred to as the "total determination result."

[0038] The collection device 2 may recognize an object in an image and determine whether the recognized object is a PET bottle before determining whether the object is recyclable based on the determination model 50. That is, the collection device 2 determines whether the object in the image is a PET bottle before inputting the image into the determination model 50, and if it determines that the object is a PET bottle, inputs the image into the determination model 50 to determine whether the object is recyclable. This makes it possible to prevent objects other than PET bottles (e.g., empty cans) from being dumped.

[0039] 7 shows a display screen when presenting the recyclability determination result to a user (a person throwing away a PET bottle). The collection device 2 is equipped with a display means such as an LCD display as the display unit 24 (not shown in FIGS. 1 and 6), and displays the screen on the display unit 24. The screen includes a PET bottle image 71, item-specific determination results 72, and an overall determination result 73. The collection device 2 displays the PET bottle image 71 captured by the camera 204, and also displays the recyclability determination results based on the determination model 50 as item-specific determination results 72 and overall determination result 73.

[0040] The item-by-item judgment result 72 is a display field that displays the judgment result for each item used as the judgment criterion. For example, the recovery device 2 displays the judgment result for each item with a check mark or an error, and highlights items that are judged to be unrecyclable (with an error mark) by color coding or other methods. For convenience, the color coding is shown by hatching in Figure 7.

[0041] The collection device 2 may not only highlight the items determined to be in an unrecyclable state, but may also present the user with the parts of the PET bottle corresponding to those items. For example, the collection device 2 may recognize an object (a label in FIG. 7) corresponding to an item determined to be in an unrecyclable state from the PET bottle image 71, and superimpose an object such as a bounding box on the image area corresponding to that object on the PET bottle image 71. In this way, the method of presenting the items determined to be in an unrecyclable state to the user is not particularly limited.

[0042] The overall assessment result 73 is a display field that displays the overall assessment result that determines whether or not the product is recyclable in all items. For example, as shown in Fig. 7, the recovery device 2 displays the overall assessment result with a check or an error.

[0043] In this way, the collection device 2 displays the recyclability determination result and presents it to the user. As shown in Fig. 7, if it is determined that the bottle is not recyclable, the items corresponding to the cause are displayed in the item-specific determination result 72. In this case, the user removes the item related to the item (the label in Fig. 7) and re-images the PET bottle to make a determination. This allows the collection of PET bottles to be carried out in an optimal manner.

[0044] In the example of Figure 5, the label covers only a portion of the product when viewed from the side (front), leaving the bottom of the bottle exposed, so this is not a problem, but depending on the product, the label may cover most of the side of the bottle, leaving the bottom unexposed. In this case, the inside of the bottle may be hidden by the label, making it impossible to determine whether the contents are present and whether the label is present at the same time.

[0045] In this case, the collection device 2 can determine whether or not a PET bottle is recyclable by performing the following process. For example, the collection device 2 is provided with a separate camera for capturing images of the bottom of the PET bottle on the lid 203 on which the PET bottle is placed, and acquires an image of the bottom of the PET bottle captured from the bottom. If the collection device 2 determines that a label is present as a result of the determination using the determination model 50, the collection device 2 determines whether or not there is a content from the bottom image. Note that the determination based on the bottom image may be performed using a machine learning model such as the determination model 50, or may be rule-based (image pattern matching). In this way, by combining an image of the PET bottle captured from the side and an image of the bottom, it is possible to determine whether or not there is a content, regardless of the size of the label.

[0046] Note that the above process is an example, and the present embodiment is not limited to this. For example, if the collection device 2 determines that a label exists on a PET bottle, it may display a guide on the screen of FIG. 7 to remove the label and re-image the bottle. For example, the collection device 2 may display "X" as the determination result for "label" in the item-specific determination result 72, and "△" (impossible to determine) as the determination result for "contents," and display a guide message to remove the label and re-image the bottle. This makes it possible to appropriately determine whether or not the PET bottle has contents without taking an image of the bottom of the bottle.

[0047] FIG. 8 is an explanatory diagram showing another example of the display of the judgment results. While FIG. 7 describes the case where the judgment results are displayed on a display screen, the means for displaying the judgment results is not limited to this. For example, as shown in FIG. 8, the recovery device 2 may be provided with a dedicated display (an indicator such as an LED lamp) as the display unit 24 for displaying the judgment results of whether or not the product is recyclable, and the judgment results may be displayed on the display. The display may include a plurality of item-specific lamps 81 that display the judgment results for each item, and an overall lamp 82 that displays the overall judgment result, and the display color of each lamp changes depending on the judgment result. This allows for a simpler means for displaying the judgment results than the display screen of FIG. 7.

[0048] Returning to Figure 6, the explanation will continue. If it is determined that the PET bottle is in a recyclable state, the collection device 2 collects the PET bottle. Specifically, as shown in the lower part of Figure 6, the collection device 2 outputs an opening instruction to open the lid 203 to the opening / closing drive unit 25, thereby opening the lid 203, and drops the PET bottle placed on the lid 203 into the collection box body 201.

[0049] Through the above process, the collection device 2 collects PET bottles that are in a recyclable state. Here, it is preferable to store the collection status of the collected PET bottles in the collection box main body 201, and have a designated collector collect the PET bottles when the collection box main body 201 becomes full.

[0050] For example, each collection device 2 counts the number of times the lid 203 is opened and closed, i.e., the number of collected PET bottles, and transmits the number of collected bottles to the server 1 via the network N. The server 1 stores the number of collected PET bottles at each collection device 2 in the collection device DB 141. The server 1 compares the current number of collected bottles with the maximum number of collected bottles, and determines, for example, whether the number of collected bottles is equal to or greater than the maximum number of collected bottles. If it determines that the number of collected bottles is equal to or greater than the maximum number of collected bottles, the server 1 notifies the terminal 3 of a collection instruction indicating the collection device 2 whose number of collected bottles has reached or exceeded the maximum number of collected bottles. PET bottles can be collected efficiently.

[0051] In the above, it is determined whether the collection box main body 201 is full or not based on the number of PET bottles, but the determination may also be made based on the capacity (volume) of the PET bottles. In this case, for example, the collection device 2 may recognize the capacity from the size of the PET bottles shown in the image, and may add up the capacity of each PET bottle placed in the collection box main body 201 and compare it with the maximum storage capacity. In this way, it is sufficient for the collection device 2 to be able to compare the total amount of PET bottles stored in the collection box main body 201 with a predetermined upper limit value.

[0052] In addition, in the above, the collection instruction is notified to the terminal 3 (collector) via the server 1, but the series of processes may be completed in the collection device 2, and the collection instruction may be notified directly from the collection device 2 to the terminal 3.

[0053] As described above, according to the first embodiment, it is possible to suitably identify PET bottles that are in a recyclable state and to support the collection of PET bottles.

[0054] 9 is a flowchart showing the procedure of the process of generating the determination model 50. The process of generating the determination model 50 by machine learning will be described with reference to FIG. The control unit 11 of the server 1 acquires training data for generating the determination model 50 (step S11). The training data is data in which correct answer data indicating whether the PET bottle is in a recyclable state is added to a captured image of the training PET bottle. As described above, the correct answer data is data indicating correct answer values of the determination results for multiple items, such as whether there is a cap, whether there is a label, whether there is content, etc.

[0055] Based on the training data, the control unit 11 generates a determination model 50 that, when an image of a plastic bottle is input, determines whether the bottle is in a recyclable state (step S12). Specifically, as described above, the control unit 11 generates a neural network such as a CNN as the determination model 50. The control unit 11 inputs an image of a training plastic bottle into the determination model 50, and obtains as output the determination results for each item, such as the presence or absence of a cap and the presence or absence of a label. The control unit 11 compares the determination results output from the determination model 50 with the correct answer data, and generates the determination model 50 by optimizing parameters such as the weights between neurons so that the two are similar. The control unit 11 ends the series of processes.

[0056] 10 is a flow chart showing the procedure of the PET bottle collection process. The process executed by the collection device 2 will be described with reference to FIG. The control unit 21 of the recovery device 2 uses the camera 204 to capture an image of a PET bottle placed on the lid 203 covering the PET bottle insertion port 202 (step S31). The control unit 21 inputs the captured image into the determination model 50 and determines whether the PET bottle is in a recyclable state (step S32). Specifically, the control unit 21 makes a determination for each of a plurality of items that are used as criteria for determining whether or not the PET bottle is recyclable. These items include, for example, whether or not there is a cap, whether or not there is a label, and whether or not there is content. Note that the control unit 21 may determine whether or not the object shown in the image is a PET bottle before performing the determination process of step S32.

[0057] The control unit 21 displays the determination result of whether or not the product is recyclable on the display unit 24 (step S33). Specifically, the control unit 21 displays the determination result for each of the above items, and also displays an overall determination result that determines whether or not the product is recyclable in all items.

[0058] As described above, if control unit 21 determines in step S32 that the PET bottle has a label, it may acquire a bottom image of the PET bottle taken from the bottom and determine whether or not there is any content. Also, if control unit 21 determines that the PET bottle has a label, it may display a guide in step S33 to remove the label and take a new image, and then re-image the PET bottle from which the label has been removed to determine whether or not there is any content.

[0059] Based on the determination result in step S32, control unit 21 determines whether the PET bottle is in a recyclable state (step S34). If it is determined that the PET bottle is not in a recyclable state (S34: NO), control unit 21 ends the series of processes. If it is determined that the PET bottle is in a recyclable state (S34: YES), control unit 21 outputs an opening instruction (recovery instruction) to open lid 203 to open / close drive unit 25, thereby opening lid 203 (step S35), and ends the series of processes.

[0060] In the above description, the collection device 2 is described as being equipped with a means for collecting PET bottles (lid 203 on which a PET bottle can be placed, and opening / closing drive unit 25 for opening and closing lid 203), but this embodiment is not limited to this. For example, the collection device 2 may simply display the determination result by the determination model 50, and if the PET bottle is in a recyclable state, display a collection instruction to the user to insert (collect) the PET bottle into the PET bottle insertion port 202. In other words, the collection device 2 (determination device) only needs to be able to output a collection instruction for the PET bottle according to the determination result of whether the bottle is recyclable, and it is not essential that the collection device for the PET bottle be configured to operate the collection instruction.

[0061] As described above, according to the first embodiment, the determination model 50 determines whether a PET bottle is in a recyclable state from an image of the PET bottle, and the lid 203 is opened or closed depending on the determination result, and the PET bottle in a recyclable state is collected. If the PET bottle is in an unrecyclable state, the determination items that caused this are presented to the user, and the user is prompted to remove the items (caps, labels, etc.) corresponding to the items. This provides optimal support for the collection of PET bottles.

[0062] (Variation 1) The recovery device 2 according to the first embodiment is a recovery box having only a PET bottle insertion port 202 as an insertion port for waste. However, an insertion port (second insertion port) for accessories such as PET bottle caps and labels may also be provided in the recovery device 2. In this modification, a recovery device 2 having a cap insertion port 206 and a label insertion port 207 in addition to the PET bottle insertion port 202 will be described.

[0063] 11 is a schematic diagram showing an example of the configuration of collection device 2 according to Modification 1. Collection box main body 201 of collection device 2 according to this modification has, in addition to PET bottle insertion port 202, cap insertion port 206 and label insertion port 207. Cap insertion port 206 and label insertion port 207 are openings for inserting caps and labels (items) attached to PET bottles into collection box main body 201, respectively, and are normally covered with lids 208 and 209. Like PET bottle insertion port 202, lids 208 and 209 of cap insertion port 206 and label insertion port 207 are configured to be able to open and close in accordance with instructions from control unit 21.

[0064] In addition to the cap insertion port 206 and the label insertion port 207, an insertion port for disposing of the contents (drinks) of the PET bottle may be provided.

[0065] The collection device 2 opens and closes the PET bottle insertion port 202, the cap insertion port 206, and the label insertion port 207 depending on the determination result of recyclability based on the determination model 50. That is, similar to the first embodiment, when the collection device 2 determines that the PET bottle is recyclable as a result of the overall determination, it opens the PET bottle insertion port 202 to collect the PET bottle. On the other hand, when it determines that a cap and / or label is present, the collection device 2 outputs an opening instruction to an opening / closing drive unit (not shown) that opens and closes the lids 208 and 209, and opens the cap insertion port 206 and / or the label insertion port 207. This encourages the separation of caps and labels and improves user convenience.

[0066] Other than the above, the present embodiment is the same as the first embodiment, and therefore detailed illustrations and explanations of flowcharts and the like will be omitted in this embodiment.

[0067] (Variation 2) In the first embodiment, the recovery device 2 determines whether a PET bottle is recyclable based on a captured image of the PET bottle and recovers the PET bottle. Here, the images of the PET bottle captured by the recovery device 2 may be used as training data for re-learning to update (re-learn) the determination model 50. In this modification, the update process of the determination model 50 will be described.

[0068] 12 is a flowchart showing the procedure of the update process for the determination model 50. The process of updating the determination model 50 will be described with reference to FIG. The control unit 11 of the server 1 acquires images of PET bottles captured by the camera 204 from each collection device 2 (step S101). For example, the collection device 2 stores the captured images of the PET bottles to be judged in the auxiliary storage unit 26 and periodically transmits the captured images to the server 1.

[0069] The server 1 may acquire images of all the PET bottles that were the subject of the judgment from the collection device 2, or may acquire images of only some of the PET bottles and use them for re-learning. For example, the server 1 may acquire images of the PET bottles that were the subject of the judgment and that were judged to be in an unrecyclable state. Furthermore, for example, the server 1 may acquire images of PET bottles whose probability value, which indicates the likelihood of the judgment result and is output from the judgment model 50 together with the recyclability judgment result, falls within a predetermined numerical range (for example, a numerical range of around 50%). This makes it possible to select images suitable as training data for re-learning, such as images of PET bottles that are in an unrecyclable state or images of PET bottles whose judgment result is uncertain.

[0070] The control unit 11 receives a setting input from the system administrator for setting correct answer data indicating whether the PET bottle is recyclable or not for the acquired image of the PET bottle (step S102). The correct answer data set in step S102 is data that indicates the correct answer value for recyclability for each item such as cap, label, etc., as in the first embodiment.

[0071] The control unit 11 updates the determination model 50 by using the image of the plastic bottle to which the correct answer data was set (assigned) in step S102 as training data for re-learning (step S103). That is, the control unit 11 inputs the image of the plastic bottle for re-learning into the determination model 50, and optimizes parameters such as the weights between neurons so that the determination result output from the determination model 50 approximates the correct answer data. The control unit 11 transmits the data of the updated determination model 50 to the collection device 2 (step S104), and ends the series of processes.

[0072] In the flowchart of FIG. 12, the server 1 is described as the entity that performs the processing to update the determination model 50, but the collection device 2 may update the determination model 50 locally.

[0073] As described above, according to the present modified example 2, the determination model 50 can be updated through the operation of the present system, and the accuracy of determining whether or not an item is recyclable can be improved.

[0074] (Embodiment 2) In the first embodiment, a system for end users that mainly uses PET bottles has been described. In the present embodiment, a system for businesses that separates a large number of PET bottles discarded by end users will be described.

[0075] 13 is a schematic diagram showing an example of the configuration of a PET bottle collection system according to embodiment 2. The PET bottle collection system according to this embodiment has a determination device 4, a belt conveyor 5, and a camera 6 instead of a collection device 2. For example, the determination device 4 is connected to a server 1 via a network N, and acquires data of a determination model 50 from the server 1 and stores it in local storage.

[0076] The determination device 4 is a device that determines whether a PET bottle is recyclable or not using the determination model 50, and is, for example, a personal computer. The server 1 and the determination device 4 may be configured as an integrated computer. As with the collection device 2 of the first embodiment, the determination device 4 may be configured as an integrated device with the belt conveyor 5, camera 6, etc.

[0077] Belt conveyor 5 is a belt conveyor for separating waste (PET bottles) and is a device that transports PET bottles in one direction. Belt conveyor 5 transports PET bottles (not shown in FIG. 13) placed on the conveyor belt in one direction, and workers separate the PET bottles on the conveyor belt into recyclable and non-recyclable ones. Note that the system in FIG. 13 may be equipped with a PET bottle recovery means (recovery device) such as a robot arm, so that PET bottles can be separated mechanically without relying on workers.

[0078] Camera 6 is an imaging device that captures images of PET bottles being transported by belt conveyor 5. For example, as shown in FIG. 13 , camera 6 is placed directly above belt conveyor 5 and captures images of the many PET bottles being transported by belt conveyor 5.

[0079] 14 is a block diagram showing an example of the configuration of the determination device 4. The determination device 4 includes a control unit 41, a main storage unit 42, a communication unit 43, a display unit 44, and an auxiliary storage unit 45. The control unit 41 is an arithmetic processing unit such as one or more CPUs, MPUs, etc., and performs various information processing, control processing, etc. by reading and executing the program P4 stored in the auxiliary storage unit 45. The main storage unit 42 is a temporary storage area such as RAM, and temporarily stores data necessary for the control unit 41 to execute arithmetic processing. The communication unit 43 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside. The display unit 44 is a display screen such as an LCD display, and displays images provided by the control unit 41. The auxiliary storage unit 45 is a non-volatile storage area such as a hard disk, large-capacity memory, etc., and stores the program P4 and other data necessary for the control unit 41 to execute processing. The auxiliary storage unit 45 also stores a determination model 50.

[0080] The determination device 4 may include a reading unit that reads a portable storage medium 4a such as a CD-ROM, and may read and execute the program P4 from the portable storage medium 4a. Alternatively, the determination device 4 may read the program P4 from the semiconductor memory 4b.

[0081] 15 is an explanatory diagram of the process for determining whether or not a product is recyclable according to embodiment 2. An outline of this embodiment will be described with reference to FIG. As described above, the camera 6 captures images of a large number of plastic bottles being transported on the belt conveyor 5. In this embodiment, it is assumed that a single captured image (frame) contains a plurality of plastic bottles.

[0082] The determination device 4 inputs the image captured by the camera 6 into the determination model 50 to determine whether the PET bottle is in a recyclable state. Here, the determination device 4 determines whether each of the multiple PET bottles included in the captured image is in a recyclable state.

[0083] For example, the determination device 4 recognizes individual PET bottles from the image before inputting the image into the determination model 50. The determination device 4 extracts an image area corresponding to each PET bottle from the image and inputs each image area individually into the determination model 50. This allows the determination device 4 to determine whether or not each PET bottle is in a recyclable state. Note that, similar to the first embodiment, the determination device 4 may make a determination for each item such as the presence or absence of a cap or a label, or may only make an overall determination as to whether or not the bottle is in a recyclable state.

[0084] In the above description, image regions of individual PET bottles are extracted from the image captured by camera 6 to make a determination, but this embodiment is not limited to this. For example, determination device 4 may prepare a determination model 50 that has been trained using captured images of a mixture of recyclable and non-recyclable PET bottles as training data, and be able to directly determine the recyclability of each PET bottle from an image that shows multiple PET bottles. In this way, the determination device 4 is only required to be able to determine the recyclability of each PET bottle from an image that shows multiple PET bottles, and the specific processing method is not particularly limited.

[0085] The determination device 4 displays the determination result of whether each PET bottle is recyclable. For example, as shown in Fig. 15, the determination device 4 displays an image captured by the camera 6 and superimposes a display label (such as a bounding box) indicating the image area of each PET bottle. The determination device 4 then indicates whether each PET bottle is recyclable by, for example, changing the display color of the display label. Note that, for convenience of illustration, the bounding boxes of non-recyclable PET bottles are shown in Fig. 15 with thick lines.

[0086] The determination device 4 outputs instructions to collect the PET bottles according to the determination result of whether the bottles are recyclable. For example, the determination device 4 outputs instructions to collect the PET bottles that are determined to be non-recyclable among the multiple PET bottles included in the image. Note that instructions to collect the recyclable PET bottles may be output instead of the non-recyclable PET bottles.

[0087] The determination device 4 may, for example, display an alert indicating that a non-recyclable PET bottle has been detected. Alternatively, the determination device 4 may issue a collection instruction to a mobile terminal (such as a smartphone) carried by a worker who separates the PET bottles. Alternatively, if the system is provided with collection means (collection device) such as a robot arm, the determination device 4 may output a collection instruction to the collection means to collect the relevant PET bottle.

[0088] 16 is a flowchart showing the procedure of the PET bottle collection process according to Embodiment 2. The process executed by the determination device 4 will be described with reference to FIG. The control unit 41 of the determination device 4 acquires images of multiple PET bottles conveyed on the belt conveyor 5 from the camera 6 (step S201). The control unit 41 inputs the acquired images into the determination model 50 and determines whether each of the multiple PET bottles included in the images is in a recyclable state (step S202). The control unit 41 displays the determination results of whether each PET bottle is in a recyclable state (step S203).

[0089] The control unit 41 determines whether or not any of the plurality of PET bottles that are the subject of the determination are in an unrecyclable state (step S204). If it is determined that any of the PET bottles are in an unrecyclable state (S204: YES), the control unit 41 outputs an instruction to collect the corresponding PET bottles (step S205). For example, the control unit 41 may display an alert on the display unit 44 of the device itself, may directly notify the worker performing the sorting work, or may control the operation of a collecting means such as a robot arm.

[0090] After executing the process of step S205, or if the result of step S204 is NO, the control unit 41 returns the process to step S201. As a result, the control unit 41 continuously determines whether or not the PET bottles conveyed by the belt conveyor 5 are recyclable, and outputs a collection instruction.

[0091] As described above, according to the second embodiment, it is possible to determine whether each PET bottle is recyclable from an image of the bottles and output a collection instruction, thereby enabling the system to handle cases where a large number of PET bottles need to be sorted.

[0092] (Embodiment 3) 17 is a functional block diagram showing the operation of the above-described recovery device 2 or determination device 4. When the control unit 21 or control unit 41 executes the program P2 or P4, the recovery device 2 or determination device 4 operates as follows. The acquisition unit 171 acquires an image of a PET bottle. The determination unit 172 inputs the acquired image into a model that has been trained to determine whether the PET bottle is in a recyclable state when the image is input, and determines whether the PET bottle is in a recyclable state. The output unit 173 outputs instructions to collect the PET bottle according to the determination result.

[0093] The third embodiment is as described above, and other aspects are the same as those of the first and second embodiments, so the corresponding parts are given the same reference numerals and detailed description thereof will be omitted.

[0094] The embodiments disclosed herein are to be considered as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0095] 1 server 11 Control section 12 Main memory 13 Communications Department 14 Auxiliary storage P1 Program 2. Recovery device (judgment device) 21 Control Unit 22 Main memory 23 Communications Department 24 Display section 25 Opening and closing drive unit 26 Auxiliary storage P2 Program 50 Decision Model 201 Collection box body 202 PET bottle slot 203, 208, 209 lid 204 Camera 205 Background board 206 Cap insertion slot (second insertion slot) 207 Label slot (second slot) 3. Terminal 4 Judgment device 41 Control Unit 42 Main memory 43 Communications Department 44 Display section 45 Auxiliary storage P2 Program 5. Conveyor belt 6. Camera

Claims

1. an acquisition unit that acquires an image of a plastic bottle; a determination unit that inputs the acquired image into a model that has been trained to determine whether the PET bottle is in a recyclable state when the image is input, and determines whether the PET bottle is in a recyclable state; an output unit that outputs a collection instruction for the PET bottle according to the determination result; Equipped with The acquisition unit acquires the image of the plastic bottle placed on a lid covering an insertion port provided on a body of the plastic bottle collection box, When the output unit determines that the PET bottle is in a recyclable state, it outputs an opening instruction to open the lid to an opening / closing drive unit that opens and closes the lid, and when it determines that the PET bottle is in an unrecyclable state, it outputs an opening instruction to open the lid that covers a second insertion opening provided in the collection box body, the second insertion opening being for inserting items attached to the PET bottle, to an opening / closing drive unit that opens and closes the lid that covers the second insertion opening. Judgment device.

2. The determination unit determines whether the plastic bottle has a cap, a label, and contents. The determination device according to claim 1 .

3. a second acquisition unit that acquires a bottom image of the plastic bottle when it is determined that the plastic bottle has a label; The determination unit determines whether or not there is content in the plastic bottle based on the bottom image. The determination device according to claim 2 .

4. a display unit that displays a message to remove the label when it is determined that the label is present on the PET bottle; the acquisition unit acquires a re-imaged image of the plastic bottle, The determination unit inputs the re-captured image into the model and determines whether or not there is content in the plastic bottle. The determination device according to claim 2 .

5. the acquisition unit acquires the images of the plurality of plastic bottles; The determination unit inputs the acquired image into the model and determines whether each PET bottle is in a recyclable state. The determination device according to any one of claims 1 to 4.

6. The output unit outputs the collection instruction for the PET bottle determined to be in an unrecyclable state among the plurality of PET bottles included in the image. The determination device according to claim 5 .

7. a second determination unit that determines whether the object shown in the image is a plastic bottle; When the determination unit determines that the object is a plastic bottle, the determination unit inputs the image into the model and determines whether the plastic bottle is recyclable. The determination device according to any one of claims 1 to 6.

8. a receiving unit that receives a setting input for setting correct answer data indicating whether the PET bottle is in a recyclable state for the image that is the subject of judgment; an update unit that updates the model based on the image and ground truth data; The determination device according to any one of claims 1 to 7, comprising:

9. An image of a plastic bottle placed on a lid covering an insertion opening provided on a body of a plastic bottle collection box is acquired; inputting the acquired image into a model that has been trained to determine whether the PET bottle is in a recyclable state when the image is input, and determining whether the PET bottle is in a recyclable state; If it is determined that the PET bottle is in a recyclable state, an opening instruction to open the lid is output to an opening / closing drive unit that opens and closes the lid, and if it is determined that the PET bottle is in an unrecyclable state, an opening instruction to open the lid that covers a second insertion opening provided in the collection box body and through which items attached to the PET bottle are inserted is output to an opening / closing drive unit that opens and closes the lid that covers the second insertion opening. A method for determining whether a process is executed by a computer.

10. An image of a plastic bottle placed on a lid covering an insertion opening provided on a body of a plastic bottle collection box is acquired; inputting the acquired image into a model that has been trained to determine whether the PET bottle is in a recyclable state when the image is input, and determining whether the PET bottle is in a recyclable state; If it is determined that the PET bottle is in a recyclable state, an opening instruction to open the lid is output to an opening / closing drive unit that opens and closes the lid, and if it is determined that the PET bottle is in an unrecyclable state, an opening instruction to open the lid that covers a second insertion opening provided in the collection box body and through which items attached to the PET bottle are inserted is output to an opening / closing drive unit that opens and closes the lid that covers the second insertion opening. A program that causes a computer to perform a process.

11. a collection box body having an insertion opening for PET bottles; A lid that covers the insertion port, on which a PET bottle can be placed, and which is configured to move the placed PET bottle to the collection box body in accordance with an opening operation; an opening / closing drive unit that opens and closes the lid; a second opening / closing drive unit that opens and closes a lid covering a second insertion port provided in the collection box body, the second insertion port being for inserting items attached to the PET bottle; an imaging unit that images the plastic bottle placed on the lid; a judgment unit that inputs a captured image of the PET bottle into a model that has been trained to judge whether the PET bottle is in a recyclable state when the captured image is input, and judges whether the PET bottle is in a recyclable state; When the plastic bottle is determined to be in a recyclable state, the opening / closing drive unit opens the lid, When the PET bottle is determined to be in an unrecyclable state, the second opening / closing drive unit opens a lid covering the second insertion opening. Recovery device.

Images