Device, system, computer program and method for outputting control signals

Abstract

According to some embodiments of the present disclosure, there is provided a device comprising processing circuitry configured to receive a polarized image of an object at least partially made of polarized material, detect a contour of the object from the polarized image, calculate a probability that the contour of the object is a stored contour, and output a control signal indicating a match based on the calculated probability being above a threshold. In some embodiments, the present disclosure enables sorting or segregating plastic objects for manufacturing, recycling, reuse, or repurposing.

Description

[Technical Field]

[0001] The present technology relates to devices, systems, computer programs and methods for outputting control signals. [Background technology]

[0002] The "Background" discussion provided herein is intended to generally provide a context for the present disclosure. The work of the currently named inventor(s) (to the extent described in the Background section) and aspects of the description that may not be admitted as prior art at the time of filing this application are not admitted, expressly or impliedly, as prior art to the present invention.

[0003] Plastic is a widely used material in modern society. Plastic products are often made from recycled plastics, which are plastics that have already been used in products.

[0004] The manufacturing process for many materials requires the use of recycled plastic, and a specific example of a material made from plastic is recycled polyester (recycled PET). Recycled PET, made from recycled polyethylene terephthalate (PET) plastic, requires 59% less energy to produce and reduces CO2 emissions by 32% compared to virgin polyester, making it an increasingly popular material for clothing; 49% of the world's clothing is made from polyester. It typically takes nine recycled PET bottles to make one T-shirt.

[0005] In this manufacturing process, recycled PET yarn manufacturers purchase large quantities of recycled PET containers (e.g., bottles) from vendors or recyclers. These recycled PET containers are manually sorted to ensure only PET bottles are supplied to the manufacturing process. This manual sorting process is very labor-intensive, and the inclusion of non-PET caps and bottoms reduces the quality of the recycled PET. Therefore, the PET containers must be manually sorted to remove any foreign material. This manual sorting process is time-consuming and carries a high risk of introducing foreign material into the remaining steps of the manufacturing process, reducing the quality of the recycled PET produced. As mentioned above, many products are made from recycled PET bottles, and the need for manual sorting and identification of foreign material is common in these manufacturing processes.

[0006] The sorted PET bottles are placed in a sterilization bath, and the clean bottles are dried and crushed into fine chips, which are washed again and dried.

[0007] The chips are emptied into a tank and heated, and the molten material is then extruded through a spinneret, a process similar to that used for virgin polyester.

[0008] To improve the manufacturing process for recycled plastic products, such as recycled PET, there is a need to increase the efficiency of the process and, in some embodiments, reduce the probability of introducing foreign material into subsequent manufacturing processes that would degrade the quality of the recycled material. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] US Patent Application Publication No. 2017 / 134086 [Non-patent literature]

[0010] [Non-Patent Document 1] TAN ZHIYING ET AL, "Identification for Recycling Polyethylene Terephthalate (PET) Plastic Bottles by Polarization Vision", IEEE ACCESS, IEEE, USA,Vol. {0} 9, 11 January 2021 (2021-01-11), page 27510-27517 [Non-patent document 2] KALRA AGASTYA ET AL, "Deep Polarization Cues for Transparent Object Segmentation", 2020 IEEE / CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), IEEE,13 June 2020 (2020-06-13), page 8599-8608 [Non-patent document 3] Pham DT: ET AL, "Smart Inspection Systems: Techniques and Applications of Intelligent Vision", "Smart Inspection Systems - Techniques and Applications of Intelligent Vision", page 49-49, 01 January 2003 (2003-01-01), London, UKAcademic Press Summary of the Invention [Problem to be solved by the invention]

[0011] An object of the present disclosure is to improve the detection of containers to achieve the goal of increasing the probability of detecting foreign objects. [Means for solving the problem]

[0012] According to some embodiments of the present disclosure, a device is provided that includes processing circuitry configured to receive a polarized image of an object at least partially made of polarized material, detect a contour of the object from the polarized image, calculate a probability that the contour of the object is a stored contour, and output a control signal indicating a match based on the calculated probability exceeding a threshold.

[0013] The preceding paragraphs have been provided by way of general introduction and are not intended to limit the scope of the claims that follow. The described embodiments, together with further advantages, will be best understood by reference to the following detailed description taken in conjunction with the accompanying drawings, in which:

[0014] The present disclosure and many of the attendant advantages thereof will be readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0015] [Figure 1] The manufacturing process for recycled polyester is shown. [Figure 2] 1 illustrates a system according to some embodiments of the present disclosure. [Figure 3] 1 illustrates a device according to some embodiments of the present disclosure. [Figure 4] 4 illustrates the field of view requirements of lenses used in image sensors according to some embodiments. [Figure 5] 1 illustrates the placement of light sources in a system according to some embodiments. [Figure 6] The images shown are taken when the angles between the light source and the object and between the image sensor and the object are in the range of 55° to 70°, and show the effect of having a second light source positioned above the object. [Figure 7] 1 illustrates edge detection for various finishes on the inside of the housing. [Figure 8] 1 shows images captured using a single light source and a second light source. [Figure 9]1 illustrates a flowchart according to some embodiments of the present disclosure. [Figure 10] 1 illustrates ground truth contours and masks and predicted contours and masks according to some embodiments. [Figure 11] 1 illustrates an example application of some embodiments of the present disclosure to determine the ratio of virgin plastic to recycled plastic. DETAILED DESCRIPTION OF THE INVENTION

[0016] Referring to the drawings, like reference characters refer to the same or corresponding parts throughout the several views.

[0017] Many modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that, within the scope of the appended claims, the present disclosure may be practiced otherwise than as specifically described herein.

[0018] Referring to Figure 1, a process for manufacturing recycled polyester is shown. A large amount of recycled material 1005 is delivered to a manufacturing plant. In some embodiments, the recycled material 1005 is polyethylene terephthalate (PET) plastic. This PET material may be provided in the form of objects such as bottles or other PET containers. Clearly, other transparent materials (e.g., glass) are also contemplated, and the present disclosure is not limited thereto.

[0019] The materials are separated and placed on a conveyor belt 125. This separation is accomplished by, for example, placing the recycled material on a vibrating plate. Alternatively, a person may manually place the material on the conveyor belt 125, or the conveyor belt 125 may vibrate to separate objects in the material. The material includes PET plastic and other non-PET plastic objects. Hereinafter, unwanted objects made of unwanted material will be referred to as "foreign objects." The conveyor belt 125 may have a flat or shaped cross section to allow the material to be transported more easily along the conveyor belt 125. FIG. 1 shows individual objects 1007 being transported along the conveyor belt 125. These individual objects 1007 include PET objects and any foreign objects.

[0020] 1, individual objects 1007 are shown on conveyor belt 125. Individual objects 1007 are fed into a system 100 according to the present disclosure, which is described below.

[0021] System 100 is provided for determining whether each of individual objects 1007 is a foreign object or an object made of PET. System 100 includes device 200, which provides a control signal indicating whether an individual object 1007 being analyzed by system 100 is a PET object or a foreign object. Device 200, described below, in some embodiments provides a control signal to a series of compressed air nozzles (not shown) that operate to blow any foreign objects from conveyor belt 125 in accordance with the control signal provided by system 100. This means that only PET objects remain on conveyor belt 125. In other words, system 100 outputs a control signal indicating whether an individual object 1007 being inspected in system 100 is a PET object. Controlling air nozzles to blow foreign objects from conveyor belt 125 is known. However, generating a control signal using system 100 according to some embodiments of the present disclosure is not known.

[0022] The PET objects 1012 are fed to a cleaning shredder 1015, which is configured to clean, dry and shred the individual PET objects 1012. The output of the cleaning shredder 1015 is PET flakes. The use of a cleaning shredder 1015 to produce PET flakes is known and will not be described in detail for the sake of brevity. The PET flakes are fed to a PET washing device 1020 where they are again washed and dried to ensure that all dirt and moisture is removed from the PET flakes, improving the quality of the recycled polyester output from the process. As the PET washing device 1020 is a known device, a description thereof will not be provided.

[0023] The output from the PET washing device 1020 is shown as washed PET flakes 1022, which are fed into a tank 1025 and melted. The melted PET is fed into a spinneret within a housing 1030 to produce recycled polyester yarn 1035.

[0024] It will be appreciated that the overall process for manufacturing recycled polyester yarn is known, but the features of system 100 are not.

[0025] Empty containers are often transparent objects, which makes identifying empty containers very difficult using traditional imaging techniques. It is an object of the present disclosure to address this problem.

[0026] 2 illustrates a system 100 according to some embodiments of the present disclosure. In the system 100, a device 200 according to some embodiments of the present disclosure is connected to an image sensor 105 (e.g., a machine vision camera, etc.) and various other components of the system 100. In particular, the device 200, in some embodiments, is connected to and controls the operation of one or more light sources 115 and a conveyor 125. The device 200 can be directly connected to and directly control the components of the system 100, although the present disclosure is not limited in this respect. For example, the device 200 sends control signals to a controller (not shown), which controls the operation of the components of the system 100, such as the speed of the conveyor 125.

[0027] As mentioned above, in some embodiments, system 100 is provided in a recycled polyester yarn manufacturing process, but the present disclosure is not limited thereto, and device 200 and system 100 are applicable to any machine that detects the contours of an object at least partially made of a polarized material. In some embodiments, the object can be a bottle, which can be made of a transparent material such as PET. Of course, the object can have a printed label, such as a brand name or barcode, or can be wrapped in film.

[0028] As will be appreciated by those skilled in the art, while the empty container is transparent, the present disclosure is not limited thereto and contemplates any object made of polarizing material, which is particularly advantageous since the outline of such objects is transparent and therefore more difficult to detect using RGB or monochrome image sensors.

[0029] These empty containers are received as a batch of material and processed by the system 100 .

[0030] If the empty container being inspected is accepted as being made of a material suitable for producing recycled material, an appropriate control signal is generated and the empty container being inspected passes through the air nozzles, but if the empty container being inspected is a foreign object, an appropriate control signal is generated and a jet of air from the air nozzles removes the foreign object from the conveyor belt 125.

[0031] System 100 comprises device 200, image sensor 105 (in some embodiments, a machine vision camera including a polarization sensor), and light source 115. Additionally, conveyor 125 is provided to transport empty containers from an opening where the empty containers are placed into the field of view of image sensor 105. Image sensor 105 is connected to lens 106, which has a fixed field of view, as will be described in more detail below.

[0032] In some embodiments, the conveyor 125 is shaped so that both sides of the conveyor belt 125 move synchronously to transport empty containers. The purpose of this shaped conveyor belt 125 is to center the empty containers on the conveyor 125 and allow the containers to move along the conveyor belt 125. In some embodiments, the conveyor belt 125 has a curved or tapered cross-section with a vertical depth suitable for holding PET containers. In other words, the z-distance from the horizontal plane to the bottom of the curved or tapered conveyor belt 125 is suitable for holding PET containers, such as bottles. This arrangement allows empty containers of various shapes to be fed into the machine. Furthermore, because the majority of empty containers are bottles, the conveyor 125, shaped with a curved or tapered cross-section, holds the bottles and prevents them from moving during transport.

[0033] While conveyor 125 has been described above as having a curved or tapered cross section, the present disclosure is not limited thereto. For example, conveyor 125 may be flat or angled and constructed from a sticky material that grips empty containers and moves them through the processing steps. In some embodiments, a conveyor is not required, and individual empty containers may be introduced into system 100 using a robotic arm or the like.

[0034] It is also envisioned that multiple objects are processed simultaneously and placed on the conveyor 125. Indeed, the wide field of view of the lens 106 allows for the capture of multiple objects in the same image, thereby increasing the throughput of object processing. These objects may be spaced apart on the conveyor 125 or may overlap each other. In embodiments where the objects are spaced apart on the conveyor 125, the multiple objects are identified and processed individually. However, when the objects overlap, the top object (i.e., the one closest to the image sensor 105) is correctly identified, while the other object(s) are partially occluded. However, by matching the partially occluded contours and masks with the ground truth versions, the partially occluded object can be identified.

[0035] If one object is made of a first material (e.g., PET) and a second object is made of a second material (e.g., glass), the refractive index of each material will be different, improving the contour of the correct material compared to the other material.

[0036] In some embodiments, the conveyor 125 moves at a speed that depends on the required throughput of the application, which is calculated from a combination of the object length, the lens's field of view, the imaging frame rate, and the camera's exposure time. This ensures that at least one complete imaging view of the object is obtained with minimal image blur. Additionally, in some embodiments, the conveyor belt 125 is made of plastic or rubber to allow for periodic cleaning, as empty containers are likely to contain residual contents that may leak during transport. Therefore, cleaning the conveyor 125 is desirable to reduce the risk of malfunction during use.

[0037] In some embodiments, the light source 115 is positioned above the conveyor 125. In particular, the light source 115 is positioned above the horizontal plane of the conveyor 125. The height of the light source 115 above the horizontal plane of the conveyor 125 is set so that the entire object is illuminated, thereby illuminating the empty container during processing. This improves the accuracy of the processing results. The light source 115 is described in more detail below. However, in some embodiments, the light source 115 extends along the length of the conveyor belt 125.

[0038] In some embodiments, empty containers are fed into housing 120 for processing. Specifically, the empty containers enter through an opening (not shown) and are transported by conveyor 125 into housing 120 for processing. Once processing is complete, the empty containers are ejected from housing 120 and travel along a conveyor belt into cleaning shredder 1015. Housing 120 is large enough so that the empty containers to be inspected can fit within the curved or tapered cross-section and within housing 120.

[0039] The dimensions of the housing should be selected to accommodate empty containers of various sizes. For example, it is desirable that the housing have a length (in the x-direction) sufficient to accommodate the empty containers. For example, the housing 120 is long enough to accommodate an empty 2-liter PET bottle. Additionally, the conveyor belt 125 within the housing 120 should be long enough to allow the containers to overlap slightly at the end of the conveyor 125. The width of the housing is approximately twice the width of the empty containers. The conveyor belt 125 extends the length of the housing 120. Of course, the present disclosure is not limited in this respect, and the housing and conveyor 125 can be any size.

[0040] In some embodiments, the image sensors 105 are positioned above the conveyor 125. Specifically, the image sensors 105 are positioned above the horizontal plane of the conveyor belt 125 so that the lenses can image the objects being inspected. Furthermore, in some embodiments, the image sensors 105 are positioned at equal intervals along the conveyor 125. The image sensors 105 are positioned such that they can capture an image that includes the entire interior of the housing. This requires the use of a wide-angle lens 106, as described below. In some embodiments, the wide-angle lens 106 is a fisheye lens.

[0041] Image sensor 105 captures polarized images. Specifically, image sensor 105 is a polarized image sensor that captures polarized images of an empty container (an example of a transparent object). Any sensor capable of capturing polarized images is contemplated.

[0042] Referring to FIG. 2 , a device 200 according to some embodiments of the present disclosure is shown. In some embodiments, the device 200 is a computer. However, the present disclosure is not limited thereto, and the device 200 may be any device, such as an Application Specific Integrated Circuit (ASIC), capable of processing information and transmitting control signals based on input information. The device 200 includes a processing circuit 205. The processing circuit 205 may be any type of circuit operable by computer-readable instructions to perform methods according to some embodiments of the present disclosure, and may be a single circuit or multiple circuits. The processing circuit 205 receives an image from the image sensor 105. Furthermore, the processing circuit 205 transmits control signals to the image sensor 105, as will be apparent below.

[0043] Processing circuitry 205 is coupled to memory 210. In some embodiments, memory 210 is included in device 200, although the disclosure is not limited in this respect and memory 210 may be located remotely from device 200. In some embodiments, memory 210 is solid-state memory, although the disclosure is not limited in this respect and may be magnetically or optically readable memory.

[0044] Additionally, processing circuitry 205 is connected to a database 215 of ground truth data for known empty containers. This ground truth data is used to train a model so that processing circuitry 205 can detect any container made of polarized material. This allows device 200 to make an accept or reject decision. In some embodiments, an accept decision allows the inspected object to pass through cleaning shredder 1015, while a reject decision, meaning the inspected object is a foreign object, causes the foreign object to be blown off conveyor belt 125 using an air nozzle so that it does not proceed to cleaning shredder 1015.

[0045] In this case, the ground truth data is provided to a neural network training controller, which supplies trained weights to processing circuitry 205 to enable an accept or reject decision for the empty container being inspected. In some embodiments, processing circuitry 205 uses a neural network to make the accept or reject decision for the empty container, providing information describing the shape and dimensions of the empty container. The neural network weights are generated by the neural network training controller using ground truth data of known containers and contours. Each ground truth data entry includes an image, a ground truth contour, and a classification. The contour and optionally the classification enable an accept or reject decision for the empty container.

[0046] Ground truth data can be provided and maintained by the manufacturer of system 100 or the overall process, or by a third party. In some embodiments, the neural network training controller can retrain the neural network and generate new weights periodically. In some embodiments, retraining can be performed to detect new bottles or objects. For example, retraining is performed using multiple images of different types and / or shapes of glass bottles as training data. Retraining can occur if the manufacturing process is changed (e.g., to a different area where other (different) types of empty containers may be present), if the intrusion of ambient light into the manufacturing process changes, or if the supplier of a batch of material changes. However, in some embodiments, a short exposure time of approximately 1 / 100 of a frame is used to mitigate the effects of ambient light. In some embodiments, the neural network is trained with a weight set that accounts for the different or expected amounts of ambient light to which image sensor 105 may be exposed.

[0047] The processing circuitry 205 generates a plurality of output control signals that control parts of the system described with reference to Figure 1 and provide output signals that indicate whether or not the empty container should be approved for delivery to subsequent parts of the manufacturing process. In some embodiments, these output control signals communicate with any controllers already present in the manufacturing process, such as air blowers, to control various components of the manufacturing process as described above.

[0048] [Lens 106] As mentioned above, the lenses used in combination with the image sensor 105 must be carefully selected to ensure that the field of view requirements are met. Figure 4 illustrates the field of view requirements for lenses used in image sensors according to some embodiments. Of course, it will be understood that the field of view requirements may vary depending on the dimensions of the object recognition system hardware in the recycled polyester manufacturing process.

[0049] The image sensor 105 is connected to the lens 106. In some embodiments, it is desirable for the vertical distance from the conveyor 125 to the lens 106 to be short in order to minimize the overall height of the system 100. In some embodiments, the minimum vertical distance from the conveyor 125 to the lens 106 is sufficient to allow transparent objects having a certain diameter to be processed by the system 100. In other words, the minimum vertical distance from the conveyor belt 125 to the lens 106 is sufficient to allow the transparent object to be inspected to be processed by the system 100.

[0050] Furthermore, by positioning the image sensor 105 and lens 106 above the object, the entire shape of the object is visible, improving the detection of narrow-neck bottles. Bottle designs may be asymmetric (e.g., incorporating a twisted structure). A view directly above such bottles may be advantageous for reliable detection of the bottles.

[0051] System 100 preferably begins processing transparent objects as soon as possible after they are deposited into the reverse vending machine (RVM) and continues to track the objects until they pass through the object recognition chamber for further processing within the RVM. Thus, in some embodiments, the viewing range of lens 106 is set to be longer than the length of conveyor belt 125 so that empty containers to be inspected can be viewed.

[0052] A fisheye lens is used to provide the desired viewing angle, but the fisheye lens introduces distortions that must be corrected using software, which is commercially available and will not be described in detail here.

[0053] Naturally, the maximum viewing angle will vary depending on the shape of the housing, and in particular the value of the vertical distance to the object. Therefore, the range of maximum viewing angle values ​​for a given vertical distance to the object is shown in Table 1 below.

[0054] [Table 1]

[0055] The lens 106 may be positioned midway along the conveyor 125 and may be offset from the center (bottom of the taper) of the conveyor 125 to reduce glare from the light source. The lens 106 may also be angled relative to the vertical to reduce glare.

[0056] Additionally, the lens has a depth of field of f=4, which allows for a trade-off between maximizing the amount of light entering the lens opening (keeping the active light intensity requirement as low as possible) and achieving a clear focus on the sides and top of the empty container for accurate detection of the contours of the empty container image.

[0057] [Light source 115] As mentioned above, system 100 includes light source 115. In some embodiments, light source 115 is used to illuminate the empty container. This is shown in FIG.

[0058] In Figure 5, light source 115 is positioned above the end of conveyor 125. Empty containers (bottles in the case of Figure 5) are shown positioned in the tapered section formed by conveyor 125. Light source 115 is positioned above the end of conveyor 125 so as to illuminate the entire container. This is shown in Figure 5.

[0059] The light source 115 is a high-intensity LED strip light with a warm color temperature, and the intensity of this light source 115 allows the exposure time of the image sensor 105 to be set to capture images of the transparent object under inspection on the moving conveyor belt 125 with moderate (30 dB) gain and minimal motion blur.

[0060] The light source 115 is positioned off-center on the conveyor 125. This is shown in Figure 5. The light source 115 is positioned at an angle to the vertical so that the light source 115 points towards the center of the symmetrical bottle. The light source is unpolarized.

[0061] 5 shows light source 115 offset from the center of conveyor 125, the present disclosure is not limited in this respect. Indeed, in some embodiments, light source 115 can be positioned anywhere relative to the center of conveyor 125. For example, light source 115 can be positioned directly above the centerline of conveyor 125.

[0062] In some embodiments, the positions of the light source 115 and the image sensor 105 can be positioned such that the angle between the image sensor 105 and the empty container (a bottle in the non-limiting example of FIG. 5) and the angle between the light source 115 and the empty container are in the range of 55° to 70°. In particular, but without limitation, the range can be any discrete angle such as 55°, 55.5°, 56°, 56.5°, 57°, 57.5°, 58°, 58.2°, 58.4°, 58.6°, 58.8°, 59°, 59.2°, 59.4°, 59.6°, 59.8°, 60°, 60.2°, 60.4°, 60.6°, 60.8°, 61°, 61.5°, 62°, 62.5°, 63°, 63.5°, 64°, 64.5°, 65°, 65.5°, 66°, 66.5°, 67°, 67.5°, 68°, 68.5°, 69°, 69.5°, or 70°. This range is advantageous because it increases as the angle between light source 115 and the empty container approaches the Brewster angle of the empty container's material, while simultaneously reducing the space required to house light source 115, image sensor 105, and the empty container. This increase in angle increases the amount of polarized light reflected from the empty container. This improves the contrast between the imaged empty container's outline and the housing background, while simultaneously reducing the space required.

[0063] If the empty container is polyethylene terephthalate (PET), a particular angle within the above range is approximately 60°, although different materials may receive slightly different amounts of polarized light at the same angle. This approximate angle is particularly advantageous because 60° is the angle at which light from light source 115 shines on the empty container, refracts into the container, and then reflects off the container, improving contrast while minimizing space usage.

[0064] As can be seen in the top image of Figure 6, there is improved contrast in region A, which represents the top of the bottle, and improved edge definition along the length of the bottle (denoted by B in the image).

[0065] The above describes a case where the angle between the image sensor 105 and the empty container and the angle between the light source 115 and the empty container are within the range of 55° to 70°, but the present disclosure is not limited to this, and naturally, any relative arrangement of the image sensor 105, the empty container, and the light source 115 is envisioned.

[0066] In some embodiments, the light source may be a polarized light source. The light source may be filtered with a polarizing filter. Polarization may be a factor in determining the placement of the light source and image sensor, which can be determined experimentally. When properly placed, the light source can enhance or amplify the polarized light applied by the sensor, producing a clearer image and therefore more robust object detection.

[0067] In some embodiments, the light source may be an infrared light source. Using an infrared light source reduces the intrusion of ambient light into the system 100 and any reflections from objects.

[0068] In some embodiments, to improve edge detection, the interior walls of the housing 120 in which the empty container is placed during image capture may have a matte finish, as shown in Figure 7. Minimizing light reflections generated by background surfaces and making the background in the captured image as uniform in appearance as possible will improve edge detection results.

[0069] Referring to FIG. 7 , the top image shows edges detected when a polarized RGB image of an empty container placed on the green conveyor 125 and without a matte finish on the inside of the housing 120 is captured. The middle image shows edges detected when a polarized RGB image of an empty container placed on the green conveyor 125 and with a matte finish on the inside of the housing 120 is captured. Finally, the bottom image shows edges detected when a polarized monochrome image of an empty container placed on the green conveyor 125 and with a matte finish on the inside of the housing 120 is captured. As can be seen, the matte finish on the inside of the housing 120 improves the detected edges, increasing the reliability and accuracy of the system. In some embodiments, the light source 115 is turned on when the image sensor 105 captures the polarized image of the empty container. This may mean that the light source 115 is always on while the empty container is being fed onto the conveyor 125.

[0070] However, in some embodiments, the light source 115 may only be on while the image sensor 115 is taking polarized images of the empty container. In other words, the image sensor 105 has an exposure time, and the light source 115 is switched on for the exposure time as the image sensor 105 takes an image. In other words, the image sensor is configured to take a series of images of a transparent object, and the light source is configured to switch on as frequently as the image sensor takes the series of images, such that the light source is on as the image sensor takes each image in the series. This reduces energy consumption of the system 100 and reduces the impact of sunlight on the conveyor belt 125.

[0071] Providing a matte finish to the interior of the housing and / or the conveyor is advantageous in that it reduces the likelihood of sensor saturation or partial saturation. This avoids producing images containing saturated pixels, such as completely white pixels that may result from internal reflections within a transparent object. Saturation can adversely affect accurate detection results or images that lack clear contours, as would normally be achieved. However, in some embodiments, portions of the interior of the housing or portions of the conveyor may include non-matte or reflective surfaces to provide illumination and assist in producing images with sufficient dynamic range. The effects of saturation can be mitigated by replacing saturated pixel values ​​with known (optionally neighboring or spatially nearby) values ​​and / or by predicting the pixel values ​​of saturated pixels.

[0072] While the system 100 has been described above as having a single light source, the present disclosure is not limited thereto. In some embodiments, a second light source is provided. This second light source may or may not be identical to the first light source 115. In some embodiments, the first light source is positioned offset from the conveyor 125 (which, in some non-limiting examples, may mean that the relative angle between the image sensor 105 and the light source 115 is within an advantageous range of 55° to 70°), and the second light source is positioned directly above the center of the conveyor 125. In this case, when illuminated simultaneously with the first light source, the second light source provides a high light intensity that can shorten exposure times compared to a single light source. Additionally, while the inclusion of a second light source increases specular reflection, the quality of the imaged empty container outline remains unchanged. This is useful when comparing the outline of a transparent object to a stored outline to determine whether the empty container can be accepted and processed in a manufacturing process.

[0073] This is shown in Figure 8. The top image in Figure 8 shows a situation where a single light source is used, where the angle between the image sensor and the object and the angle between the light source and the object are within the range of 55° to 70°, while the bottom image in Figure 8 shows a situation where a first light source is placed at a position where the angle between the image sensor and the object and the angle between the light source and the object are within the range of 55° to 70°, and a second light source is placed above the center of the conveyor 125. In other words, the bottom image is the same as the top image, with the addition of a second light source above the center of the conveyor 125.

[0074] 9 shows a flowchart 800 illustrating the operation of the system 100 and the device 200. The flowchart 800 is divided into portions performed by the system 100 as a whole (which may be a controller for the manufacturing process) and portions performed by the device 200, which may send control signals to the air nozzles.

[0075] Flowchart 800 begins when an object, which is a bundle of material, is detected on the conveyor 125. The process begins at step 805, where the image sensor 105 is initialized. In particular, the image sensor 105's automatic gain control, auto exposure (AE), and frame synchronization signal (framing strobe) are initialized. The process proceeds to step 810, where it is determined whether an object is present in the image captured by the image sensor 105. Because the image sensor 105's field of view covers both sides of the conveyor belt 125, the conveyor belt 125 does not need to be activated to detect the presence of an object (or part of an object). If no object is detected by the image sensor 105, the No route is taken and the system waits until an object is detected. A timer may be set to reset the system if no object is detected within a certain period of time (e.g., 60 seconds).

[0076] If an object is photographed by the image sensor 105, the "Yes" route is taken and the process proceeds to step 830. In step 830, the conveyor belt 125 is started and the object is tracked as the conveyor belt 125 moves. A frame timer is started in step 825. The frame timer duration is set so that at least one full image of a very large bottle is captured within the field of view of the object recognition chamber before the conveyor begins to move the bottle from the far end of the chamber. In some embodiments, the frame timer is set to allow a particular frame rate. The process proceeds to step 820, where it checks whether the frame timer has expired. If the timer has not yet expired, the "No" route is taken and the conveyor 125 continues to operate. However, if the frame timer has expired, the "Yes" route is taken and the conveyor 125 is stopped in step 815 and the flowchart returns to step 810.

[0077] Returning to step 830, simultaneously with setting the frame timer, image sensor 105 begins taking images in step 835. As conveyor 125 moves, image sensor 835 takes images of the empty container at the frame rate. Once an image is taken, the image and an image identifier (an identifier that uniquely identifies the image from other images) are sent to a storage unit. This storage unit may be a buffer or the like that stores the image in association with the image identifier. The image and image identifier are then provided to a tracking mechanism that identifies the location of the empty container on conveyor 125 when the image was taken.

[0078] A bounding box may be established around each empty container on the conveyor 125. A tracking mechanism may track the position of the empty container on the conveyor 125 and store the position in association with the image and the image identifier, allowing the position of the empty container to be tracked along the conveyor 125. In some embodiments, the tracking mechanism is a single-pass convolutional neural network and therefore has low processing overhead.

[0079] If the empty container is determined to be optimally positioned for the lens and light arrangement (e.g., the empty container is centered within the lens's field of view or the edge of the empty container is included in the captured image), the associated image is retrieved from storage and used by the rest of the system. In other words, the retrieved image is output from step 835.

[0080] The process then proceeds to step 840, where the extracted image is corrected for barrel distortion and other defects caused by the wide-angle lens, and for sensor defects such as flat-field correction and gamma correction. The process then proceeds to step 845. In step 845, the polarization angle of the image sensor 105 is decoded and the captured polarization data is extracted for image enhancement. In some embodiments, the image sensor has four polarization angles. To improve the image for edge detection, the polarization extraction gain is set to 6 dB and the depth of polarization gain is set to 9 dB. This processing results in a polarization image angle, polarization image degree, and intensity image. The use of an image sensor with multiple polarization angles or an image sensor with a configurable polarization angle is optional but advantageous. Applying a polarization filter to a conventional image sensor (e.g., a CCD or CMOS image sensor) produces only one phase angle unless a complex configurable filter is applied. These examples allow multiple (e.g., four) images to be effectively combined in different ways in real time with different weightings to produce an enhanced contour image. Multiple polarization angles can define respective Stokes parameters.

[0081] In step 850, the contrast of the image is enhanced and maximized as much as possible, which assists in the contour extraction performed in step 855. In particular, in step 855, three Region-based Convolutional Neural Networks (RCNNs) process the angle of polarization image, the degree of polarization image, and the intensity image, respectively. The outputs of these three neural networks are combined to generate an optimal predicted empty vessel contour and a predicted empty vessel mask. Examples of these are shown in FIG. 10.

[0082] Additionally, FIG. 10 shows an example of ground truth contours and ground truth masks used to train a region-based convolutional neural network to calculate the probability of an inspected container being empty. The use of a neural network makes it possible to predict the probability of an inspected container being empty despite physical deformation or partial crushing of the empty container. This is because some empty containers may be designed with material weaknesses that allow them to be manually deformed to reduce bulk for easier transport to a recycling center and to be compacted when bundled at the recycling center. Such deformations may be deformations to a predetermined size or shape.

[0083] In some embodiments, ground truth contours may be provided for these deformed empty containers. As will be appreciated, for a region-based convolutional neural network, the object must be classified. In some embodiments, the object classification is set to PET bottles, glass bottles, or non-bottle objects as examples of classifiable objects. While this disclosure is described with respect to a region-based convolutional neural network, other types of neural networks or combinations of neural networks may also be used.

[0084] The ground truth contour and ground truth mask can be provided to device 200 from database 215. In some embodiments, the database is a training database containing many examples of different ground truth contours and masks for any transparent object. The output of step 855 is a detection probability, a predicted contour, and a predicted mask. The detection probability indicates the probability that the inspected empty container is an object acceptable for automatic collection. In other words, it is the probability that the observed object is the same class of acceptable object defined in the ground truth data; the neural network can use any features detected in the input data in this comparison. Then, based on the calculated probability exceeding a threshold, a control signal indicating a match is output. This control signal is provided to the air nozzle; if the probability exceeds the threshold, the transparent object is allowed to continue in the manufacturing process. In some embodiments, device 200 can output a control signal instructing the air nozzle to accept or reject the inspected empty container for the remainder of the manufacturing process.

[0085] As can be seen, region-based convolutional neural networks have a higher processing overhead than single-pass convolutional neural networks used in the tracking mechanism. Combining a single-pass convolutional neural network for empty container tracking with a region-based convolutional neural network for the contour detection system means that the system can take many images but provide the same accuracy information. When using only the region-based convolutional neural network, the frame rate is limited by the region-based convolutional neural network. This results in a frame rate of approximately 5-6% of the frame rate of the combination of a single-pass convolutional neural network and a region-based convolutional neural network being typical for a system provided with only the region-based convolutional neural network. Therefore, in some embodiments, it is advantageous to have an object detection device comprising processing circuitry configured to receive a polarized image of an object at least partially made of polarized material, detect a location of the object using a single-pass convolutional network, detect a contour of the object from the polarized image when the object is at a predetermined location, calculate a probability that the contour of the object is a stored contour using a region-based convolutional neural network, and output a control signal indicative of a detected object based on the calculated probability being above a threshold.

[0086] In some embodiments, any features extracted from the predicted contour (including, but not limited to, shape and dimensions) are used to determine whether the empty container being inspected should be accepted into the manufacturing process. This ensures that only the correct type of container, made from the appropriate material and size, is accepted into the manufacturing process. Additionally, in some embodiments, crushed or otherwise deformed objects can be detected and rejected.

[0087] In some embodiments, the input image, predicted contour, predicted mask, and predicted classification may be stored for quality control. In testing, the predicted contour shows an average accuracy of 1 mm compared to the ground truth contour, resulting in 100% correct bottle detection in over 1000 images.

[0088] In some embodiments, as described above, some objects may have complex design features, such as bottles with narrow necks, asymmetrical design features, or complex lids. Such features may be fully or partially occluded from the sensor's view. This may result in the ground truth contour and mask being segmented into a bottle base and neck, a bottle and lid, or three or more portions. In other words, the object has a first portion and a second portion, and the processing circuitry is configured to receive polarized images of at least a portion of the first and second portions. For a positive and statistically significant detection of the entire object, it is sufficient that a portion is detected against the segmented mask, and that adjacently contiguous portions are detected, even if the adjacently contiguous portions do not entirely match another mask. This approach, for example, will not detect more than one bottle from a bottle cut into two or more portions, but will still detect a single object with multiple portions or edges occluded. This is advantageous in that it increases the likelihood of correctly identifying the object.

[0089] While the above description concerns cases where the entire object is made of polarizing material (a material that reflects or emits polarized light), the present disclosure is not limited thereto, and those skilled in the art will understand that in some embodiments, at least a portion of the object is made of polarizing material.

[0090] The present disclosure generally includes the following steps: first, receiving a polarized image of an object at least partially composed of polarized material; then, a processing circuit detects an outline of the object from the polarized image; calculates a probability that the outline of the object is a stored outline; and finally, outputting a control signal indicating a match based on the calculated probability exceeding a threshold.

[0091] In addition to embodiments used to sort containers for recycled polyester production, some embodiments of the present disclosure can be used in other scenarios.

[0092] As mentioned above, recycling centers collect PET bottles and other containers used to produce recycled materials. In many areas, consumers pay a bottle deposit when they purchase a product packaged in a bottle made from a renewable material, such as PET or glass. The deposit is refunded when the consumer returns the bottle to the recycling center. Given the large number of products contained in bottles or other containers or objects, some recycling programs can pay out very large amounts to consumers in a given year. For example, [1] reports that California's recycling program pays out approximately $1.5 billion in bottle deposits annually. With such high amounts in mind, it has been reported that over $200 million of bottle deposits fall into the hands of criminals.

[0093] In some cases, foreign objects have been mixed into collected shipments of PET containers, and deposits have been fraudulently refunded for these foreign objects. These foreign objects are not PET containers. The inclusion of foreign objects in collected shipments allows deposits to be fraudulently provided, which, while important from a recycling perspective, contaminates the PET container shipments. Therefore, it is desirable to quickly and accurately detect attempts to fraudulently claim deposits.

[0094] In some areas, so-called automated collection machines have been installed. These machines are located in recycling centers and other locations where customers can return PET containers and receive a refund.

[0095] A system similar to system 100 installed in a manufacturing process can be installed in an automated collection machine or other suitable mechanism used to return deposits to consumers.

[0096] In particular, a user may provide a container to be inspected to system 100 via conveyor belt 125. Device 200 within system 100, if there is a match (i.e., the inspected object is a container made of the correct material, such as PET), sends a control signal to a controller within the vending machine. Such a control signal causes the vending machine to implement known refund techniques to return the deposit to the consumer and keep the container. In other words, while in the manufacturing process described above, a control signal indicating a match would control the use of air nozzles to prevent the object from being removed from the manufacturing process, in the vending machine embodiment, the control signal indicating a match controls the storage of the inspected object and the return of the deposit.

[0097] In the event of a negative comparison (i.e., the object being tested is a foreign object), device 200 sends an appropriate control signal and the object being tested is returned to the consumer without a refund of the deposit, similar to a manufacturing process in which a control signal indicating a discrepancy controls an air nozzle to blow the object being tested off conveyor belt 125.

[0098] In this embodiment, the system 100 receives each empty container, verifies that the empty container is suitable for the collection machine (i.e., is the correct size, is made of the correct material, is weighed to not accept fully or partially filled objects, and is processable), and, if appropriate, refunds any deposit the consumer paid for that container when purchasing the original product.

[0099] If a refund is to be issued, device 200 in system 100 (or a controller to which device 200 provides control signals) verifies that the empty container is not an attempt to fraudulently receive a refund.

[0100] To efficiently utilize the vending machine's storage, empty containers are typically stored in receptacles that are compacted and periodically emptied to maximize space within the vending machine. This disclosure relates to the empty container acceptance process, not the compaction and storage of empty containers, and will not be described in detail below.

[0101] If the machine does not accept the container (for example because it is full of liquid, is not the right size, or is in any way unsuitable for the machine), the rejected container will be returned to the customer. If a rejected container is returned to the customer, an alarm may sound and the customer may lose any opportunity to receive a refund for the container. Of course, rejected containers may also be stored in another bin within the machine.

[0102] In some embodiments, a barcode scanner may be installed at the opening where the customer places the empty container. The barcode scanner searches for a barcode on the empty container. Typically, the barcode is uniquely associated with the product, and therefore with the outline of the empty container. Thus, when a barcode is detected, it is associated with the outline of the particular empty container. Thus, in some embodiments, the barcode is used to verify that the detected outline matches the outline associated with the barcode. If there is no match, the empty container may be rejected.

[0103] In some embodiments, the empty container is weighed at the opening where the customer places it. A weight check determines whether the empty container has any remaining contents (if so, it is rejected as not empty). If the empty container is heavier than the acceptable range of empty containers, it is immediately rejected. It will be appreciated that many of the advantageous features of the devices and systems used in the present manufacturing process may be applied to automated collection machines. For example, advantageous features such as a tapered conveyor belt that carries objects through the system may be applied where appropriate.

[0104] Further possible applications of some embodiments of the present disclosure are described in connection with FIG. 11 . In the manufacture of plastic products (e.g., those made using extrusion techniques), it is common to produce plastic products from a combination of recycled and virgin plastics. Often, the proportion of virgin to recycled plastic varies depending on the use for which the plastic product is being produced. For example, medical-grade plastic products will contain a higher proportion of virgin plastic than non-medical-grade products. In some cases, product vendors may sell their products on the condition that the containers are made from a specified percentage, or at least a specified percentage, of recycled plastic.

[0105] FIG. 11 illustrates an example application of some embodiments of the present disclosure to determine the ratio of virgin plastic to recycled plastic. In the top portion of FIG. 11, a first hopper 1110 and a second hopper 1120 feed plastic onto a conveyor belt 125. The first hopper 1110 contains virgin plastic beads, and the second hopper 1120 contains recycled plastic flakes. In other words, the first hopper 1110 contains virgin plastic beads, and the second hopper 1120 contains recycled plastic flakes. Note that in some embodiments, the contours of the plastic beads and the contours of the plastic flakes are different. In fact, according to some embodiments, the shapes of the recycled plastic and the virgin plastic are not limited; however, the shapes of the two plastics are different. This plastic mixture is melted and fed into an extruder or other device to feed the plastic manufacturing process.

[0106] The ratio (by weight) of virgin plastic beads to recycled plastic flakes is selected depending on the plastic product being produced in the extrusion process. As mentioned above, products made from medical-grade plastics typically contain a higher proportion of virgin plastic than products made from single-use plastics, which tend to have a higher proportion of recycled plastic. This makes it important to regularly sample the ratio of virgin to recycled plastic being melted and fed to the extrusion process for quality control purposes.

[0107] To improve this process, the system 100 of some embodiments of the present disclosure is used. The plastic mixture 1150 from the first hopper 1110 and the second hopper 1120 is shown in plan view at the bottom of FIG. 11. It is fed into the system 100 of some embodiments of the present disclosure. A close-up of this mixture is shown. As can be seen, the mixture is a combination of recycled plastic flakes 1170 and virgin plastic beads 1160. As the mixture passes through the system 100, some embodiments of the present disclosure are implemented to derive the contours of each plastic component in the mixture. This allows for an accurate count of the number of plastic flakes and the number of plastic beads in any one sample. This allows for a determination of whether the ratio of recycled plastic to virgin plastic is correct.

[0108] To the extent that embodiments of the present disclosure are described as being implemented at least in part by a software-controlled data processing apparatus, it will be understood that non-transitory machine-readable media, such as optical disks, magnetic disks, semiconductor memories, and the like, that store such software also are considered embodiments of the present disclosure.

[0109] It will be appreciated that the above description has, for clarity, described embodiments with reference to different functional units, circuits and / or processors. However, it will be apparent that any suitable distribution of functionality between different functional units, circuits and / or processors may be used without detracting from the embodiments.

[0110] The described embodiments may be implemented in any suitable form including hardware, software, firmware, or any combination of these. The described embodiments may optionally be implemented at least in part as computer software running on one or more data processors and / or digital signal processors. The elements and components of any embodiment may be physically, functionally, and logically implemented in any suitable way. Indeed, functionality may be implemented in a single unit, in multiple units, or as part of other functional units. Thus, the disclosed embodiments may be implemented in a single unit or may be physically and functionally distributed between different units, circuits, and / or processors.

[0111] Although the present disclosure has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Moreover, even if a feature appears to be described in connection with a particular embodiment, those skilled in the art will recognize that the various features of the described embodiments can be combined in any manner suitable for implementing the technology.

[0112] Embodiments of the present technology can be generally described by the following numbered clauses: (1) receiving a polarized image of an object at least partially composed of polarized material; Detecting the contour of the object from the polarization image; calculating a probability that the contour of the object is a stored contour; and configured to output a control signal indicating a match based on the calculated probability exceeding a threshold. Processing Circuit Equipped with device. (2) The device according to paragraph (1), The object is at least partially made of a transparent polarizing material. device. (3) The device according to paragraph 1 or 2, the object having a first portion and a second portion; The processing circuitry is configured to receive the polarized image of at least a portion of the first portion and the second portion. device. (4) A device according to any preceding paragraph; an image sensor configured to output the polarized image of the object; Light source and Equipped with system. (5) The system according to paragraph (4), The light source is a polarized light source. system. (6) The system according to paragraph 4 or 5, The image sensor is configured to capture a series of images of the object, and the light source is configured to switch on at a frequency that is equal to the frequency at which the image sensor captures the series of images, such that the light source is on as the image sensor captures each image in the series. system. (7) The system according to any one of paragraphs 4 to 6, The light source and the image sensor are arranged so that the angle between the image sensor and the object and the angle between the light source and the object are within a range of 55° to 70°. system. (8) The system according to claim 7, The angle is about 60° system. (9) The system according to paragraph 7 or 8, a second light source; The second light source is configured to be positioned above the center of the object and is turned on simultaneously with the first light source. system. (10) The system according to any one of paragraphs 4 to 9, The image sensor is configured to capture an image including a plurality of objects, and the device is configured to calculate a probability that a contour of each of the objects is a stored contour, and to output a control signal indicating a match for each of the objects based on the calculated probability for each of the objects exceeding a threshold. system. (11) receiving a polarized image of an object at least partially composed of a polarized material; Detecting the contour of the object from the polarization image; calculating a probability that the contour of the object is a stored contour; outputting a control signal indicating a match based on the calculated probability exceeding a threshold value; method. (12) The method according to claim 11, The object is at least partially made of a transparent polarizing material. method. (13) The method according to item 11 or 12, the object having a first portion and a second portion; receiving the polarized image of at least a portion of the first portion and the second portion; method. (14) Any one of the methods according to items 11 to 13, taking a series of images of the object and switching the light source on as often as the series of images are taken so that the light source is on when each image in the series is taken; method. (15) The method according to claim 14, The light source is a polarized light source. method. (16) The method according to item 14 or 15, The image sensor that captures the series of images and the light source are arranged so that the angle between the image sensor and the object and the angle between the light source and the object are within a range of 55° to 70°. method. (17) The method according to claim 16, The angle is about 60° method. (18) The method according to item 16 or 17, A second light source is placed above the center of the object and turned on simultaneously with the first light source. method. (19) Any one of the methods according to items 11 to 18, capturing an image including a plurality of objects, calculating a probability that the contour of each of the objects is a stored contour, and outputting a control signal indicating a match for each of the objects based on the calculated probability for each of the objects exceeding a threshold value; method. (20) A computer program product comprising computer-readable instructions that, when loaded into a computer, configures the computer to perform the method of any one of paragraphs 11 to 19.

[0113] [reference] [1] https: / / californiaglobe.com / environment / calrecycle-loses-200-million-a-year-due-to-bottle-deposit-fraud /

Claims

1. receiving a polarized image of an object at least partially composed of polarized material; Detecting the contour of the object from the polarization image; calculating a probability that the contour of the object is a stored contour; and configured to output a control signal indicating a match based on the calculated probability exceeding a threshold. Processing circuit (100) Equipped with Device (200).

2. 10. The device of claim 1, The object is at least partially made of a transparent polarizing material. device.

3. 10. The device of claim 1, the object having a first portion and a second portion; The processing circuitry is configured to receive the polarized image of at least a portion of the first portion and the second portion. device.

4. A device according to claim 1; an image sensor configured to output the polarized image of the object; Light source and Equipped with System (100).

5. 5. The system of claim 4, The light source is a polarized light source. system.

6. 5. The system of claim 4, The image sensor is configured to capture a series of images of the object, and the light source is configured to switch on at a frequency that is equal to the frequency at which the image sensor captures the series of images, such that the light source is on as the image sensor captures each image in the series. system.

7. 5. The system of claim 4, The light source and the image sensor are arranged so that the angle between the image sensor and the object and the angle between the light source and the object are within a range of 55° to 70°. system.

8. 8. The system of claim 7, The angle is about 60° system.

9. 8. The system of claim 7, a second light source; The second light source is configured to be positioned above the center of the object and is turned on simultaneously with the first light source. system.

10. 6. The system of claim 5, The image sensor is configured to capture an image including a plurality of objects, and the device is configured to calculate a probability that a contour of each of the objects is a stored contour, and to output a control signal indicating a match for each of the objects based on the calculated probability for each of the objects exceeding a threshold. system.

11. receiving a polarized image of an object at least partially composed of polarized material; Detecting the contour of the object from the polarization image; calculating a probability that the contour of the object is a stored contour; outputting a control signal indicating a match based on the calculated probability exceeding a threshold value; method.

12. 12. The method of claim 11, The object is at least partially made of a transparent polarizing material. method.

13. 12. The method of claim 11, the object having a first portion and a second portion; receiving the polarized image of at least a portion of the first portion and the second portion; method.

14. 12. The method of claim 11, taking a series of images of the object and switching the light source on as often as the series of images are taken so that the light source is on when each image in the series is taken; method.

15. 15. The method of claim 14, The light source is a polarized light source. method.

16. 15. The method of claim 14, The image sensor that captures the series of images and the light source are positioned so that the angle between the image sensor and the object and the angle between the light source and the object are within a range of 55° to 70°. method.

17. 17. The method of claim 16, The angle is about 60° method.

18. 17. The method of claim 16, A second light source is placed above the center of the object and turned on simultaneously with the first light source. method.

19. 12. The method of claim 11, capturing an image including a plurality of objects, calculating a probability that the contour of each of the objects is a stored contour, and outputting a control signal indicating a match for each of the objects based on the calculated probability for each of the objects exceeding a threshold value; method.

20. A computer program product comprising computer readable instructions that, when loaded into a computer, configures the computer to perform the method of claim 11.

Images