Recycling coins from scrap metal

Abstract

To provide, in general, the sorting of materials, and in particular, the sorting of certain valuables from scrap.SOLUTION: A material sorting system sorts materials utilizing a vision system that implements a machine learning system so as to identify or classify each of the materials. The materials are sorted into separate groups based on the identification or classification determining that the materials have specified geometric shapes. Such a system can sort monetary coins or other valuable metals from other forms of scrap.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This application is a continuation-in-part of U.S. Patent Application No. 15 / 963,755, which claims priority to U.S. Provisional Patent Application No. 62 / 490,219, both of which are incorporated herein by reference.

[0002] Government Licensing Rights This disclosure was made with U.S. Government support under Grant No. DE-AR0000422 from the U.S. Department of Energy. The Government may have certain rights in this disclosure.

[0003] Technical Field FIELD OF THE DISCLOSURE The present disclosure relates generally to material sorting, and more particularly to the sorting of certain valuable items from scrap. [Background technology]

[0004] This section is intended to introduce various aspects of the art that may be related to exemplary embodiments of the present disclosure. The discussion is believed to be helpful in providing a framework to facilitate a better understanding of certain aspects of the present disclosure. Accordingly, it should be understood that this section should be read in this light, and not necessarily as an admission of prior art.

[0005] Recycling is the process of collecting, treating, and transforming materials that would otherwise be discarded as trash into new products. Recycling benefits local communities and the environment by reducing the amount of waste sent to landfills and incinerators, conserving natural resources, increasing economic security by using domestic materials, preventing pollution by reducing the need to collect new raw materials, and saving energy. After collection, recyclables are typically sent to materials recovery facilities where they are sorted, cleaned, and processed into materials that can be used in manufacturing. Summary of the Invention [Problem to be solved by the invention]

[0006] Many automobiles designated for shredding and subsequent recycling processes have been found to contain relatively significant numbers of numismatic coins located between the seats, under the floor mats, etc. At least one study has estimated that there may be approximately US$10-15 in coins per vehicle. Similarly, such vehicles may contain lost jewelry.

[0007] Automobiles also contain printed circuit boards ("PCBs") that contain valuable metals (eg, copper, gold, silver, etc.) that can be recycled.

[0008] Given the large number of vehicles recycled each year, the recycling industry is seeking techniques to recover valuable scrap pieces as a beneficial by-product of the normal vehicle recycling process. Furthermore, federal legislation was recently passed that requires the U.S. government to pay the face value of even damaged coins. [Additional note 1] capturing image data with a camera for each piece of a heterogeneous mixture of materials moving in a stream past the camera, the materials of the heterogeneous mixture of materials having a variety of different shapes, including one or more different geometric shapes; classifying the material having one or more specified geometric shapes into a first classification; classifying the material that does not have the one or more specified geometric shapes into a second classification; separating, by an automated sorting device, the materials classified in the first category from the materials classified in the second category; A method for providing [Additional note 2] 2. The method of claim 1, wherein the one or more different geometric shapes include a circle. [Additional note 3] 10. The method of claim 1, wherein the one or more different geometric shapes include a polygon. [Additional note 4] The method according to claim 1, wherein the material has a circular shape and also has holes formed therein, and is classified into the second category. [Additional note 5] 10. The method of claim 1, wherein the material having the specified geometric shape comprises a monetary coin. [Additional note 6] 2. The method of claim 1, wherein the first classification is numismatic coins and designated gemstones. [Additional note 7] 2. The method of claim 1, further comprising passing the bulk of the material through a sieve before capturing the image data to generate a heterogeneous mixture of the material having a size less than a predetermined size. [Additional note 8] a camera configured to capture image data of each piece of a heterogeneous mixture of materials moving in a stream past the camera, the materials of the heterogeneous mixture of materials having a variety of different shapes, including one or more different closed geometric shapes; a circuit configured to classify the material having a specified closed geometric shape into a first classification; a circuit configured to classify the material that does not have the specified closed geometric shape into a second classification; an automated sorting device configured to separate the materials classified in the first category from the materials classified in the second category; A system comprising: [Additional note 9] 9. The system of claim 8, wherein the material having the specified closed geometric shape has a circular shape. [Additional Note 10] 10. The system of claim 9, further comprising a circuit configured to classify a material having a circular shape and having holes formed therein into the second classification. [Additional Note 11] 9. The system of claim 8, further comprising a sieve for separating heterogeneous mixtures of the material having a size below a predetermined size prior to capturing the image data. [Additional Note 12] 9. The system of claim 8, wherein the one or more different closed geometric shapes include a circle. [Additional Note 13] 9. The system of claim 8, wherein the one or more different closed geometric shapes include a polygon. [Additional Note 14] 10. The system of claim 9, wherein the first classification is monetary coins. [Additional Note 15] 9. The system of claim 8, wherein the heterogeneous mixture of materials comprises a zorba. [Additional Note 16] 9. The system of claim 8, wherein the heterogeneous mixture of materials includes scrap from end-of-life automobiles. [Additional Note 17] 1. A computer program product stored on a computer-readable storage medium that, when executed, performs a method for classifying material for sorting, comprising: receiving image data of each piece of a heterogeneous mixture of materials moving in a stream past a vision system camera, the materials of the heterogeneous mixture of materials having a variety of different shapes including one or more different closed geometric shapes; classifying said material having a designated closed geometric shape into a first classification designated as a numismatic coin; classifying the materials that do not have the specified closed geometric shape into a second classification; sending classification information to an automated sorting device such that the automated sorting device can separate the materials classified in the first classification from the materials classified in the second classification, and wherein the one or more different closed geometric shapes are selected from the group consisting of circles and polygons; A computer program product comprising: [Additional Note 18] The computer program product of claim 17, wherein a material having a circular shape and having holes formed therein is classified into the second category. [Additional Note 19] The computer program product of claim 17, wherein the material of the electrical box knockout falls into the second category. [Additional Note 20] 18. The computer program product of claim 17, wherein the heterogeneous mixture of materials includes scrap from end-of-life automobiles. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows a schematic diagram of a sorting system configured in accordance with an embodiment of the present disclosure. [Figure 2] 1 shows a flowchart of the operation of a sorting device configured in accordance with an embodiment of the present disclosure. [Figure 3A] 1 shows visual images of various exemplary numismatic coins. [Figure 3B] 1 shows a visual image of an exemplary numismatic coin mixed with other scrap pieces. [Figure 3C] 1 shows visual images of various exemplary parts of a gemstone. [Figure 3D] 1 shows a visual image of an exemplary piece of jewelry mixed with other scrap pieces. [Figure 4] FIG. 1 shows a flow chart diagram configured in accordance with an embodiment of the present disclosure. [Figure 5] 1 illustrates a block diagram of a data processing system configured in accordance with an embodiment of the present disclosure. [Figure 6] 1 illustrates a flowchart of an exemplary configuration of a machine learning system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Various detailed embodiments of the present disclosure are disclosed herein. However, it should be understood that the disclosed embodiments are merely exemplary of the disclosure, which may be embodied in various alternative forms. The figures are not necessarily to scale, and some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for teaching those skilled in the art how to use various embodiments of the present disclosure.

[0011] Embodiments of the present disclosure utilize the machine learning-based vision system described herein to effectively recycle certain valuable scrap pieces (e.g., monetary coins, jewelry, PCBs, copper, brass, etc.) from shredded scrap (e.g., automobile scrap).

[0012] As used herein, "material" may include any physical item, including, but not limited to, scrap pieces. Classes or types of materials may include (ferrous and non-ferrous) metals, alloys, numismatic coins, jewelry (e.g., rings, earrings, necklaces, bracelets, etc.), gold, silver, buttons, electrical box knockouts, washers, plastics (including, but not limited to, PCBs, HDPE, UHMWPE, and various colored plastics), rubber, foam, glass (including, but not limited to, borosilicate or soda-lime glass and various colored glasses), ceramics, paper, cardboard, Teflon, PE, bundled wire, insulation-covered wire, rare earth elements, etc. As used herein, the terms "scrap" and "scrap pieces" refer to pieces of material in a solid state. In this disclosure, the terms "scrap," "scrap pieces," "material," and "pieces of material" may be used interchangeably.

[0013] As used herein, a heterogeneous mixture of materials means a collection of individual materials of different classes or types. As used herein, a homogeneous set of materials means a collection of individual materials of the same or substantially similar classes or types.

[0014] As defined in the non-ferrous scrap guidelines promulgated by the Institute of Scrap Recycling Industries, Inc., the term "Zorba" is a collective term for shredded non-ferrous metals, including, but not limited to, those derived from end-of-life vehicles (ELVs) or waste electronic and electrical equipment (WEEE). The Institute of Scrap Recycling Industries, Inc. (ISRI) established the Zorba specifications in the United States. In Zorba, each scrap piece may be composed of a combination of non-ferrous metals (e.g., aluminum, copper, lead, magnesium, stainless steel, nickel, tin, and zinc, in elemental or alloyed (solid) form). Additionally, the term "Twitch" refers to fragmented aluminum scrap. Twitch may be produced by a float process, in which heavier metal scrap sinks while the aluminum scrap floats (e.g., some processes may incorporate sand to change the density of the water in which the scrap is immersed).

[0015] As used herein, the terms “identify” and “classify,” as well as “identification” and “classification,” may be used interchangeably. For example, in certain embodiments of the present disclosure, a vision system (as described further herein) may be configured to collect any type of information (e.g., using a machine learning system) available within the sorting system to selectively classify materials (e.g., scrap pieces) as a function of a (user-defined) set of one or more physical characteristics, including, but not limited to, the material's color, size, shape, texture, appearance, uniformity, hue, and / or manufacturing type.

[0016] It should be noted that at least some of the material being sorted may have irregular sizes and shapes (see, e.g., FIGS. 3B and 3D). For example, such material (e.g., a zorba and / or twitch) may have been through some type of shredding mechanism that shreds the material into irregularly shaped and sized pieces (producing scrap pieces) that can be fed into a conveyor system.

[0017] Embodiments of the present disclosure are described herein as sorting materials (e.g., scrap pieces) into separate groups by physically depositing (e.g., discharging) the materials (e.g., scrap pieces) into separate receptacles or containers as a function of user-defined classifications. By way of example, in certain embodiments of the present disclosure, materials (e.g., scrap pieces) can be sorted into separate containers to separate scrap pieces of designated value from other scrap materials. Such scrap pieces of designated value (by a user of the system 100) can be monetary coins, jewelry (e.g., rings, earrings, necklaces, bracelets, etc.), precious metals (e.g., gold, silver, platinum, copper, brass, etc.), or PCBs (which can include valuable metals, e.g., gold, silver, copper).

[0018] FIG. 1 illustrates an example of an automated material sorting system 100 configured in accordance with various embodiments of the present disclosure to sort materials automatically (i.e., without requiring manual human intervention). While embodiments of the present disclosure are described below as sorting scrap pieces, such embodiments are applicable to sorting any type of material. A conveyor system 103 is embodied to transport one or more streams of individual scrap pieces 101 through the sorting system 100, where each individual scrap piece 101 is tracked, classified, and sorted into predetermined desired groups. Such a conveyor system 103 may be implemented using one or more conveyor belts along which the scrap pieces 101 typically move at a predetermined constant speed. However, certain embodiments of the present disclosure may also be embodied with other types of conveyor systems, including vibratory or mechanical conveyors, or systems in which the scrap pieces free-fall past various components of the sorting system. Hereinafter, the conveyor system 103 will be referred to simply as the conveyor belt 103.

[0019] 1 depicts a single stream of scrap pieces 101 on conveyor belt 103, embodiments of the present disclosure may be implemented such that multiple streams of such scrap pieces pass through various components of sorting system 100 in parallel with one another. Or, such that a collection of scrap pieces randomly deposited on conveyor belt 103 is passed by various components of sorting system 100. Thus, certain embodiments of the present disclosure may simultaneously track, classify, and sort multiple such parallel-moving streams of scrap pieces, or scrap pieces randomly deposited on a conveyor belt. According to embodiments of the present disclosure, singulation of scrap pieces 101 is not required for the vision system that tracks, classifies, and sorts the scrap pieces.

[0020] According to certain embodiments of the present disclosure, some suitable feeder mechanism may be utilized to feed the scrap pieces 101 to the conveyor belt 103, which may then transport the scrap pieces 101 through various components within the sortation system 100. Within certain embodiments of the present disclosure, the conveyor belt 103 is operated to move at a predetermined speed by a conveyor belt motor 104. This predetermined speed may be programmable and / or adjustable by an operator in any known manner. Monitoring of the predetermined speed of the conveyor belt 103 may alternatively be performed using a belt speed detector 105. Within certain embodiments of the present disclosure, control of the conveyor belt motor 104 and / or belt speed detector 105 may be performed by an automated control system 108. Such an automated control system 108 may be operated under the control of a computer system 107, and / or functionality for performing the automated control may be embodied in software within the computer system 107.

[0021] The conveyor belt 103 may be a conventional endless belt conveyor that uses a conventional drive motor 104 suitable for moving the conveyor belt 103 at a predetermined speed. A belt speed detector 105, which may be a conventional encoder, may be coupled to the conveyor belt 103 and the automated control system 108 to provide information corresponding to the operation (e.g., speed) of the conveyor belt 103. Thus, as described further herein, by utilizing control over the conveyor belt drive motor 104 and / or the automated control system 108 (or alternatively including the belt speed detector 105), each of the scrap pieces 101 moving on the conveyor belt 103 may be identified and tracked by location and time (relative to various components of the system 100) so that various components of the sorting system 100 may be activated / deactivated when each scrap piece 101 passes within their vicinity. As a result, the automated control system 108 may track the position of each of the scrap pieces 101 as they move along the conveyor belt 103.

[0022] According to certain embodiments of the present disclosure, after the scrap pieces 101 are received by the conveyor belt 103, a tumbler and / or vibrator (not shown) may be utilized to separate individual scrap pieces from the collection of scrap pieces. According to another embodiment of the present disclosure, the scrap pieces may be arranged into one or more singulated (i.e., single file) streams, which may be performed by an optional active or passive singulator 106. As previously mentioned, the incorporation or use of a singulator is not required. Instead, the conveyor system (e.g., conveyor belt 103) may simply transport a collection of scrap pieces that are randomly placed on the conveyor belt 103.

[0023] 1 , embodiments of the present disclosure may utilize a vision or optical recognition system 110 as a means to begin tracking each of the scrap pieces 101 as they move on the conveyor belt 103. The vision system 110 may utilize one or more still or live action cameras 109 (which may include one or more three-dimensional cameras) to record the position (i.e., location and time) of each of the scrap pieces 101 on the moving conveyor belt 103. The vision system 110 may be further configured to perform a particular type of identification (e.g., classification) of all or some of the scrap pieces 101. For example, such a vision system 110 may be utilized to obtain information about each of the scrap pieces 101. For example, the vision system 110 can be configured (e.g., using a machine learning system) to collect any type of information available within the system 100 and selectively sort the scrap pieces 101 as a function of a (user-defined) set of one or more physical characteristics, including, but not limited to, the color, size, shape, texture, appearance, uniformity, composition, and / or manufacturing type of the scrap pieces. The vision system 110 captures images of each of the scrap pieces 101, for example, by using an optical sensor such as those utilized in typical digital cameras and video equipment. Such images captured by the optical sensor can be stored in a memory device as image data. According to embodiments of the present disclosure, such image data represents images captured within wavelengths of light (i.e., wavelengths of light observed by the typical human eye). However, alternative embodiments of the present disclosure can utilize optical sensors configured to capture images of materials at wavelengths of light outside the visual wavelengths of the typical human eye.

[0024] Additionally, such a vision system 110 may be configured to identify which scrap pieces 101 are not of the type to be sorted by the sorting system 100 (e.g., scrap pieces classified as other than certain valuable scrap pieces) and send a signal to reject such scrap pieces. Such identified scrap pieces 101 may be ejected utilizing one of the mechanisms described herein for physically moving classified scrap pieces to individual bins.

[0025] Referring now to FIG. 2, a system and process 200 for actuating each of the automated sorting devices (e.g., sorting devices 126, 127, 128, 129) to discharge classified scrap pieces into sorting bins is shown. Such a system and process 200 may be embodied within the automated control system 108 described above with respect to FIG. 1, or within an overall computer system (e.g., computer system 107) that controls the sorting system. At process block 201, a signal is received from the automated control system 108 that a designated, tracked scrap piece is in position for sorting. At process block 202, a determination is made whether the timing associated with this signal is equal to the current time. The system and process 200 determines whether the timing associated with the classified scrap piece corresponds to the expected time that the classified scrap piece will pass near the particular sorting device (e.g., air jet, pneumatic plunger, paintbrush-type plunger, etc.) associated with the classification associated with the classified scrap piece. If the timing signals do not correspond, a determination is made at process block 203 as to whether the signal is greater than the current time. If yes, the system may return an error signal 204. In such a case, the system may not be able to discharge the pieces into the appropriate bin. If the system and process 200 determine that a classified scrap piece is passing near a sorting device associated with that classification, the sorting device is activated at process block 205 to discharge the classified scrap piece into the sorting bin associated with that classification. This may be accomplished by activating a pneumatic plunger, a paintbrush-type plunger, an air jet, or the like. At process block 206, the selected sorting device is stopped.

[0026] As previously mentioned, the sorting device may include any known mechanism for redirecting selected scrap pieces to desired locations, including, but not limited to, ejecting scrap pieces from a conveyor belt system into multiple sorting bins. For example, the sorting device may utilize air jets, each assigned to one or more classifications. When one of the air jets (e.g., 127) receives a signal from the automatic control system 108, the air jet emits a stream of air to eject scrap pieces 101 from the conveyor belt 103 into its corresponding sorting bin (e.g., 137). A high-speed air valve (e.g., available from Mac Industries) may be used to supply the air jet with an appropriate air pressure configured to eject scrap pieces 101 from the conveyor belt 103, for example.

[0027] Although the example shown in FIG. 1 uses air jets to eject the scrap pieces, other mechanisms can be used to eject the scrap pieces, such as robotically removing the scrap pieces from the conveyor belt, pushing the scrap pieces off the conveyor belt (e.g., with a paintbrush-type plunger), creating an opening in the conveyor belt (e.g., a trap door) through which the scrap pieces can fall, using one or more air jets to separate the scrap pieces into separate containers as they fall off the end of the conveyor belt, or using a robotic arm to pick up designated scrap pieces from the conveyor belt 103.

[0028] In addition to the N sorting bins 136, 137, 138, 139 into which scrap pieces 101 are discharged, system 100 may also include a receptacle or container 140 that accepts scrap pieces 101 that are not discharged from conveyor belt 103 into one of the aforementioned sorting bins 136, 137, 138, 139. For example, if the classification of a scrap piece 101 cannot be determined (or if the sorting device simply fails to properly discharge the piece), the scrap piece 101 may not be discharged from conveyor belt 103 into one of the N sorting bins 136, 137, 138, 139. Thus, container 140 may serve as a default container into which unclassified scrap pieces are dumped. Alternatively, container 140 may be used to receive one or more classifications of scrap pieces that are not intentionally assigned to any of the N sorting bins 136, 137, 138, 139. For example, according to an embodiment of the present disclosure, scrap pieces that are not classified as specific valuable scrap pieces may pass through to receptacle 140 .

[0029] According to certain embodiments of the present disclosure, one or more sets of air jets may be configured to direct scrap pieces classified as designated valuable scrap pieces into a first receptacle as they fall off the end of conveyor belt 103, while scrap pieces not classified as designated valuable scrap pieces are simply allowed to fall off the end of conveyor belt 103 into a separate second receptacle (e.g., container 140). Alternatively, the reverse may be performed, where scrap pieces classified as designated valuable scrap pieces are simply allowed to fall off the end of conveyor belt 103.

[0030] According to certain embodiments of the present disclosure, monetary coins may be sorted separately based on their different denominations and therefore sorted into separate containers accordingly.

[0031] Depending on the variety of scrap piece sorting required, multiple classifications (e.g., coins of specific different denominations) can be mapped to a single sorting device and associated sorting bins. In other words, there need not be a one-to-one correlation between classifications and sorting bins. For example, a user may desire to sort a particular class or type of material into the same sorting bin (e.g., one or more different denominations of monetary coins, or both monetary coins and copper and / or brass). To achieve such sorting, if scrap pieces 101 are sorted to fall into a predetermined classification group (e.g., one or more different denominations of monetary coins, or both monetary coins and copper and / or brass), the same sorting device can be operated to sort them into the same sorting bin. Such combinatorial sorting can be applied to generate any desired combination of sorted scrap pieces. To generate such desired combinations, the classification mapping may be programmed by the user (e.g., using a sorting algorithm operated by computer system 107 (see, e.g., FIG. 4)). Additionally, the classification of scrap pieces is user definable and is not limited to specific known classifications of scrap pieces. As non-limiting examples of the foregoing, the machine learning system of the present disclosure may be configured to separately classify two or more denominations of monetary coins for sorting into the same container (e.g., one or more of containers 136, 137, 138, 139), or may be configured to classify a particular denomination (e.g., pennies) into the same container as scrap pieces not classified as monetary coins.

[0032] In another non-limiting example of the foregoing, the machine learning system of the present disclosure may be configured to classify and separate both monetary coins and other classes or types of valuables into a common container. The other classes or types of valuables may be jewelry (e.g., rings, earrings, bracelet components, necklace components, etc., as shown in FIG. 3C ), specific classes or types of metal pieces (e.g., gold, silver, copper, brass, etc.), and / or scrap pieces identified by the machine learning system as containing specific metals (e.g., PCBs containing copper, gold, or silver). Such scrap pieces collected in the common container may then be passed through system 100 again (or such scrap pieces may be transported to a second similar system such as system 100) to separate the collected valuable scrap pieces (e.g., between monetary coins and copper and / or brass).

[0033] Conveyor system 103 may include a carousel (not shown) that returns unsorted scrap pieces (or scrap pieces of two or more classes or types of material for resorting) to the beginning of sorting system 100 for re-passing through system 100. System 100 may also specifically track each scrap piece 101 as it moves through conveyor system 103, allowing certain sorting devices (e.g., sorting device 129) to eject scrap pieces 101 that system 100 was unable to classify (e.g., monetary coins, jewelry, PCBs, jewelry, etc.) after a predetermined number of cycles through sorting system 100.

[0034] Within certain embodiments of the present disclosure, the conveyor belt 103 may be divided into multiple belts configured in series, such as two belts, where a first belt transports scrap pieces past a vision system and a second belt transports scrap pieces from the vision system to a sorting device. Furthermore, such a second conveyor belt may be at a lower height than the first conveyor belt, such that scrap pieces fall from the first belt onto the second belt.

[0035] As previously mentioned, embodiments of the present disclosure may implement one or more vision systems (e.g., vision system 110) to identify, track, and / or sort scrap pieces. Such a vision system is comprised of one or more devices for capturing or acquiring images of scrap pieces passing on a conveyor system. The devices may be configured to capture or acquire any desired range of wavelengths reflected by the scrap pieces, including, but not limited to, visible, infrared (IR), and ultraviolet (UV) light. For example, the vision system may be comprised of one or more cameras (still and / or video cameras that can be configured to capture two-dimensional, three-dimensional, and / or holographic images) positioned near (e.g., on) the conveyor system such that visual images of the scrap pieces passing through the conveyor system are captured by the vision system.

[0036] Regardless of the type of image captured of the scrap pieces, the image may then be transmitted to a computer system (e.g., computer system 107) and processed by a machine learning system to subsequently identify and / or classify each scrap piece in order to sort the scrap pieces in a desired manner. Such a machine learning system may implement one or more well-known machine learning algorithms, including those that implement the following: Neural networks (e.g., artificial neural networks, deep neural networks, convolutional neural networks, recurrent neural networks, autoencoders, reinforcement learning), fuzzy logic, artificial intelligence (AI), deep learning algorithms, deep structured learning hierarchical learning algorithms, support vector machines (SVMs) (e.g., linear SVM, nonlinear SVM, SVM regression), decision tree learning (e.g., classification and regression trees (CART)), ensemble methods (e.g., ensemble learning, random forests, bagging & pasting, patching & subspace, boosting, stacking), dimensionality reduction (e.g., projection, manifold learning, principal component analysis), and / or deep machine algorithms as described and published at deeplearning.net (including all software, publications, and hyperlinks to available software referenced within this website), which are incorporated herein by reference.Non-limiting examples of publicly available machine learning algorithms, software, and libraries that can be used within embodiments of the present disclosure include Python, OpenCV, Inception, Theano, Torch, PyTorch, Pylearn2, Numpy, Blocks, TensorFlow, MXNet, Caffe, Lasagne, Keras, chainer, Matlab Deep Learning®, CNTK, MatConvNet (a toolbox for MATLAB® that implements convolutional neural networks for computer vision applications), DeepLearnToolbox (Matlab® Toolbox for Deep Learning (Rasmus Berg Palm)), BigDL, Cuda-Convnet (a fast C++ / CUDA implementation of convolutional (or more generally feedforward) neural networks), Deep Belief Networks, RNNLM, RNNLIB-RNNLIB, matrbm, deeplearning4j, Eblearn.lsh, deepmat, MShadow, Matplotlib, SciPy, Includes CXXNET, Nengo-Nengo, Eblearn, cudamat, Gnumpy, 3-way factored RBM and mcRBM, mPoT (Python code using CUDAMat and Gnumpy to train a model on natural images), ConvNet, Elektronn, OpenNN, NeuralDesigner, Theano Generalized Hebbian Learning, Apache Singa, Lightnet, and SimpleDNN.

[0037] Machine learning often occurs in two stages, or phases. For example, initially, training occurs offline, and sorting system 100 is not utilized to perform actual sorting of scrap pieces. According to certain embodiments of the present disclosure, a portion of system 100 is utilized to train the machine learning system where one or more homogenous sets of scrap pieces (i.e., representative sets of one or more denominations of numismatic coins (e.g., see FIG. 3A ), rings, bracelets, necklaces, and / or earrings, etc., representative scrap pieces of PCBs, or representative scrap pieces of a particular type of precious metal (e.g., gold, silver, copper, brass)) are passed through vision system 110 using conveyor system 103 (each set of homogenous scrap pieces may be collected in a common unsorted receptacle (e.g., receptacle 140)). Alternatively, training can be performed at a separate location, remote from system 100, and may involve using other mechanisms to collect images of homogenous sets of particular valuable scrap pieces.

[0038] It should be noted that, according to certain embodiments of the present disclosure, a set of homogeneous numismatic coins may be a collection of numismatic coins of the same denomination (and thus having the same shape, size, color, hue, etc.) or may be a collection of numismatic coins of different denominations (and thus having different shapes, sizes, colors, hues, etc.), but sharing at least one physical characteristic (shape, etc., e.g., circular, polygonal) that is the same or substantially the same. Furthermore, according to certain embodiments of the present disclosure, because most numismatic coins are approximately circular, this may be a particular physical characteristic utilized by a machine learning system to classify and sort materials. Because there are some foreign coins that are not circular (e.g., polygonal), such particular non-circular shapes (but substantially polygonal (e.g., octagonal) shapes) may also be utilized by a machine learning system to classify and sort materials. For purposes of describing various embodiments of the present disclosure, it will be understood that most numismatic coins have closed geometric shapes (e.g., circular, polygonal).

[0039] During this training phase, a machine learning algorithm extracts features from the captured images using image processing techniques well known in the art. Non-limiting examples of training algorithms include, but are not limited to, linear regression, steepest descent, feedforward, polynomial regression, learning curve, regularized learning model, and logistic regression. During this training phase, the machine learning algorithm can be configured to learn relationships between designated valuable scrap pieces (e.g., monetary coins (which may include different denominations), rings, bracelets, necklaces, earrings, PCBs, etc.) and their features (e.g., color, texture, hue, shape (e.g., round, polygonal), brightness, etc., as captured by the images) to form a knowledge base for classifying heterogeneous mixtures of scrap pieces received by sorting system 100 to separate monetary coins from heterogeneous mixtures of scrap pieces. According to certain embodiments of the present disclosure, such a knowledge base may include the requirement that scrap pieces to be classified as monetary coins have a substantially circular and / or polygonal shape (e.g., coins are within a predetermined threshold of being substantially circular and / or polygonal because they may be somewhat damaged in a vehicle or by an automobile shredder, as shown by some of the coins in FIG. 3A). Such a knowledge base may include the rejection of circular scrap pieces with holes formed therein, in order to not classify metal washers as monetary coins. Such a knowledge base may further include the rejection of circular scrap pieces having a particular color or hue (e.g., in order to not classify U.S. pennies as other monetary coins).

[0040] Such a knowledge base may include one or more libraries, each library containing parameters for utilization by the vision system 110 in classifying and sorting scrap pieces during the second stage or phase. For example, a particular library may contain parameters configured during the training stage to recognize and classify coins of a particular denomination. According to certain embodiments of the present disclosure, such libraries may be input into the vision system, and a user of the system 100 may then adjust certain ones of the parameters to tune the operation of the system 100 (e.g., adjusting the effectiveness of a threshold for how well the vision system recognizes a particular denomination of monetary coins from a heterogeneous mixture of materials (e.g., see FIG. 3B )).

[0041] For example, FIG. 3A shows a captured or acquired image of a homogenous set of exemplary monetary coins that may be used during the aforementioned training phase. During the training phase, a control sample of such monetary coins (e.g., a homogenous set of exemplary monetary coins of one or more specified denominations) is delivered through the vision system (e.g., by conveyor system 103), and the machine learning system detects, extracts, and learns what features visually represent such exemplary monetary coins. In other words, images of monetary coins such as those shown in FIG. 3A can be first passed through such a training phase to allow the machine learning system to “learn” how to detect, recognize, and classify monetary coins among a heterogeneous mixture of scrap pieces (e.g., as shown in FIGS. 3B and 3D). This creates a library of parameters specific to the specified monetary coins.

[0042] It is important to note that the detected / extracted features do not necessarily have to be simple color, brightness, or circular or polygonal shape. They can be abstract formulations that can only be expressed mathematically, or not at all. Nevertheless, the machine learning system analyzes all data to look for patterns that can classify control samples (e.g., actual monetary coins) during the training phase. The machine learning system may take subsections of captured images of scrap pieces and try to find correlations between predefined classifications (e.g., one or more different monetary coin denominations).

[0043] According to certain embodiments of the present disclosure, a machine learning system can be configured to classify scrap pieces that are approximately, but not exactly, circular as monetary coins. For example, when monetary coins contained in a scrap yard vehicle are processed (e.g., shredded), they may be damaged (e.g., slightly bent or notched). FIG. 3A shows examples of some such damaged coins. The machine learning system can adjust tolerance parameters to classify such scrap pieces as monetary coins. For example, a scrap piece may be classified as a monetary coin if it is not perfectly circular or does not have a completely closed circular shape, but its overall size (e.g., diameter) and / or color, hue, texture, etc., match a particular denomination of monetary coin (e.g., U.S. quarter, nickel, dime, etc.). The machine learning system can also be trained to classify scrap pieces as monetary coins by including representative samples of damaged (e.g., notched, bent, etc.) coins in the aforementioned control sample (see FIG. 3A).

[0044] Additionally, because there are non-U.S. monetary coins that are not circular in shape and may have other polygonal shapes, the machine learning system of the present disclosure may be configured to classify such objects in the scrap piece stream as monetary coins.

[0045] Because some scrap being sorted may be manufactured from materials that include metal electrical boxes, the scrap pieces may contain circular knockouts that resemble monetary coins. However, a machine learning system configured in accordance with embodiments of the present disclosure may be configured to not classify such knockouts as monetary coins. This can be achieved by running a homogenous set of such knockouts through the machine learning system during a training phase. The machine learning system can "learn" not to classify knockouts as monetary coins based on how the knockouts appear different from monetary coins (e.g., texture, color, lack of a pattern imprinted on their surface, etc.).

[0046] According to some embodiments of the present disclosure, a machine learning system may be configured to not classify as a monetary coin any circular scrap piece that does not have a diameter equivalent to one or more specific monetary coins, including, but not limited to, scrap pieces with diameters greater than and / or less than a predetermined diameter (e.g., U.S. quarters, nickels, dimes, etc.). This would prevent, for example, a clothing button from being classified as a monetary coin. Such diameter specifications can be utilized to sort monetary coins by denomination.

[0047] According to certain embodiments of the present disclosure, training a machine learning system to identify numismatic coins for sorting using a homogenous set of exemplary coins (e.g., see FIG. 3A ) enables the machine learning system 100 to sort specific numismatic coins from a heterogeneous mixture of scrap pieces (e.g., see FIG. 3B ).

[0048] According to certain embodiments of the present disclosure, to enable the machine learning system 100 to classify specific gemstone scrap pieces from a heterogeneous mixture of scrap pieces (e.g., see FIG. 3D ), the machine learning system can be trained to identify specific types of gemstones by running exemplary samples of gemstone pieces (e.g., see FIG. 3C ) through the previously disclosed machine learning system. As previously disclosed, the machine learning system can identify and classify gemstone scrap pieces from a heterogeneous mixture of such scrap pieces by learning specific physical characteristics of such specific gemstone scrap pieces. FIG. 3D provides a non-limiting example of how such gemstone scrap pieces can be visually distinguished from other scrap pieces.

[0049] Although not shown in the figures, the exemplary pieces of PCB may be run through the machine learning system as a homogenous set to enable the machine learning system 100 to identify and classify such PCB scrap pieces from a heterogeneous mixture of scrap pieces. For example, the machine learning system might do this by looking for scrap pieces that are green or look like green plastic sheeting.

[0050] 6 depicts at an abstract level examples of various possible embodiments of the present disclosure. A machine learning algorithm may essentially embody one or more aspects of the system and process 600, but not necessarily exactly as outlined in the flowchart of FIG.

[0051] In block 601, the vision system 110 acquires images of scrap pieces 101 as described herein. Block 602 represents that a machine learning system may be configured to identify scrap pieces that resemble certain valuable scrap pieces (e.g., whether a monetary coin is round or polygonal in shape). Other physical characteristics (e.g., color, shade, tone, texture, stamped features, diameter, etc.) can be utilized to identify specific features (e.g., coin-related) in the scrap pieces 101.

[0052] Optional block 603 abstractly represents how a machine learning system may be configured to exclude scrap pieces 101 that resemble (i.e., have similar physical characteristics) electrical box knockouts from the monetary coin classification.

[0053] Optional block 604 represents how the learning machine of the abstract system can be further configured to not classify scrap pieces that are not of a desired denomination (e.g., having the color of a U.S. penny, smaller than a dime, larger than a quarter) as monetary coins. Block 604 abstractly represents how the machine learning system separately sorts coins of various denominations.

[0054] Once the machine learning algorithms are established and the machine learning system has fully learned the differences between material classifications, a library of different classes or types of materials (e.g., coins of one or more denominations, rings, bracelets, necklaces, earrings, PCBs, etc.) is implemented in the material classification system (e.g., system 100) and used to identify and / or classify and sort specific scrap pieces from a heterogeneous mixture of scrap pieces.

[0055] FIG. 4 depicts a flowchart diagram illustrating an exemplary embodiment of a process 400 for sorting scrap pieces using a vision system, according to certain embodiments of the present disclosure. Aspects of process 400 may be configured to operate within any of the embodiments of the present disclosure described herein, including sorting system 100 of FIG. 1. Operations of process 400 may be performed by hardware and / or software, including within a computer system (e.g., computer system 3400 of FIG. 5) that controls the sorting system (e.g., computer system 107 and / or vision system 110 of FIG. 1). In optional process block 401, the scrap pieces may be passed through some type of well-known sieve (not shown), which may be configured to allow scrap pieces smaller than a predetermined size to pass through the sieve. For example, slots formed in the sieve may be configured to allow objects similar in size to monetary coins to pass through. However, any of the devices described herein or another sorting system may be utilized to initially separate the smaller scrap pieces from the larger scrap pieces.

[0056] At process block 402, scrap pieces may be deposited onto a conveyor belt. FIG. 3B shows a digital photograph of an exemplary heterogeneous collection of such scrap pieces, including various numismatic coins, deposited on a conveyor belt. FIG. 3A shows a digital photograph of an exemplary heterogeneous collection of scrap pieces, including various jewelry scrap pieces, deposited on a conveyor belt. In a non-limiting example, a sieve may be positioned to deposit passing scrap pieces onto the belt conveyor. For example, with reference to FIG. 1, such a sieve may be positioned between the ramp or chute 102 and the conveyor belt 103. The position of each scrap piece 101 on the conveyor belt 103 is detected to track each scrap piece as it passes through the sorting system. This is performed by the vision system 110 (e.g., by distinguishing the scrap pieces from the underlying conveyor belt material while communicating with a conveyor belt speed detector (e.g., belt speed detector 105)), and this information is collected and monitored by the automation control system 108. Alternatively, a linear sheet laser beam can be used to identify the location of the scrap pieces (or any system capable of producing a light source (including but not limited to visible light, UV, VIS, and IR) and with detectors that can be used to identify the location of parts). At process block 403, one or more images of the scrap pieces are captured / obtained as the scrap pieces move near the vision system 110. At process block 404, the machine learning system can perform image pre-processing, as previously disclosed, which can be used to detect or identify (extract) each of the scrap pieces from the background (e.g., the conveyor belt 103). In other words, image pre-processing can be used to identify the difference between the scrap pieces and the background. Well-known image processing techniques, such as dilation, thresholding, and contouring, can be used to identify the scrap pieces as distinct from the background. At process block 405, image segmentation can be performed. For example, one or more images captured by the vision system's camera may include one or more images of the scrap pieces.Additionally, a particular scrap piece may be located at a seam in the conveyor belt when its image is captured. Therefore, in such cases, it may be desirable to separate the image of the individual scrap piece from the image background. In the exemplary technique of process block 405, the first step is to apply high contrast to the image. In this way, background pixels are reduced to substantially all black pixels, and at least some pixels associated with the scrap piece are brightened to substantially all white pixels. Next, the image pixels of the white scrap piece are expanded to cover the entire size of the scrap piece. After this step, the location of the scrap piece becomes a high-contrast image of all white pixels on a black background. Next, a contour algorithm is utilized to detect the boundaries of the scrap piece. The boundary information is saved, and the boundary location is transferred to the original image. Segmentation is performed on the original image for areas larger than the previously defined boundaries. In this way, each scrap piece is identified and separated from the background. In process block 406, the size and shape of each scrap piece can be determined.

[0057] Image post-processing can be performed at process block 407. Image post-processing may include resizing the image for use with a neural network. This may also include modifying certain image properties (e.g., increasing image contrast, changing the image background, applying a filter) in a manner that enhances the machine learning system's ability to classify scrap pieces. Following image post-processing, normalization of various images can be performed at process block 408 so that images of various different scrap pieces can be more easily compared to each other. At process block 409, data representing each image may be resized. Image resizing may be necessary under certain circumstances to match the data input requirements of certain machine learning systems, such as neural networks. Neural networks require image sizes (e.g., 225x255 pixels or 299x299 pixels) that are much smaller than the size of images captured by a typical digital camera. Additionally, smaller image sizes require less processing time for classification. Therefore, reducing image size ultimately increases the throughput and value of sorting systems.

[0058] In process blocks 410 and 411, each scrap piece is identified / classified based on the detected features. For example, process block 410 can be configured with a neural network employing one or more machine learning algorithms that compare the extracted features (e.g., circular / polygonal, non-porous, color, etc.) with those stored in a knowledge base generated during a training phase and, based on such comparison, assign each scrap piece a classification that best matches it. The machine learning algorithms can process the captured images hierarchically using automatically trained filters. The filter responses are then successfully incorporated into the next level of the algorithm until a probability is obtained in the final step. In process block 411, these probabilities are calculated using a function N(N >1) Each of the classifications can be used to determine which of N sorting bins a respective scrap piece should be sorted into. For example, each of the N classifications can be assigned to a respective sorting bin, and the scrap piece under consideration is sorted into the bin corresponding to the classification that returns the highest probability greater than a predetermined threshold. Within embodiments of the present disclosure, such predetermined threshold can be pre-set by a user. If none of the probabilities are greater than the predetermined threshold (e.g., the scrap piece is not classified as a monetary coin), the particular scrap piece can be sorted into a miss bin (e.g., sorting bin 140).

[0059] At process block 412, a sorting device corresponding to one or more classifications of scrap pieces is activated (see, e.g., FIG. 2). Between the time an image of the scrap piece 101 is captured by the vision system 110 and the time the sorting device is activated, the scrap piece 101 moves from near the vision system 110 to a position downstream of the conveyor belt 103 at the conveying speed of the conveyor belt 103. In an embodiment of the present disclosure, the operation of the sorting devices (e.g., 126, 127, 128, 129) is timed such that when the scrap piece 101 passes a sorting device mapped to the scrap piece's classification, the sorting device is activated and the scrap piece is sent to the associated sorting bin (e.g., 136, 137, 138, 139). In an embodiment of the present disclosure, activation of the sorting device can be timed by an automatic control system in communication with the belt speed detector 105, which detects the time that a scrap piece passes in front of the sorting device and sends a signal to activate the sorting device. At process block 413, the sorting bin corresponding to the activated sorting device receives the indicated scrap piece.

[0060] According to certain embodiments of the present disclosure, at least some of the systems 100 may be linked together in series to perform multiple iterations or layers of sorting. For example, when two or more systems 100 are so linked, the conveyor system may be implemented with a single conveyor belt or multiple conveyor belts that transport scrap pieces of a first set of heterogeneous material through a first vision system configured to sort scrap pieces of a first set of heterogeneous material into a first set of one or more receptacles (e.g., sorting bins 136, 137, 138, 139) by a sorter (e.g., first automated control system 108 and associated sorting device(s) 126, 127, 128, 129), and then transport the scrap pieces through a second vision system configured to sort scrap pieces of a second set of heterogeneous material into a second set of one or more sorting bins by a second sorter.

[0061] Such a series of systems 100 may include any number of such systems linked together in this manner. According to certain embodiments of the present disclosure, each successive vision system may be configured to sort different materials than the previous vision system (e.g., first sorting coins and copper / brass from scrap, then sorting coins and copper / brass pieces).

[0062] As described herein, embodiments of the present disclosure can be implemented to perform various functions described for identifying, tracking, classifying, and sorting materials, such as scrap pieces. Such functions may be implemented in hardware and / or software, such as in one or more data processing systems (e.g., data processing system 3400 of FIG. 5 ), the aforementioned computer system 107, vision system 110, and / or automation control system 108. However, the functions described herein are not limited to implementation on any particular hardware / software platform.

[0063] As will be appreciated by those skilled in the art, aspects of the present disclosure may be embodied as a system, process, method, and / or program product. Accordingly, various aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, referred to herein as a "circuit," "module," or "system." Furthermore, aspects of the present disclosure may take the form of a program product embodied in one or more computer-readable storage medium(s) having computer-readable program code embodied therein. (However, any combination of one or more computer-readable medium(s) may be utilized; the computer-readable medium(s) may be a computer-readable signal medium or a computer-readable storage medium.)

[0064] A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, biological, atomic, or semiconductor system, apparatus, controller, or device, or any suitable combination of the foregoing, where the computer-readable storage medium is not itself a transitory signal. More specific examples (a non-exhaustive list) of computer-readable storage media may include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM) (e.g., RAM 3420 in FIG. 5), a read-only memory (ROM) (e.g., ROM 3435 in FIG. 5), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device (e.g., hard drive 3431 in FIG. 5), or any suitable combination of the foregoing. As used herein, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, controller, or device. The program code embodied in a computer readable signal medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination of the foregoing.

[0065] A computer-readable signal medium may include a propagated data signal having computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium is not a computer-readable storage medium but may be any computer-readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, controller, or device.

[0066] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, processes, and program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of code, including one or more executable program instructions for performing the specified logical function(s). It should be noted that in some implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently. Or, the blocks may be executed in the reverse order, depending on the functionality involved.

[0067] Modules implemented in software for execution by various types of processors (e.g., GPU 3401, CPU 3415) can include, for example, one or more physical or logical blocks of computer instructions and can be organized, for example, as an object, procedure, or function. Executable identified modules need not be physically located together but may include different instructions stored in different locations that, when logically combined, comprise the module and achieve the module's specified purpose. In practice, a module of executable code may be a single instruction or multiple instructions and may be distributed across multiple different code segments, different programs, and multiple memory devices. Similarly, operational data (e.g., a material classification library described herein) can be identified and represented within modules herein and can be embodied in any suitable format and organized within any suitable type of data structure. The operational data may be collected as a single data set or distributed in various locations, including various storage devices. The data may provide electronic signals over a system or network.

[0068] These program instructions are provided to one or more processors and / or controllers of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus (e.g., a controller) capable of producing a machine, and the instructions, which execute via the processor (e.g., GPU 3401, CPU 3415) of the computer or other programmable data processing apparatus, create circuits or means for implementing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0069] It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems (which may include, for example, one or more graphics processing units (e.g., GPU 3401, CPU 3415)) that perform particular functions or acts, or combinations of special purpose hardware and computer instructions. For example, a module can be implemented as hardware circuits including custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, controllers, or other discrete components. A module can also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, etc.

[0070] Computer program code, i.e., instructions, for carrying out operations of aspects of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smolltalk, Python, C++, etc.; traditional procedural programming languages ​​such as the "C" programming language or similar programming languages; or any of the machine learning software disclosed herein. The program code may execute entirely on the user's computer system, partially on the user's computer system as a standalone software package, partially on the user's computer system (e.g., a computer system utilized for classification), partially on a remote computer system (e.g., a computer system utilized to train a machine learning system), or entirely on a remote computer system or server. In the latter scenario, the remote computer system may be connected to the user's computer system via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer system (e.g., via the Internet using an Internet Service Provider). As an example of the foregoing, various aspects of the present disclosure may be configured to run on one or more of computer system 107, automation control system 108, and vision system 110.

[0071] These program instructions may also be stored on a computer-readable storage medium that can direct a computer system, other programmable data processing apparatus, controller, or other device to function in a particular manner, such that the instructions stored on the computer-readable medium create an article of manufacture including instructions that implement the functions / acts specified in the flowchart and / or block diagram blocks.

[0072] The program instructions may also be loaded into a computer, other programmable data processing apparatus, controller, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to create a computer-implemented process, the instructions executing on the computer or other programmable apparatus providing a process for implementing the functions / acts specified in the flowchart and / or block diagram blocks.

[0073] A host may include one or more databases to store and provide access to data in various implementations. Those skilled in the art will also appreciate that, for security reasons, any database, system, or component of the present disclosure may include any combination of databases or components in a single location or multiple locations. Here, each database or system may include any of a variety of appropriate security features, such as firewalls, access codes, encryption, and decryption. Databases can be any type of database, such as relational, hierarchical, or object-oriented. Common database products that can be used to implement a database include IBM's DB2, database products available from Oracle Corporation, Microsoft Access from Microsoft Corporation, or other database products. Databases can be organized in any suitable manner, such as data tables and lookup tables.

[0074] Association of specific data (e.g., for each scrap piece processed by the sorting system described herein) can be achieved by any data association technique known and practiced in the art. For example, association can be performed manually or automatically. Automatic association techniques can include, for example, database lookup, database merge, GREP, AGREP, SQL, etc. The association step can be performed, for example, by a database merge function using key fields in each manufacturer and retailer data table. The key field divides the database according to high-level classes of objects defined by the key field. For example, a particular class can be designated as a key field in both a first data table and a second data table, and the two data tables can be merged based on the class data in the key field. In these embodiments, the data corresponding to each key field in the merged data tables is preferably the same. However, data tables with similar but not identical data in the key field can also be merged using, for example, AGREP.

[0075] As used herein, reference is made to "configuring" a device or a device "configured" to perform some function. It should be understood that this may involve selecting predefined logic blocks and logically associating them to provide a particular logical function, including monitoring or control functions. This may also involve programming computer software-based logic in a retrofit control device, wiring discrete hardware components, or a combination of any or all of the foregoing. A device so configured is physically designed to perform a specified function.

[0076] In the description herein, numerous specific details of examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, controllers, etc. are provided to provide a thorough understanding of embodiments of the present disclosure. However, one skilled in the art will recognize that the disclosure can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations may not be shown or described in detail to avoid obscuring aspects of the disclosure.

[0077] Referring now to FIG. 5, a block diagram illustrates a data processing (computer) system 3400 in which aspects of an embodiment of the present disclosure may be implemented. (The terms “computer,” “system,” “computer system,” and “data processing system” may be used interchangeably herein.) The computer system 107, the automation control system 108, and / or the vision system 110 may be configured similarly to the computer system 3400. The computer system 3400 may employ a local bus 3405 (e.g., a Peripheral Component Interconnect (“PCI”) local bus architecture). Any suitable bus architecture, such as Accelerated Graphics Port (“AGP”) or Industry Standard Architecture (“ISA”), may be utilized. One or more processors 3415, volatile memory 3420, and non-volatile memory 3435 may be connected to the local bus 3405 (e.g., via a PCI bridge (not shown)). An integrated memory controller and cache memory may be coupled to the one or more processors 3415. The one or more processors 3415 may include one or more central processing units and / or one or more graphics processing units and / or one or more tensor processing units. In certain embodiments of the present disclosure, one or more GPUs 3401 (e.g., GPGPUs, or general-purpose computing for graphics processing units) are implemented within the computer system 107 to operate any one or more of the disclosed machine learning systems. Additional connections to the local bus 3405 can be made via direct component interconnections or add-in boards. In the illustrated example, a communications (e.g., network (LAN)) adapter 3425, an I / O (e.g., small computer system interface ("SCSI") host bus) adapter 3430, and an expansion bus interface (not shown) are connected to the local bus 3405 by direct component connections.An audio adapter (not shown), a graphics adapter (not shown), and a display adapter 3416 (coupled to a display 3440) can be connected to the local bus 3405 (e.g., by boards inserted into add-in expansion slots).

[0078] User interface adapter 3412 may provide connections for a keyboard 3413 and mouse 3414, a modem (not shown), and additional memory (not shown). I / O adapter 3430 may provide connections for a hard disk drive 3431, a tape drive 3432, and a CD-ROM drive (not shown).

[0079] An operating system may run on one or more processors 3415 and may be used to coordinate and provide control of various components within computer system 3400. In FIG. 5, the operating system may be a commercially available operating system. An object-oriented programming system (e.g., Java, Python, etc.) may run in conjunction with the operating system and provide calls to the operating system from one or more programs (e.g., Java, Python, etc.) executing on system 3400. The operating system, instructions for the object-oriented operating system, and programs may be located on a non-volatile memory 3435 storage device, such as a hard disk drive 3431, and may be loaded into volatile memory 3420 for execution by processor 3415.

[0080] Those skilled in the art will appreciate that the hardware in Figure 5 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash ROM (or equivalent non-volatile memory) or optical disk drives, may be used in addition to or in place of the hardware shown in Figure 5. Also, any of the processes disclosed herein may be applied to a multi-processor computer system or performed by multiple such systems 3400. For example, training of vision system 110 may be performed by a first computer system 3400, while operation of vision system 110 for sorting may be performed by a second computer system 3400.

[0081] As another example, computer system 3400 may be a standalone system configured to be bootable independent of some type of network communication interface, regardless of whether computer system 3400 includes any type of network communication interface. As a further example, computer system 3400 may be an embedded controller configured with ROM and / or flash ROM that provides non-volatile memory for storing operating system files or user-generated data.

[0082] 5 and above-described examples are not meant to imply architectural limitations, and computer program forms of aspects of the present disclosure may reside on any computer-readable storage medium (i.e., floppy disk, compact disc, hard disk, tape, ROM, RAM, etc.) used by a computer system.

[0083] References throughout this specification to an "embodiment" or similar language mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the terms "in one embodiment," "in an embodiment," "embodiment," "particular embodiment," "various embodiments," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment. Furthermore, the described features, structures, aspects, and / or features of the present disclosure may be combined in any suitable manner in one or more embodiments. Correspondingly, even if features are initially described as working in a particular combination, in some cases one or more features from the described combination may be deleted from the combination, and the described combination may lead to subcombinations or variations of the subcombinations.

[0084] Benefits, advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and elements that may cause or make the benefits, advantages, or solutions more noticeable may not be construed as critical, essential, or critical features or elements in any or all of the claims. Furthermore, no element described herein is required for the practice of the present disclosure unless expressly described as essential or critical.

[0085] Those skilled in the art who have read this disclosure will recognize that changes and modifications can be made to the embodiments without departing from the scope of the disclosure. It should be understood that the specific implementations shown and described herein may be illustrative of the disclosure and its best mode, and are not intended to otherwise limit the scope of the disclosure. Other variations may be within the scope of the following claims.

[0086] While this specification contains many details, these should not be construed as limitations on the scope of the disclosure or what can be claimed, but rather as descriptions of features inherent in particular implementations of the disclosure. The headings herein may not be intended to limit the disclosure, the embodiments of the disclosure, or other matters disclosed under the heading.

[0087] As used herein, the term "or" may be intended to be inclusive, such that "A or B" includes A or B, and also includes both A and B. As used herein, the term "and / or," when used in the context of a list of components, refers to the components present either singly or in combination. Thus, for example, the phrase "A, B, C, and / or D" includes not only A, B, C, and D individually, but also any and all combinations and subcombinations of A, B, C, and D.

[0088] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" can be intended to include the plural forms as well, unless the context clearly dictates otherwise.

[0089] Corresponding structure, material, acts, and equivalents of all means- or step-plus-function elements in the following claims may be intended to include any structure, material, or act for performing the function in combination with other claimed elements that are specifically claimed.

[0090] As used herein with respect to an identified characteristic or circumstance, "substantially" and "approximately" refer to a degree of deviation small enough that it does not measurably impair the identified characteristic or circumstance. The exact degree of deviation permitted may vary depending on the particular situation.

[0091] As used herein, a plurality of items, structural elements, components, and / or materials may be presented in common lists for convenience; however, these lists should be construed as though each member of the list were individually identified as a separate and unique member. Accordingly, individual members of such lists should not be construed as de facto equivalents to other members of the same list solely based on their presentation in a common grouping, absent indication to the contrary.

[0092] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the presently disclosed subject matter belongs. Although any methods, devices, and materials similar or equivalent to those described herein can be used in the practice or testing of the presently disclosed subject matter, representative methods, devices, and materials are now described.

[0093] Unless otherwise indicated, all numbers expressing quantities of ingredients, reaction conditions, and the like used in the specification and claims are to be understood in all instances as modified by the term "about." Accordingly, unless otherwise indicated, the numerical parameters set forth in the specification and appended claims are approximations that may vary depending on the desired properties sought to be obtained by the subject matter disclosed herein. As used herein, the term "about," when referring to a value or mass, weight, time, volume, concentration, or percentage, is meant to encompass variations of ±20%, ±10%, ±5%, ±1%, ±0.5%, and ±0.1%, in some embodiments, where such variations are appropriate for carrying out the disclosed methods. [Explanation of symbols]

[0094] 100 Machine Learning Systems 101 Scrap Piece 102 shots 103 Conveyor System 104 Drive motor 105 Belt speed detector 106 Passive Singulator 107 Computer Systems 108 Automation Control Systems 110 Visual System 126, 127, 128, 129 Sorting device 136, 137, 138, 139 Sorting containers

Claims

1. capturing image data for each piece of a heterogeneous mixture of materials moving in a stream past the camera, the materials of the heterogeneous mixture of materials being a collection of different classes or types of materials; classifying the material having one or more designated classes or types of material into a first classification; classifying the materials that do not have the one or more specified classes or types of material into a second classification; separating, by an automated sorting device, the materials classified in the first category from the materials classified in the second category; Equipped with wherein the classification of the materials is performed by a neural network employing one or more machine learning algorithms that compare features detected in the image data captured from each piece of the heterogeneous mixture of materials with features stored in a knowledge base generated in a prior training phase; During the training phase, control samples that are a collection of one or more specified classes or types of materials classified into the first classification are delivered through the camera, and the one or more machine learning algorithms detect, extract, and learn features that are visually representative of the one or more specified classes or types of materials classified into the first classification.

2. 10. The method of claim 1, wherein during the training phase, the one or more machine learning algorithms learn relationships between the one or more specified classes or types of materials and their features extracted from captured image data creating a knowledge base.

3. 2. The method of claim 1, wherein materials having a circular shape and also having holes formed therein fall into said second category.

4. 10. The method of claim 1, wherein the materials classified in the first category are selected from the list consisting of at least one of: monetary coins, gemstones, precious metals, printed circuit boards, and combinations thereof.

5. The method of claim 1 , further comprising passing a bulk of material through a sieve prior to capturing the image data to generate a heterogeneous mixture of the material having a size below a predetermined size.

6. a camera configured to capture image data of each piece of a heterogeneous mixture of materials moving in a stream past the camera, the materials of the heterogeneous mixture of materials being a collection of different classes or types of materials; a circuit configured to classify the material having a specified class or type of material into a first classification; a circuit configured to classify the materials not having the specified class or type of material into a second classification; an automated sorting device configured to separate the materials classified in the first category from the materials classified in the second category; Equipped with wherein the classification of the materials is performed by a neural network employing one or more machine learning algorithms that compare features detected in the image data captured from each piece of the heterogeneous mixture of materials with features stored in a knowledge base generated in a prior training phase; During the training phase, control samples that are a collection of one or more specified classes or types of materials classified into the first classification are delivered through the camera, and the one or more machine learning algorithms detect, extract, and learn features that are visually representative of the one or more specified classes or types of materials classified into the first classification.

7. 7. The system of claim 6, wherein during the training phase, the one or more machine learning algorithms learn relationships between the one or more specified classes or types of materials and their features extracted from captured image data creating a knowledge base.

8. The system of claim 7 , further comprising a circuit configured to classify materials having a circular shape and having holes formed therein into the second classification.

9. The system of claim 6 , further comprising a sieve for separating the heterogeneous mixture of materials having a size below a predetermined size prior to capturing the image data.

10. 7. The system of claim 6, wherein the first classification comprises at least one of monetary coins, jewelry, precious metals, printed circuit boards, and combinations thereof.

11. The system of claim 6 , wherein the heterogeneous mixture of materials comprises zorba or the heterogeneous mixture of materials comprises scrap from end-of-life automobiles.

Images