Metal separation in scrap yards

An AI-driven system for metal scrap sorting improves recycling efficiency by accurately classifying and separating metal alloys, reducing pollution and emissions, and increasing the quality and quantity of recyclable materials.

JP2024543006APending Publication Date: 2024-11-19ソルテラテクノロジーズインコーポレイテッド
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2024525473
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-10-29
Filing Date
2022-10-24
Publication Date
2024-11-19

Smart Images

  • Figure 2024543006000001_ABST
    Figure 2024543006000001_ABST
Patent Text Reader

Abstract

An apparatus for classifying materials utilizing a vision system, said vision system may implement an artificial intelligence system to identify or classify each of the materials, which may then be separated from a pile in a scrap yard into separate groups, such as other piles, based on such identification or classification. The artificial intelligence system may utilize a neural network and be pre-trained to recognize and classify several types of materials.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Related Patents and Patent Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 273,535. This application is a continuation-in-part of U.S. Provisional Patent Application No. 17 / 752,669, which is a continuation-in-part of U.S. Provisional Patent Application No. 17 / 667,397, which is a continuation-in-part of U.S. Provisional Patent Application No. 17 / 495,291, which is a continuation-in-part of U.S. Provisional Patent Application No. 17 / 491,415 (issued as U.S. Patent No. 11,278,937), which is a continuation-in-part of U.S. Provisional Patent Application No. 17 / 380,928, which is a continuation-in-part of U.S. Provisional Patent Application No. 17 / 227,245, which is a continuation-in-part of U.S. Provisional Patent Application No. 16 / 939,011 (issued as U.S. Patent No. 11,278,937), ... No. 16 / 375,675 (issued as U.S. Patent No. 10,722,922), which is a continuation-in-part of U.S. Patent Application No. 15 / 963,755 (issued as U.S. Patent No. 10,710,119), which is a continuation-in-part of U.S. Patent Application No. 15 / 213,129 (issued as U.S. Patent No. 10,207,296), which claims priority to U.S. Provisional Patent Application No. 62 / 193,332, all of which are hereby incorporated by reference herein. U.S. Patent Application No. 17 / 491,415 (issued as U.S. Patent No. 11,278,937) is a continuation-in-part of U.S. Patent Application No. 16 / 852,514 (issued as U.S. Patent No. 11,260,426), which is a divisional application of U.S. Patent Application No. 16 / 358,374 (issued as U.S. Patent No. 10,625,304), which is a continuation-in-part of U.S. Patent Application No. 15 / 963,755 (issued as U.S. Patent No. 10,710,119), which claims priority to U.S. Provisional Patent Application No. 62 / 490,219, all of which are hereby incorporated by reference herein.

[0002] Government License Rights This disclosure was made with U.S. Government support under Grant No. DE-AR0000422 awarded by the U.S. Department of Energy. The U.S. Government may have certain rights in this disclosure.

[0003] FIELD OF THE DISCLOSURE This disclosure relates generally to the separation of materials, and more particularly to the sorting and / or screening of materials collected over a large area, such as a scrap metal yard. [Background technology]

[0004] This section is intended to introduce various aspects of the art that may be related to exemplary embodiments of the present disclosure. The discussion is believed to be helpful in providing a framework to facilitate a better understanding of certain aspects of the present disclosure. As such, it should be understood that this section should be read in this light, and not necessarily as admissions of prior art.

[0005] Recycling is the process of collecting, processing, and transforming materials that would otherwise be discarded as trash into new products. Recycling benefits local communities and the environment because it reduces the amount of waste sent to landfills and incinerators, conserves natural resources, increases economic security by utilizing domestic material sources, prevents pollution by reducing the need to collect new raw materials, and saves energy.

[0006] After collection, recyclables are typically sent to a materials recovery facility to be sorted, cleaned, and processed into materials that can be used in manufacturing. As a result, a high-throughput automated sorting platform that economically sorts highly mixed waste streams would be beneficial across a variety of industries. Thus, there is a need for a cost-effective sorting platform that can identify, analyze, and separate mixed industrial or municipal waste streams at high throughput to economically produce higher quality feedstock (which may also have lower levels of trace contaminants) for subsequent processing. Typically, materials recovery facilities are unable to distinguish between many materials, limiting the scrap to lower quality, lower value markets, or are very slow, labor intensive, and inefficient, limiting the amount of material that can be economically recycled or recovered.

[0007] Because scrap metal is often shredded, sorting is required to facilitate recycling of the metal. Sorting scrap metal reuses metals that may otherwise end up in landfills. In addition, using sorted scrap metal reduces pollution and emissions compared to refining virgin materials from ores. If the quality of the sorted metal meets certain standards, manufacturers can use it instead of virgin materials. Scrap metals can include ferrous and non-ferrous metals, heavy metals, high-value metals such as nickel or titanium, cast or wrought metals, and various other alloy types. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] U.S. Patent Application Serial No. 17 / 667,397 [Patent Document 2] International Published Patent Application No. WO 99 / 2248 [Patent Document 3] U.S. Patent No. 10,207,296 [Non-patent literature]

[0009] [Non-Patent Document 1] K. Alipour et al., "Point to-Point Stable Motion Planning of Wheeled Mobile Robots with Multiple Arms for Heavy Object Manipulation," 2011 IEEE International Conference on Robotics and Automation, pp. 6162-6167, May 9-13, 2011. [Non-Patent Document 2] Krizhevsky et al., “ImageNet Classification with Deep Convolutional Networks,” Proceedings of the 25th International Conference on Neural Information Processing Systems, December 3-6, 2012. [Non-Patent Document 3] LeCun et al., "Gradient-Based Learning Applied to Document Recognition," Proceedings of the IEEE, Institute of Electrical and Electronic Engineers (IEEE), November 1998. Summary of the Invention [Means for solving the problem]

[0010] Various detailed embodiments of the present disclosure are disclosed herein. However, it should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be embodied in various alternative forms. The figures are not necessarily to scale, and some features may be exaggerated or reduced to show details of specific components. Therefore, specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to use various embodiments of the present disclosure. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram of a material handling system that can be utilized to train an artificial intelligence ("AI") system in accordance with an embodiment of the present disclosure. [Diagram 2] FIG. 1 illustrates an exemplary representation of a control set of pieces of material used during the training phase in an artificial intelligence system. [Diagram 3] FIG. 2 is a flow chart diagram arranged in accordance with an embodiment of the present disclosure. [Figure 4] FIG. 1 shows piles of scrap metal collected in a scrap yard. [Diagram 5] FIG. 1 is a block diagram of a data processing system configured in accordance with an embodiment of the present disclosure. [Figure 6] FIG. 1 illustrates a schematic diagram of an apparatus constructed in accordance with an embodiment of the present disclosure for sorting / identifying and / or screening / separating material collected into one or more piles, such as scrap metal at a scrap yard. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Embodiments of the present disclosure utilize an artificial intelligence ("AI") / vision system configured to identify / sort various metal alloys collected in a scrap yard.

[0013] As used herein, "material" means any material, including but not limited to, metals (ferrous and non-ferrous), metal alloys, metal alloy scrap pieces, heavy, zorba, twitch, metal pieces embedded in another, different material, plastics (including but not limited to any of the plastics known in the industry, disclosed herein, or newly created in the future), rubber, foam, glass (including but not limited to borosilicate or soda lime glass, and glass of various colors), ceramics, paper, cardboard, Teflon, PE, bundles of wire, insulated wire, rare earth elements, leaves, wood, plants, plant parts, fibers, biowaste, packaging, electronic waste, batteries and accumulators, end-of-life vehicles, scrap from mining, construction, and demolition waste. The waste stream may include any item or object, including but not limited to, plastic wrap, crop waste, forest residues, purpose-grown grasses, woody energy crops, microalgae, urban food waste, food waste, hazardous chemical and biomedical waste, construction debris, farm waste, biological items, non-biological items, objects with a particular carbon content, any other object that may be found in urban solid waste, and any other object, item, or material disclosed herein, including further types or classes of any of the foregoing that can be distinguished from one another by, but not limited to, one or more sensor systems, including but not limited to, any of the sensor technologies disclosed herein.

[0014] In a more general sense, a "material" may include any item or object composed of a chemical element, a compound or mixture of one or more chemical elements, or a compound or mixture of compounds or mixtures of chemical elements, where the compound or mixture may range in complexity from simple to complex (all of which may be referred to herein as materials having a particular "chemical composition"). "Chemical element" means a chemical element of the Periodic Table of Chemical Elements, including chemical elements that may be discovered after the filing date of this application. Within this disclosure, the terms "scrap," "scrap piece," "material," and "material piece" may be used interchangeably. As used herein, material piece or scrap piece referred to as having a metal alloy composition is a metal alloy having a particular chemical composition that distinguishes it from other metal alloys.

[0015] As used herein, a "pile" of material refers to a pile of stacked or overlapping things. As used herein, a "pile" of things is usually messy and often has the shape of a hill or mound.

[0016] As used herein, the term "chemical signature" refers to a unique pattern (e.g., a fingerprint spectrum) that may be generated by one or more analytical instruments that indicates the presence of one or more particular elements or molecules (including polymers) in a sample. The elements or molecules may be organic and / or inorganic. Such analytical instruments include any of the sensor systems disclosed herein and in U.S. Patent Application No. 17 / 667,397, which is hereby incorporated by reference herein. According to embodiments of the present disclosure, one or more such sensor systems may be configured to generate a chemical signature of a piece of material.

[0017] As is well known in the art, a "polymer" is a substance or material composed of very large molecules, or macromolecules, made up of many repeating subunits. A polymer can be a natural polymer found in nature or a synthetic polymer. A "multilayer polymer film" is composed of two or more different polymer compositions. The layers are at least partially contiguous and preferably, but optionally, coextensive. As used herein, the terms "plastic," "plastic piece," and "plastic material piece" (all of which may be used interchangeably) refer to any object that includes or is composed of one or more polymers and / or polymer compositions of a multilayer polymer film.

[0018] As used herein, "fraction" refers to any specified combination of organic and / or inorganic elements or molecules, polymer type, plastic type, polymer composition, chemical signature of a plastic, physical properties of a plastic piece (e.g., color, clarity, strength, melting point, density, shape, size, manufacturing type, uniformity, response to stimuli, etc.), and the like, including any and all of the various classifications and types of plastics disclosed herein. Non-limiting examples of fractions are one or more different types of plastic pieces including LDPE with a relatively high percentage of aluminum, LDPE and PP with a relatively low percentage of iron, PP with zinc, combinations of PE, PET, and HDPE, any type of red LDPE plastic piece, any combination of plastic pieces except PVC, black plastic pieces, combinations of #3-#7 types of plastics containing a specified combination of organic and inorganic molecules, combinations of one or more different types of multilayer polymer films, combinations of specified plastics that do not contain specified contaminants or additives, any type of plastic with a melting point greater than a specified threshold, any thermosetting plastic of multiple specified types, specified plastics that do not contain chlorine, combinations of plastics with similar densities, combinations of plastics with similar polarity, and plastic bottles without caps or vice versa.

[0019] As used herein, the term "predetermined" refers to something that is pre-established or determined, for example, by a user of an embodiment of the present disclosure.

[0020] As used herein, "spectral imaging" is imaging that uses multiple bands across the electromagnetic spectrum. A typical camera captures an image composed of light across three wavelength bands in the visible spectrum, red, green, and blue (RGB), but spectral imaging encompasses a variety of techniques that include and go beyond RGB. For example, spectral imaging can use infrared, visible, ultraviolet, and / or x-ray spectrum, or some combination of the above. Spectral data, or spectral image data, is a digital data representation of a spectral image. Spectral imaging can include simultaneous acquisition of spectral data in visible and non-visible bands, illumination from outside the visible range, or use of optical filters to capture specific spectral ranges. It is also possible to capture hundreds of wavelength bands for each pixel in a spectral image. As used herein, the term "image data packet" refers to a packet of digital data associated with a captured spectral image of an individual piece of material.

[0021] As used herein, the terms "identify" and "classify", "identification" and "classification", and any derivatives of the foregoing, may be utilized interchangeably. As used herein, to "classify" a piece of material is to determine (i.e., identify) the type or class of material to which the piece of material belongs. For example, according to some embodiments of the present disclosure, a sensor system (as described further herein) may be configured to collect and analyze any type of information to classify materials and distinguish such classified materials from other materials, which classification may be utilized in a separation device that selectively separates pieces of material according to a set of one or more physical and / or chemical properties (e.g., which may be user-defined), including, but not limited to, color, texture, hue, shape, brightness, weight, density, chemical composition, size, uniformity, manufacturing type, chemical signature, predefined fraction, radioactive signature, transmittance to light, sound, or other signals, and response to various field-like stimuli, including emitted and / or reflected electromagnetic radiation ("EM") of the pieces of material.

[0022] The types or classes (i.e., classifications) of materials can be user-definable (e.g., predetermined) and are not limited to any known classification of materials. The granularity of these types or classes can range from very coarse to very fine. For example, these types or classes can include plastics, ceramics, glass, metals, and other materials, which are relatively coarse in granularity, different metals and metal alloys, such as zinc, copper, brass, chrome plating, and aluminum, which are finer in granularity, or between specific types of metal alloys, which are relatively fine in granularity. Thus, these types or classes can be configured to distinguish between materials of widely different chemical compositions, such as, for example, plastics and metal alloys, or to distinguish between materials of nearly identical chemical compositions, such as, for example, different types of metal alloys. It should be understood that the methods and systems discussed herein can be applied to accurately identify / classify pieces of material whose chemical composition is entirely unknown before being classified.

[0023] As used herein, "production type" refers to the type of manufacturing process by which a piece of material is produced, such as a forged metal part formed by a forging process, cast (including but not limited to expendable die casting, permanent die casting, and powder metallurgy), material removal process, and the like.

[0024] As used herein, a heterogeneous mixture of multiple pieces of material includes at least one piece of material that has a different chemical composition than one or more other pieces of material, and / or at least one piece of material in the heterogeneous mixture is physically distinguishable from the other pieces of material, and / or at least one piece of material in the heterogeneous mixture is a different class or type of material from the other pieces of material in the mixture, and the devices and methods are configured to identify / classify / separate / separate the piece of material into a group separate from such other pieces of material. In contrast, a homogeneous set or group of materials all fall into the same distinguishable class or type of material.

[0025] Embodiments of the present disclosure may be described herein as separating pieces of material (e.g., different metal alloys) into separate piles, heaps, containers, or bins by physically separating such pieces of material into separate groups according to user-defined or pre-defined groupings (e.g., classifications of material types). As an example, within some embodiments of the present disclosure, pieces of material may be separated into separate piles or containers to separate pieces of material classified as belonging to a particular class or type of material that is distinguishable from other pieces of material (e.g., that are classified as belonging to a different class or type of material).

[0026] It should be noted that the materials to be separated may have irregular sizes and shapes, for example, such materials may have previously been passed through some type of shredding mechanism that chopped them into pieces of such irregular shapes and sizes (generating scrap pieces).

[0027] Although embodiments of the present disclosure are described with respect to the separation of different steel alloys, the present disclosure is not so limited and is applicable to the identification and / or separation of any class / type of material that may be collected in a group, pile, or heap, for example in a warehouse or in a scrap yard. The terms "alloy steel" and "steel alloy" refer to steel with other alloying elements added in addition to carbon. Common alloying elements include manganese (Mn), nickel (Ni), chromium (Cr), molybdenum (Mo), vanadium (V), silicon (Si), and boron (B). Less common alloying elements include aluminum (Al), cobalt (Co), copper (Cu), cerium (Ce), niobium (Nb), titanium (Ti), tungsten (W), tin (Sn), zinc (Zn), lead (Pb), and zirconium (Zr).

[0028] For example, Figure 4 depicts an exemplary scrap yard 400 including N (N>1) piles 401...403 of a mixture of different types of steel alloy scrap pieces in each pile. A non-limiting advantage of the present disclosure is the separation of large metal pieces before they are shredded into much smaller pieces, which may enable such mixtures of metal alloys to be separated before they are shredded into much smaller pieces, many of which may be discarded or shredded to much smaller dimensions such that they are substantially lost and not effectively recycled. Such large metal pieces may be as large as engine blocks or large portions of engine blocks, steel beams, etc.

[0029] Referring now to Figure 6, there is shown a simplified diagram of an apparatus 600 configured in accordance with an embodiment of the present disclosure for sorting / identifying and / or separating pieces of material collected into one or more piles, such as scrap metal pieces in a scrap yard (e.g., see Figure 4). For purposes of illustrating embodiments of the present disclosure, the pieces of material are various metal (e.g., steel) alloy pieces of different chemical compositions. Figure 6 depicts the metal alloy pieces collected in a pile of metal alloy pieces, indicated as metal alloy pile 605, where the different metal alloy pieces may be randomly mixed within metal alloy pile 605. Metal alloy pile 605 may be located in a warehouse or at a scrap yard, as depicted in Figure 4.

[0030] A camera 610 can be mounted (e.g., on a pillar or wall of a building) within the vicinity of the pile 605 to capture images and / or video of the metal alloy flakes in the metal alloy pile 605. The camera 610 can be similar to the camera 109 described herein with respect to FIG. 1. Image data collected by the camera 610 can be transmitted to a computer system 612 by any suitable means, including wired or wireless transmission. The computer system 612 can include a vision system similar to the vision system 110 described herein with respect to FIG. 1. The computer system 612 can be configured to process the image data according to the system and process 300 described herein with respect to FIG. 3 to determine the location of each of the metal alloy flakes in the pile 605 relative to one another (or at least the metal alloy flakes that can be visualized by the camera 610, since some metal alloy flakes may be hidden under other metal alloy flakes and therefore cannot be visualized until exposed), and then to classify / identify which of the metal alloy flakes belong to one or more of the metal alloy classifications.

[0031] This location and classification information can then be transmitted to controller 614, which controls the operation of separation device 620 to grasp / collect and remove each of the classified metal alloy pieces from pile 605 and then optionally separate them into different metal alloy piles 601, 602, and 603. Separation device 620 can include any suitable robotic arm manipulator that can be controlled by controller 614 to automatically select a particular metal alloy piece in pile 605 that has been identified by the vision system as belonging to a metal alloy classification, and physically grasp or pick that particular metal alloy piece by any suitable means and remove it from pile 605. Separation device 620 can also be configured to deposit it in a predetermined location (e.g., one of metal alloy piles 601, 602, or 603). One example of a robotic arm is disclosed in K. Alipour et al., "Point to-Point Stable Motion Planning of Wheeled Mobile Robots with Multiple Arms for Heavy Object Manipulation," 2011 IEEE International Conference on Robotics and Automation, pp. 6162-6167, May 9-13, 2011, which is hereby incorporated by reference. Other robotic manipulators that may be used within the embodiments of the present disclosure are well known in the art. Alternatively, the gripper may be a large magnet to physically pick up the identified debris.

[0032] Apparatus 600 can be configured to remove a particular classification of metal alloys from pile 605, or to remove all but one or more classifications of metal alloy flakes from pile 605. Alternatively, apparatus 600 can be configured to remove one or more particular classifications of metal alloy flakes from pile 605 and place them in one or more other locations, such as one or more containers or bins, or one or more other piles (e.g., one or more of piles 601, 602, 603).

[0033] As described further herein, training of the vision system can be performed according to various suitable techniques as disclosed herein such that the vision system is able to use image data from captured images to classify / identify different steel alloys.

[0034] According to some embodiments of the present disclosure, the camera 610 can be configured with one or more devices for capturing or acquiring images of the pieces of material in the pile 605. These devices can be configured to capture or acquire any desired wavelength ranges that are emitted or reflected by the pieces of material, including but not limited to visible light, infrared (IR) light, and ultraviolet (UV) light. For example, the camera 610 can be configured with one or more cameras (still and / or video, any of which can be configured to capture two-dimensional, three-dimensional, and / or holographic images) positioned near (e.g., above) the pile 605 such that images of the pieces of material are captured (e.g., as image data). Alternatively, the camera 610 can include one or more laser lights for illuminating some of the pieces and a camera for then capturing images of the illuminated pieces. For example, some types of plastics can be illuminated with some wavelengths of light to distinguish them from other types of plastics.

[0035] Regardless of the type of sensed characteristics / captured information of the material pieces, this information can then be transmitted to the computer system 612 and processed (e.g., by a vision system and an AI system) to identify and / or classify each of the material pieces. The AI ​​system may be any well-known AI system (e.g., artificial narrow intelligence ("ANI"), artificial general intelligence ("AGI"), and artificial superintelligence ("ASI")), machine learning systems including those implementing neural networks (e.g., artificial neural networks, deep neural networks, convolutional neural networks, recurrent neural networks, autoencoders, reinforcement learning, etc.), supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, self-learning, feature learning, sparse dictionary learning, anomaly detection, robot learning, association rule learning, fuzzy logic, deep learning algorithms, deep structured learning hierarchical learning algorithms, support vector machines ("SVM") (e.g., linear, The machine learning system may implement a class of algorithms that implements techniques such as SVM, nonlinear SVM, SVM regression, etc.), decision tree learning (e.g., classification and regression trees ("CART")), ensemble methods (e.g., ensemble learning, random forests, bagging and pasting, patch and subspace, boosting, stacking, etc.), dimensionality reduction (e.g., projection, manifold learning, principal component analysis, etc.), and / or deep machine learning algorithms, such as those described and publicly available at the deeplearning.net website, which is hereby incorporated by reference (including hyperlinks to all software, publications, and available software referenced within this website).Non-limiting examples of publicly available machine learning software and libraries that may be utilized within embodiments of the present disclosure include Python, OpenCV, Inception, Theano, Torch, PyTorch, Pylearn2, Numpy, Blocks, TensorFlow, MXNet, Caffe, Lasagne, Keras, Chainer, Matlab Deep Learning, CNTK, MatConvNet (a MATLAB toolbox implementing convolutional neural networks for computer vision applications), DeepLearnToolbox (a Matlab toolbox for Deep Learning (from Rasmus Berg Palm)), BigDL, Cuda-Convnet (a fast C++ / CUDA implementation of convolutional (or more generally, feedforward) neural networks), Deep Belief Networks, RNNLM, RNNLIB-RNNLIB, matrbm, deeplearning4j, Eblearn.lsh, deepmat, MShadow, Matplotlib, SciPy, CXXNET, Nengo-Nengo, Eblearn, cudamat, Gnumpy, 3-way factor RBM and mcRBM, mPoT (Python code to train a model on natural images using CUDAMat and Gnumpy), ConvNet, Elektronn, OpenNN, NeuralDesigner, Theano Generalized Hebbian Learning, Apache Singa, Lightnet, and SimpleDNN.

[0036] According to some embodiments of the present disclosure, some types of machine learning can be performed in two stages. For example, first, training is performed, which can be performed offline, and the system 100 is not utilized to perform the actual classification / sorting of the material pieces. The system 100 can be utilized to train the machine learning system, where a homogenous set of material pieces (also referred to herein as control samples) (i.e., having the same type or class of material or falling within the same predefined fraction) can be passed through the system 100 (e.g., by the conveyor system 103) and collected in a common container (e.g., container 140). Alternatively, training can include using some other mechanism to collect sensory information (characteristics) of the control set of material pieces. During this training phase, algorithms within the machine learning system extract features from the captured information (e.g., using image processing techniques well known in the art). Non-limiting examples of training algorithms include, but are not limited to, linear regression, gradient descent, feedforward, polynomial regression, learning curves, regularized learning models, and logistic regression. It is during this training phase that algorithms within the machine learning system learn the relationships between materials and their features / characteristics (e.g., as captured by a vision system and / or a sensor system) to create a knowledge base for subsequent classification (e.g., by device 600) of heterogeneous mixtures of material pieces that can then be separated according to a desired classification. Such a knowledge base can include one or more libraries, each library including parameters (e.g., neural network parameters) for the machine learning system to utilize in classifying the material pieces. For example, one particular library can include parameters configured by the training phase to recognize and classify a particular type or class of material, or one or more materials that fall into a given fraction.According to some embodiments of the present disclosure, such a library can be input into computer system 612, and a user of device 600 may then be able to adjust some of the parameters to tune the operation of device 600 (e.g., adjusting the effectiveness of a threshold for how well the machine learning system recognizes particular pieces of material within pile 605).

[0037] Additionally, the inclusion of a particular material in a piece of material results in identifiable physical characteristics (e.g., visually distinguishable properties) of the material. As a result, when multiple pieces of material containing such a particular composition are put through the aforementioned training phase, the machine learning system can learn how to distinguish such pieces of material from others. Thus, a machine learning system configured according to some embodiments of the present disclosure can be configured to distinguish between pieces of material according to their respective material / chemical composition. For example, such a machine learning system can be configured to classify / identify pieces of material containing a particular element according to the proportion (e.g., weight or volume percentage) of that element contained within the piece of material.

[0038] 2, during the training phase, one or more examples of material pieces 201 of a particular class or type of material, which may be referred to herein as a set of one or more control samples, are conveyed past the vision system (e.g., by conveyor system 203) such that one or more algorithms in the machine learning system detect, extract, and learn characteristics or features representative of such type or class of material. Note that material pieces 201 can be any of the "materials" disclosed herein (e.g., metal alloy pieces representative of those that will be present in pile 605).

[0039] For example, each of the pieces of material 201 may represent one or more particular types or classes of metal alloys, which are put through a training phase such that one or more algorithms in the machine learning system "learn" (train) how to detect, recognize, and classify such pieces of material. When a vision system (e.g., vision system 110) is trained, it is trained to visually discriminate between pieces of material. This creates a library of parameters specific to such homogenous classes of pieces of material. The same process can be performed on images of any classification of pieces of material to create a library of parameters specific to such classification of pieces of material. For each type of material to be classified by the vision system, any number of exemplary pieces of material of that classification of material can be passed through the vision system. Given the captured sensory information as input data, the algorithms in the machine learning system can use N classifiers, each of which tests one of the N different material types. It should be noted that a machine learning system can be "taught" (trained) to detect any type, class, or fraction of material, including any of the types, classes, or fractions of materials disclosed herein.

[0040] After the algorithms are established and the machine learning system has sufficiently learned the differences for material classification (e.g., within a user-defined statistical confidence level), the library of parameters for different materials can then be implemented into a material classification and / or separation system (e.g., apparatus 600) to be used to identify and / or classify material pieces from a mixture of material pieces (e.g., in pile 605) and then possibly separate such classified material pieces as described with respect to FIG. 6.

[0041] Techniques for building, optimizing, and utilizing machine learning systems are known to those of skill in the art as can be found in the relevant literature, examples of which include Krizhevsky et al., "ImageNet Classification with Deep Convolutional Networks," Proceedings of the 25th International Conference on Neural Information Processing Systems, December 3-6, 2012, Lake Tahoe, NV, and LeCun et al., "Gradient-Based Learning Applied to Document Recognition," Proceedings of the IEEE, Institute of Electrical and Electronic Engineers (IEEE), November 1998, both of which are hereby incorporated by reference in their entireties.

[0042] It should be understood that the present disclosure is not exclusively limited to machine learning techniques. Other techniques for material classification / identification can also be used. For example, the vision system can utilize optical spectroscopy techniques using a multi- or hyperspectral camera for the camera 610 to provide a signal that can indicate the presence or absence of a type of material (e.g., containing one or more specific elements) by examining the spectral emission of the material. A photograph of the piece of material can also be used in a template matching algorithm to compare the captured image with a database of images to find the presence or absence of a particular type of material from the database. A histogram of the captured image can also be compared with a database of histograms. Similarly, a bag-of-words model can be used in conjunction with feature extraction techniques such as scale-invariant feature transform ("SIFT") to compare extracted features between the captured image and those in the database. According to some embodiments of the present disclosure, instead of utilizing a training phase in which control samples of the pieces of material are passed through the vision system, the training of the machine learning system can be performed using labeling / annotation techniques (or any other supervised learning techniques) whereby as the data / information for the pieces of material is captured by the vision system, a user inputs a label or annotation that identifies each piece of material, which is then used to create a library used by the machine learning system when classifying pieces of material within a heterogeneous mixture of pieces of material (e.g., pile 605). In other words, a previously generated knowledge base of properties captured from one or more samples of a class of material can be achieved by any of the techniques disclosed herein, and such knowledge base is then utilized to automatically classify the materials.

[0043] Thus, as disclosed herein, some embodiments of the present disclosure provide for identification / classification of one or more different materials to separate pieces of material from one another in pile 605. According to some embodiments, machine learning techniques can be utilized to train (i.e., configure) a neural network to identify a variety of one or more different classes or types of materials.

[0044] One point to mention here is that according to some embodiments of the present disclosure, the collected / captured / detected / extracted features / characteristics of the pieces of material are not necessarily just specifically identifiable physical characteristics; they may be abstract formulas that can only be expressed mathematically or not at all. Nevertheless, the machine learning system can be configured to analyze all of the data to look for patterns that will allow it to classify the control samples during the training phase. Furthermore, the machine learning system can take subsections of the captured information of the pieces of material and try to find correlations between predefined classifications.

[0045] It should be noted that a person skilled in the art would be able to distinguish the machine learning system described herein from a machine vision device or system. Due to the prior use of the term in the industry, electronic machine vision devices are generally used in conjunction with automated processing, assembly and inspection devices, especially robotic types. Television cameras are commonly used to observe the objects being processed, assembled, read, displayed or inspected, and the signals received and transmitted by the camera can be compared to standard signals or databases to determine whether the imaged article has been properly processed, finished, oriented, assembled, determined, etc. Machine vision devices are widely used in inspection and defect detection applications, allowing for rapid identification of inconsistencies and imperfections in both hard and soft goods, and immediate adjustment or rejection. Machine vision devices detect anomalies by comparing signals generated by the camera with predefined signals indicative of proper dimensions, appearance, orientation, etc. See International Published Patent Application No. WO 99 / 2248, which is hereby incorporated by reference herein. Nevertheless, machine vision systems do not perform any kind of further data processing (e.g., image processing), which would include further processing of the captured information through algorithms. See the definition of machine vision in Wikipedia, which is hereby incorporated by reference. Thus, it can be easily understood that a machine vision device or system does not include any kind of algorithm, such as a machine learning algorithm. Instead, a machine vision system essentially compares an image of a part to a template of images.

[0046] 3 shows a flow chart diagram illustrating an example embodiment of a process 300 for classifying / identifying and then separating pieces of material utilizing a vision system according to some embodiments of the present disclosure. The process 300 can be configured to operate within any of the embodiments of the present disclosure described herein, including the system 100 of FIG. 1 and the apparatus 600 of FIG. 6. For example, the process 300 can be utilized in the system 100 to train a machine learning system as described herein. In such a case, process blocks 302, 306, 312, and / or 313 may not be utilized. Additionally, the process 300 can be utilized in the apparatus 600 to remove / separate pieces of material from a pile 605 as described herein. In such a case, process block 306 may not be utilized.

[0047] The operations of process 300 may be performed by hardware and / or software, including within a computer system (e.g., computer system 3400 of FIG. 5) that controls a system (e.g., a vision system implemented within computer system 107, vision system 110, and / or computer system 612). At process block 301, an image of the piece of material is taken with a camera (e.g., camera 109, camera 610). At process block 302, the location of each piece of material in pile 605 is detected by the vision system to identify the location of each piece of material to be sorted / identified. At process block 303, sensed information / characteristics of the piece of material are captured / obtained from the image by the vision system. In process block 304, a vision system as disclosed above (e.g., vision system 110, or one implemented in computer system 612) can perform pre-processing of the captured information, which can be used to detect (extract) each of the pieces of material (e.g., from the background (e.g., other pieces of material in pile 605)), in other words, pre-processing can be used to identify the differences between the pieces of material and the background. Well-known image processing techniques such as dilation, thresholding, and contouring can be used to identify the pieces of material as separate from the background. In process block 305, segmentation can be performed. For example, the captured information can include information about one or more pieces of material. Thus, in such cases, it may be desirable to isolate the image of the individual pieces of material from the background of the image. In an exemplary technique for process block 305, the first step is to apply high contrast to the image, such that the background pixels are reduced to substantially all black pixels and at least some of the pixels related to the pieces of material are brightened to substantially all white pixels. The image pixels of the white pieces of material are then dilated to cover the entire size of the pieces of material. After this step, the piece location is a high contrast image of all white pixels on a black background. A contour algorithm can then be utilized to detect the piece boundary.The boundary information is saved and the boundary locations are then transferred to the original image. Segmentation is then performed on the original image on areas larger than the previously defined boundaries. In this way, pieces of material are identified and separated from the background. According to an embodiment of the present disclosure, process block 305 can implement an image segmentation process such as Mask R-CNN.

[0048] At optional process block 306, the size and / or shape of the piece of material may be determined. At process block 307, post-processing may be performed. Post-processing may involve resizing the captured information / data to prepare it for use in the neural network. This may also include modifying some properties (e.g., enhancing image contrast, changing image background, or applying filters) that will enhance the ability of the machine learning system to classify the piece of material. At process block 309, the data may be resized. Data resizing may be desired in some circumstances to match the data input requirements for some machine learning systems, such as neural networks. For example, neural networks may require image sizes (e.g., 225×255 pixels or 299×299 pixels) that are much smaller than the size of images captured by a typical digital camera. Also, the smaller the size of the input data, the less processing time is required to perform the classification.

[0049] In process blocks 310 and 311, for each piece of material, a type or class of material is identified / classified based on the sensed / detected features. For example, process block 310 comprises a neural network using one or more machine learning algorithms that compare the extracted features with those stored in a knowledge base generated during a training phase and assign each piece of material a most consistent classification based on such comparison. The algorithms of the machine learning system can process the captured information / data hierarchically by using automatically trained filters. The filter responses are then successfully combined in the next level of the algorithm until a probability is obtained in a final step. In process block 311, these probabilities can be used for each of the N classifications (e.g., to determine which of the N separate piles 601...603 each piece of material should be classified into). For example, each of the N classifications can be assigned to one of the piles 601...603 and the piece of material under consideration is removed from pile 605 and placed into the pile corresponding to the classification that returns the highest probability greater than a predefined threshold. In an embodiment of the present disclosure, such a predefined threshold can be pre-set by a user. If none of the probabilities are greater than a predefined threshold, then the particular piece of material can be sorted into the outlier pile.

[0050] Next, at process block 312, the separation device 620 can be actuated via the controller 614 to pick or grasp and remove the sorted / identified pieces of material from the piles 605. At process block 313, the separation device 620 can then place the sorted / identified pieces of material into an appropriate one of the piles 601...603.

[0051] The vision system can be adjusted to compensate for different environmental conditions in which images of the piece are obtained, such as differences in how long the piece has been in the scrap yard, the amount of dirt on the piece, different weather conditions (e.g., snow, rain, sunshine), pieces that have been sprayed with water mixed with chemicals / detergents and pieces that have not, and pieces obtained from different sources.

[0052] FIG. 1 illustrates an example of a material handling system 100 that can be configured in accordance with various embodiments of the present disclosure to train a machine learning system implemented in a computer 612 to classify / identify different types / classes of materials, such as different metal alloys.

[0053] A conveyor system 103 may be implemented to transport individual (i.e., physically separable) pieces of material 101 through the system 100 such that each of the individual pieces of material 101 may be sorted into a predetermined desired group. Such a conveyor system 103 may be implemented with one or more conveyor belts (e.g., conveyor belts 102, 103) along which the pieces of material 101 typically move at a predetermined constant speed. However, some embodiments of the present disclosure may be implemented with other types of conveyor systems.

[0054] The conveyor belt 103 may be a conventional endless belt conveyor that uses a suitable conventional drive motor 104 to move the conveyor belt 103 at a predetermined speed. According to some embodiments of the present disclosure, any type of suitable feeder mechanism may be utilized to feed the pieces of material 101 onto the conveyor belt 103, which then transports the pieces of material 101 through various components in the system 100. Within some embodiments of the present disclosure, the conveyor belt 103 is operated to move at a predetermined speed by the conveyor belt motor 104. A belt speed detector 105 (e.g., a conventional encoder) may be operatively coupled to the conveyor belt 103 to provide information corresponding to the movement (e.g., speed) of the conveyor belt 103.

[0055] According to some embodiments of the present disclosure, after the pieces of material 101 are received by the conveyor belt 103, a tumbler and / or vibrator may be utilized to separate the individual pieces of material from the collection of pieces of material and then place them into one or more singulated (i.e., in-line) streams. According to alternative embodiments of the present disclosure, the pieces of material may be placed into one or more singulated (i.e., in-line) streams, which may be performed by an active or passive singulator 106. An example of a passive singulator is further described in U.S. Pat. No. 10,207,296. As previously discussed, the incorporation or use of a singulator is not required. Instead, the conveyor system (e.g., conveyor belt 103) may simply randomly transport the collection of pieces of material placed onto the conveyor belt 103.

[0056] The vision system 110 may utilize one or more stationary or live-moving cameras 109 to collect or capture information about each of the pieces of material 101. For example, the vision system 110 may be configured (e.g., with a machine learning system) to collect or capture any type of information that may be utilized within the system 100 to selectively classify the pieces of material 101 according to a set of one or more (user-defined) physical characteristics, including, but not limited to, the color, hue, size, shape, texture, overall physical appearance, uniformity, composition, and / or manufacturing type of the pieces of material 101. The vision system 110 captures images (including one-dimensional, two-dimensional, three-dimensional, or holographic imaging) of each of the pieces of material 101, for example, by using optical sensors such as those utilized in typical digital cameras and video equipment. Such images captured by the optical sensors are then stored in a memory device as image data. According to certain embodiments of the present disclosure, such image data represents images captured within optical wavelengths of light (i.e., wavelengths of light observable by a typical human eye). However, alternative embodiments of the present disclosure may utilize sensors capable of capturing images of materials composed of wavelengths of light outside the visible wavelengths of the typical human eye. This captured image data may then be utilized to classify a control set of material pieces and to train a machine learning system (as described herein) for each different type of material to be separated by the apparatus 600.

[0057] 5, a block diagram is depicted illustrating a data processing ("computer") system 3400 in which aspects of an embodiment of the present disclosure may be implemented. (The terms "computer", "system", "computer system", and "data processing system" may be used interchangeably herein.) The computer system 107, the computer system 612, and / or the vision system 110 may be configured similarly to the computer system 3400. The computer system 3400 may use a local bus 3405 (e.g., a peripheral component interconnect ("PCI") local bus architecture). Any suitable bus architecture may be utilized, such as Accelerated Graphics Port ("AGP") and Industry Standard Architecture ("ISA"), among others. One or more processors 3415, volatile memory 3420, and non-volatile memory 3435 may be connected to the local bus 3405 (e.g., through a PCI bridge (not shown)). An integrated memory controller and cache memory may be coupled to the one or more processors 3415. The one or more processors 3415 may include one or more central processing units and / or one or more graphics processing units and / or one or more tensor processing units. Additional connections to the local bus 3405 may be made through direct component interconnects or through add-in boards. In the illustrated example, a communications (e.g., network (LAN)) adapter 3425, an I / O (e.g., small computer system interface ("SCSI") host bus) adapter 3430, and an expansion bus interface (not shown) may be connected to the local bus 3405 by direct component connections. An audio adapter (not shown), a graphics adapter (not shown), and a display adapter 3416 (coupled to a display 3440) may be connected to the local bus 3405 (e.g., by an add-in board inserted into an expansion slot).

[0058] User interface adapter 3412 may provide connections for a keyboard 3413 and mouse 3414, a modem / router (not shown), and additional memory (not shown). I / O adapter 3430 may provide connections for a hard disk drive 3431, a tape drive 3432, and a CD-ROM drive (not shown).

[0059] One or more operating systems may be run on one or more processors 3415 and used to coordinate and provide control of various components within the computer system 3400. In FIG. 5, the operating systems may be commercially available operating systems. An object oriented programming system (e.g., Java, Python, etc.) may work in conjunction with the operating system to provide calls to the operating system from programs (e.g., Java, Python, etc.) running on the system 3400. Instructions for the operating systems, object oriented operating systems, and programs may be located on a non-volatile memory 3435 storage device, such as a hard disk drive 3431, and loaded into the volatile memory 3420 for execution by the processor 3415.

[0060] Those skilled in the art will appreciate that the hardware in FIG. 5 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash ROM (or equivalent non-volatile memory) or optical disk drives, etc., may be used in addition to or in place of the hardware shown in FIG. 5. Also, any of the processes of this disclosure may be applied to a multi-processor computer system or performed by multiple such systems 3400. For example, training of the vision system 110 may be performed by a first computer system 3400, while the operations of the computer system 612 for classification may be performed by a second computer system 3400.

[0061] As another example, computer system 3400 can be a stand-alone system configured to be bootable without depending on any type of network communication interface, regardless of whether computer system 3400 includes any type of network communication interface. As a further example, computer system 3400 can be an embedded controller configured with ROM and / or flash ROM that provides non-volatile memory for storing operating system files or user generated data.

[0062] 5 and above-described examples are not meant to imply architectural limitations. Moreover, the computer program format of the aspects of the present disclosure may reside on any computer readable storage medium (i.e., floppy disk, compact disk, hard disk, tape, ROM, RAM, etc.) for use by a computer system.

[0063] As described herein, embodiments of the present disclosure may be implemented to perform various described functions for identifying, locating, sorting, and / or separating pieces of material. Such functionality may be implemented in hardware and / or software, such as in one or more data processing systems (e.g., data processing system 3400 of FIG. 5), such as computer system 107, vision system 110, and / or computer system 612 described above. However, the functionality described herein should not be limited to implementation on any particular hardware / software platform.

[0064] As will be appreciated by those skilled in the art, aspects of the present disclosure may be embodied as a system, process, method, program product, and / or apparatus. Accordingly, various aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may be generally referred to herein as a "circuit," "circuitry," "module," or "system." Additionally, aspects of the present disclosure may take the form of a program product embodied in one or more computer-readable storage mediums having computer-readable program code embodied therein. (However, any combination of one or more computer-readable media may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.)

[0065] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, processes, program products, and apparatus according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of code, which includes one or more executable program instructions for implementing the specified logical function. It should also be noted that in some implementations, the functions noted in the blocks may be executed in a different order than that noted in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may be executed in the reverse order depending on the functionality involved.

[0066] Modules implemented in software for execution by various types of processors (e.g., CPU 3415) may include, for example, one or more physical or logical blocks of computer instructions, which may be organized, for example, as objects, procedures, or functions. However, executable files of identified modules need not be physically located together, but may include different instructions stored in different locations that, when logically coupled together, comprise the module and accomplish a specified purpose for the module. In fact, a module of executable code may be a single instruction, or many instructions, and may be distributed across several different code segments, among different programs, and across several memory devices. Similarly, operational data (e.g., a material classification library as described herein) may be identified and illustrated in modules herein, and may be embodied in any suitable form and organized in any suitable type of data structure. The operational data may be collected as a single data set or distributed across different locations, including different storage devices. The data may provide electronic signals over a system or network.

[0067] These program instructions can be provided to one or more processors and / or controllers of a general purpose computer, special purpose computer, or other programmable data processing apparatus (e.g., a controller) to produce a machine, such that the instructions, executed via the processor (e.g., CPU 3415) of the computer or other programmable data processing apparatus, create circuits or means for implementing the functions / acts specified in the flowchart and / or block diagram blocks.

[0068] It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems (e.g., including one or more graphics processing units) that perform the specified functions or operations, or by a combination of special purpose hardware and computer instructions. For example, a module can be implemented as a hardware circuit including custom VLSI circuits or gate arrays, off-the-shelf semiconductors, e.g., logic chips, transistors, controllers, or other discrete components. A module can also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, and the like.

[0069] In the description herein, the techniques of the flow charts may be described as a series of sequential operations. The order of operations, and the elements performing the operations, may be freely modified without departing from the scope of the present teachings. Operations may be added, removed, or modified in several ways. Similarly, the operations may be reordered or looped. Furthermore, while a process, method, algorithm, or the like may be described in a sequential order, such a process, method, algorithm, or any combination thereof, may be operable to be performed in an alternative order. Furthermore, some operations within a process, method, or algorithm may be performed simultaneously (e.g., operations performed in parallel), at least for some time, and may also be performed in whole, in part, or in any combination thereof.

[0070] References are made herein to "configuring" a device or a device "configured to" perform some function. It should be understood that this can include selecting and logically associating predefined logic blocks to provide a particular logic function, including monitoring or control functions. It can also include programming the computer software-based logic of a retrofit control device, wiring discrete hardware components, or a combination of any or all of the above. A device so configured is physically designed to perform a specified function.

[0071] Unless described herein, many of the details regarding specific materials, process acts, and circuits are conventional and can be found in textbooks and other sources in the computing, electronics, and software arts.

[0072] Computer program code, i.e., instructions, for carrying out operations for aspects of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, Python, C++, etc., traditional procedural programming languages ​​such as the "C" programming language or similar programming languages, programming languages ​​such as MATLAB or LabVIEW, or any of the machine learning software disclosed herein. The program code can be executed entirely on the user's computer system, partially on the user's computer system, as a standalone software package, partially on the user's computer system (e.g., the computer system utilized for sorting) and partially on a remote computer system (e.g., the computer system utilized to train the machine learning system), or entirely on a remote computer system or server. In the latter scenario, the remote computer system can be connected to the user's computer system through any type of network, including a local area network ("LAN") or wide area network ("WAN"), or can be connected to an external computer system (e.g., through the Internet using an Internet Service Provider). As an example of the foregoing, various aspects of the disclosure may be configured to execute on one or more of computer system 107, vision system 110, and computer system 612.

[0073] According to an alternative embodiment of the present disclosure, instead of or in addition to utilizing a camera 610 and associated vision system, a sensor device can be utilized to classify each of the pieces of material in pile 605. Such a sensor device can be mounted anywhere on separation device 620, such as on the arm or gripping mechanism of a robotic manipulator.

[0074] The sensor device may be any sensor that utilizes emitted or reflected electromagnetic radiation (e.g., infrared ("IR"), Fourier transform IR ("FTIR"), forward looking infrared ("FLIR"), very near infrared ("VNIR"), near infrared ("NIR"), short wavelength infrared ("SWIR"), long wavelength infrared ("LWIR"), mid wavelength infrared ("MWIR" or "MIR"), x-ray transmission ("XRT"), gamma ray, ultraviolet ("UV"), x-ray fluorescence ("XRF"), laser induced breakdown spectroscopy ("LIBS"), Raman spectroscopy, anti-Stokes spectroscopy, etc. It can be comprised of any type of sensor technology, including sensors utilizing Raman spectroscopy, gamma spectroscopy (which can be utilized to sense objects hidden by other objects, such as in mountains), hyperspectral spectroscopy (e.g., any range beyond visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, terahertz spectroscopy (including one-dimensional, two-dimensional, or three-dimensional imaging in any of the foregoing), or any other type of sensor technology, including but not limited to chemical or radioactive. Implementations of XRF systems are further described in U.S. Pat. No. 10,207,296.

[0075] The following sensor systems can also be used in some embodiments of the present disclosure to determine the chemical signature of plastic pieces and / or to classify the plastic pieces. Various forms of infrared spectroscopy disclosed above can be utilized to take a chemical signature unique to each plastic piece that provides information about the base polymer of any plastic material, as well as other components present in the material (mineral fillers, copolymers, polymer blends, etc.). Differential scanning calorimetry ("DSC") is a thermal analysis technique that obtains thermal transitions generated during heating of an analyzed material that is unique to each material. Thermogravimetric analysis ("TGA") is another thermal analysis technique that provides quantitative information about the composition of a plastic material in terms of the percentage of polymers, other organic components, mineral fillers, carbon black, etc. Capillary and rotational rheometry can determine the rheological properties of a polymer material by measuring its creep and deformation resistance. Optical and scanning electron microscopy ("SEM") can provide information about the structure of the analyzed material in terms of the number and thickness of layers in a multilayer material (e.g., a multilayer polymer film), the dispersion size of pigment or filler particles in a polymer matrix, coating defects, interface morphology between components, etc. Chromatography (e.g., LC-PDA, LC-MS, LC-LS, GC-MS, GC-FID, HS-GC) can quantify minor components of plastic materials such as UV stabilizers, antioxidants, plasticizers, anti-slip agents, etc., as well as residual monomers, residual solvents from inks or adhesives, decomposition products, etc.

[0076] An aspect of the disclosure provides an apparatus including a camera configured to capture images of individual metal alloy flakes contained within a first pile of a heterogeneous mixture of metal alloy flakes including at least one metal alloy flake having a first metal alloy composition and at least one metal alloy flake having a second metal alloy composition, a data processing system implemented with an AI system, the AI ​​system configured to assign a first classification to the first metal alloy flakes having the first metal alloy composition responsive to processing the captured images of the first metal alloy flakes through the AI ​​system, and a separation device configured to automatically grasp and remove the first metal alloy flakes from the first pile in response to the first classification. The separation device can be configured to deposit the first metal alloy flakes onto a second pile of metal alloy flakes. The second pile can consist of metal alloy flakes having the first metal alloy composition. The AI ​​system is configured to assign a second classification to the second metal alloy flakes having a second metal alloy composition in response to processing the captured images of the second metal alloy flakes through the AI ​​system, and the separation device is configured to automatically grip and remove the second metal alloy flakes from the first pile in response to the second classification, and the separation device is configured to deposit the second metal alloy flakes onto a third pile of metal alloy flakes. The third pile can be a homogenous collection of metal alloy flakes having the second metal alloy composition. The artificial intelligence system can be comprised of a neural network that uses one or more algorithms to compare features detected in the captured images with those stored in a knowledge base generated during a training phase, during which the one or more algorithms learn relationships between one or more specified classes of metal alloys and their features extracted from the captured image data creating the knowledge base. The camera can be configured to capture visual images of the individual metal alloy flakes. The first metal alloy flakes can be steel alloy flakes. The separating device can include a robotic arm having a controller that receives instructions from a data processing system that identifies a first location of the first metal alloy pieces in the first pile and identifies a second location of the second pile. The first and second piles can be located in a scrap metal yard.

[0077] An aspect of the disclosure provides an apparatus including a sensor configured to capture one or more characteristics of each of a mixture of pieces of material contained within a first pile of pieces of material, the mixture of pieces of material including pieces of material having different chemical compositions, a data processing system configured to assign a first classification to the first pieces of material having the first chemical composition in response to processing of the captured one or more characteristics of the first pieces of material, and a separation device configured to automatically grip and remove the first pieces of material from the first pile in response to the first classification. The separation device can include a robotic arm having a controller that receives instructions from the data processing system that identifies a first location of the first pieces of material within the first pile and identifies a second location of the second pile, and the sensor is an x-ray fluorescence system mounted on the arm of the separation device. The sensor can be a camera configured to capture visual images of the pieces of material, and the data processing system is implemented with an AI system, and the AI ​​system is configured to assign a first classification to the first pieces of material having the first chemical composition in response to processing of the captured visual images of the first pieces of material through the AI ​​system. The AI ​​system can be configured to assign a second classification to the second piece of material having a second chemical composition in response to processing the captured visual image of the second piece of material through the AI ​​system, and the separation device is configured to automatically grasp and remove the second piece of material from the first pile in response to the second classification, the first and second classifications being different from one another, the separation device is configured to deposit the first piece of material onto the second pile of material pieces, and the separation device is configured to deposit the second piece of material onto a third pile of material pieces. The second pile can be a homogenous collection of material pieces having the first chemical composition, and the third pile is a homogenous collection of material pieces having the second chemical composition. The pieces of material can be scrap metal alloy pieces, and the first pieces of material can be first scrap metal alloy pieces, and the first and second piles are located within a scrap metal yard, and the separation device is configured to deposit the first scrap metal alloy pieces onto the second pile of scrap metal alloy pieces.The separating device may include a robotic arm having a controller that receives instructions from a data processing system to identify a first location of the first scrap metal alloy pieces in the first pile and to identify a second location in the second pile. The AI ​​system may be comprised of a neural network that uses one or more algorithms to compare features detected in the captured images with those stored in a knowledge base generated during a training phase, during which the one or more algorithms learn relationships between one or more specified classes of scrap metal alloys and their features extracted from the captured image data creating the knowledge base.

[0078] In the description herein, numerous specific details are provided, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, controllers, robotic manipulators, etc., to provide a thorough understanding of embodiments of the present disclosure. However, those skilled in the relevant art will recognize that the present disclosure can be practiced without one or more of these specific details or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations may not be shown or described in detail to avoid obscuring aspects of the present disclosure.

[0079] Those skilled in the art who have read this disclosure will recognize that changes and modifications can be made to the embodiments without departing from the scope of the disclosure. It should be understood that the specific implementations shown and described herein can be illustrative of the disclosure and its best mode, and are in no way intended to otherwise limit the scope of the disclosure. Other variations may be within the scope of the following claims.

[0080] References throughout this specification to "one embodiment," "embodiment," or similar language mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of the phrases "in one embodiment," "in an embodiment," "embodiment," "several embodiments," "various embodiments," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment. Furthermore, the described features, structures, aspects, and / or characteristics of the present disclosure may be combined in any suitable manner in one or more embodiments. Correspondingly, even if some features are initially claimed as acting in a particular combination, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0081] Benefits, advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and any elements that may provide or make the benefits, advantages, or solutions more pronounced should not be construed as critical, necessary, or essential features or elements of any or all of the claims. Moreover, no element described herein is required to practice the present disclosure unless expressly described as essential or critical.

[0082] As used herein, the term "or" may be intended to be inclusive, such that "A or B" includes A or B, and also includes both A and B. As used herein, the term "and / or," when used in the context of a list of entities, refers to whether the entities are present alone or in combination. Thus, for example, the phrase "A, B, C, and / or D" includes A, B, C, and D individually, but also any and all combinations and subcombinations of A, B, C, and D.

[0083] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" may be intended to include the plural forms as well, unless the context clearly dictates otherwise.

[0084] Corresponding structure, materials, acts, and equivalents of all means or step-plus-function elements in the following claims may be intended to include any structure, material, or act for performing that function in combination with other claimed elements as specifically claimed.

[0085] As used herein with respect to a particular characteristic or circumstance, "substantially" refers to a degree of deviation that is small enough so as not to measurably impair that particular characteristic or circumstance. The exact degree of deviation that is permitted may depend on the specific context in some cases.

[0086] As used herein, a plurality of items, structures, components, and / or materials may be presented in common lists for convenience. However, these lists should be construed as though each member of the list is individually identified as a separate and unique member. Thus, each member of such a list should not be construed as a de facto equivalent of any other member of the same list solely based on their presentation in a common group, absent indication to the contrary.

[0087] Unless otherwise defined, all technical and scientific terms used herein (such as acronyms used for chemical elements in the periodic table) have the same meaning as commonly understood by one of ordinary skill in the art to which the presently disclosed subject matter belongs. Although any methods, devices, and materials similar or equivalent to those described herein can be used in the practice or testing of the presently disclosed subject matter, representative methods, devices, and materials are described herein.

[0088] The term "coupled," as used herein, is not intended to be limited to a direct or mechanical coupling. Unless otherwise noted, terms such as "first" and "second" are used to arbitrarily distinguish between the elements that such terms describe. As such, these terms are not necessarily intended to indicate a temporal or other priority of such elements. [Explanation of symbols]

[0089] 100 Material Handling Systems 101 Material pieces 102 Conveyor Belt 103 Conveyor System 104 Motor 105 Belt Speed ​​Detector 106 Singulator 107 Computer Systems 109 Live Motion Camera 110 Vision System 140 Container 201 Material pieces 203 Conveyor System 400 Scrap Yard 401…403 Mountain 3400 Computer System 3405 Local Bus 3412 User Interface Adapter 3413 Keyboard 3414 Mouse 3415 processor 3416 Display Adapter 3420 Volatile Memory 3425 Communication Adapter 3430 I / O Adapter 3431 Hard Disk Drive 3432 Tape Drive 3435 Non-volatile Memory 3440 Display 600 equipment 601 Mountain 602 Mountain 603 Mountain 605 Mountain 610 Camera 612 Computer Systems 614 Controller 620 Separation Device

Claims

1. a camera configured to capture images of individual metal alloy flakes contained within a first pile of heterogeneous mixture of metal alloy flakes, the first pile including at least one metal alloy flake having a first metal alloy composition and at least one metal alloy flake having a second metal alloy composition; a data processing system implemented with an artificial intelligence ("AI") system configured to assign a first classification to a first metal alloy flake having the first metal alloy composition in response to processing the captured image of the first metal alloy flake through the AI ​​system; and a separation device configured to automatically grasp and remove the first metal alloy pieces from the first pile in response to the first classification; An apparatus comprising:

2. The apparatus of claim 1 , wherein the separation device is configured to deposit the first metal alloy pieces onto a second pile of metal alloy pieces.

3. 3. The apparatus of claim 2, wherein the second pile is comprised of metal alloy pieces having the first metal alloy composition.

4. 3. The apparatus of claim 2, wherein the AI ​​system is configured to assign a second classification to second metal alloy pieces having the second metal alloy composition responsive to processing of the captured images of the second metal alloy pieces through the AI ​​system, the separation device is configured to automatically grasp and remove the second metal alloy pieces from the first pile in response to the second classification, and the separation device is configured to deposit the second metal alloy pieces onto a third pile of metal alloy pieces.

5. 5. The apparatus of claim 4, wherein the third pile is a homogenous collection of metal alloy pieces having the second metal alloy composition.

6. 10. The apparatus of claim 1, wherein the artificial intelligence system is comprised of a neural network that uses one or more algorithms to compare features detected in the captured images with those stored in a knowledge base generated during a training phase, during which the one or more algorithms learn relationships between one or more specified classes of metal alloys and their features extracted from the captured image data creating the knowledge base.

7. The apparatus of claim 1 , wherein the camera is configured to capture a visual image of the individual metal alloy pieces.

8. 10. The apparatus of claim 1, wherein the first metal alloy piece is a steel alloy piece.

9. 3. The apparatus of claim 2, wherein the separating device is a robotic arm having a controller that receives instructions from the data processing system to identify a first location of the first metal alloy pieces within the first pile and to identify a second location of the second pile.

10. 10. The apparatus of claim 9, wherein the first and second piles are located within a scrap metal yard.

11. a sensor configured to capture one or more characteristics of each of a mixture of particles contained within the first pile of particles, the mixture of particles including particles having different chemical compositions; a data processing system configured to assign a first classification to a first piece of material having a first chemical composition in response to processing the captured one or more characteristics of the first piece of material; a separation device configured to automatically grasp and remove the first piece of material from the first pile in response to the first classification; An apparatus comprising:

12. 12. The apparatus of claim 11, wherein the separating device is a robotic arm having a controller that receives instructions from the data processing system to identify a first location of the first piece of material within the first pile and to identify a second location of a second pile, and the sensor is an x-ray fluorescence system mounted on the arm of the separating device.

13. 12. The apparatus of claim 11, wherein the sensor is a camera configured to capture a visual image of the piece of material, and the data processing system is implemented with an artificial intelligence ("AI") system, the AI ​​system configured to assign a first classification to a first piece of material having a first chemical composition responsive to processing the captured visual image of the first piece of material through the AI ​​system.

14. 14. The apparatus of claim 13, wherein the AI ​​system is configured to assign a second classification to a second piece of material having a second chemical composition in response to processing the captured visual image of the second piece of material through the AI ​​system, the separation device is configured to automatically grasp and remove the second piece of material from the first pile in response to the second classification, the first and second classifications being different from one another, the separation device is configured to deposit the first piece of material onto a second pile of pieces of material, and the separation device is configured to deposit the second piece of material onto a third pile of pieces of material.

15. 15. The apparatus of claim 14, wherein the second pile is a homogenous collection of material pieces having the first chemical composition and the third pile is a homogenous collection of material pieces having the second chemical composition.

16. 14. The apparatus of claim 13, wherein the pieces of material are scrap metal alloy pieces, the first pieces of material are first scrap metal alloy pieces, the first and second piles are located within a scrap metal yard, and the separation device is configured to deposit the first scrap metal alloy pieces onto the second pile of scrap metal alloy pieces.

17. 17. The apparatus of claim 16, wherein the separating device is a robotic arm having a controller that receives instructions from the data processing system to identify a first location of the first scrap metal alloy pieces within the first pile and to identify a second location of the second pile.

18. 18. The apparatus of claim 17, wherein the AI ​​system is comprised of a neural network that uses one or more algorithms to compare features detected in the captured images with those stored in a knowledge base generated during a training phase, during which the one or more algorithms learn relationships between one or more specified classes of scrap metal alloys and their features extracted from the captured image data creating the knowledge base.

Citation Information

Patent Citations

  • Recycling coins from scrap metal

    JP2021522070A

  • US10,207,296

  • Sorting of plastics

    US11969764B2

  • WO99/2248