Removal of Airbag Modules from Automobile Scrap
Artificial intelligence and spectral imaging are employed to detect and separate airbag modules from automotive scrap, addressing the safety and identification challenges, ensuring efficient recycling.
Patent Information
- Application Number
- JP2023560679
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-05-24
- Filing Date
- 2022-08-05
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-08-05
AI Technical Summary
The challenge in recycling automobile scrap lies in the safe and efficient removal of airbag modules, which contain toxic sodium azide and can explode, posing risks to conveyor systems and equipment, and are difficult to identify and separate due to their small size and varied shapes.
Utilizing artificial intelligence techniques, including region proposal neural networks and deep neural networks, for detection and classification of airbag modules, combined with spectral imaging and sensor systems, to identify and separate airbag modules from a heterogeneous mixture of scrap materials on conveyor belts.
Effectively identifies and separates airbag modules from automotive scrap, preventing explosions and ensuring safe handling, while maintaining the integrity of the recycling process.
Smart Images

Figure 0007584678000001 
Figure 0007584678000002 
Figure 0007584678000003
Abstract
Description
[Technical field]
[0001] [Related patents and patent applications] This application claims priority to U.S. Provisional Patent Application No. 63 / 229,724. This application is a continuation-in-part of U.S. Patent Application No. 17 / 752,669, which is a continuation-in-part of U.S. Patent Application No. 17 / 667,397, which is a continuation-in-part of U.S. Patent Application No. 17 / 495,291, which is a continuation-in-part of U.S. Patent Application No. 17 / 491,415 (issued as U.S. Patent No. 11,278,937), which is a continuation-in-part of U.S. Patent Application No. 17 / 380,928, which is a continuation-in-part of U.S. Patent Application No. 17 / 227,245, which is a continuation-in-part of U.S. Patent Application No. 16 / 939,011 No. 16 / 375,675 (issued as U.S. Patent No. 10,722,922), which is a continuation-in-part of U.S. Patent Application No. 15 / 963,755 (issued as U.S. Patent No. 10,710,119), which is a continuation-in-part of U.S. Patent Application No. 15 / 213,129 (issued as U.S. Patent No. 10,207,296), which claims priority to U.S. Provisional Patent Application No. 62 / 193,332, all of which are incorporated herein by reference. U.S. Patent Application No. 17 / 491,415 (issued as U.S. Patent No. 11,278,937) is a continuation-in-part of U.S. Patent Application No. 16 / 852,514 (issued as U.S. Patent No. 11,260,426), which is a divisional application of U.S. Patent Application No. 16 / 358,374 (issued as U.S. Patent No. 10,625,304), which is a continuation-in-part of U.S. Patent Application No. 15 / 963,755 (issued as U.S. Patent No. 10,710,119), which claims priority to U.S. Provisional Patent Application No. 62 / 490,219, all of which are incorporated herein by reference.
[0002] [Government Licensing Rights] This disclosure was made with U.S. Government support under Grant No. DE-AR0000422 awarded by the U.S. Department of Energy. The U.S. Government may have certain rights in this disclosure.
[0003] FIELD OF THEINVENTION This invention relates to recycling automobile scrap, and more particularly to removing airbag modules from automobile scrap. [Background technology]
[0004] This section is intended to introduce various aspects of technology that may be related to exemplary embodiments of the present disclosure. The discussion is believed to be helpful in providing a framework to facilitate a better understanding of certain aspects of the present disclosure. As such, it should be understood that this section is to be read in this light, and not necessarily as admissions of prior art.
[0005] Recycling is the process of collecting and treating materials that would otherwise be discarded as garbage, and turning them into new products. Recycling benefits local communities and the environment by reducing the amount of waste sent to landfills and incinerators, conserving natural resources, increasing economic security by utilizing domestic material sources, preventing pollution by reducing the need to collect new raw materials, and saving energy.
[0006] Scrap metals are often shredded and must be separated to facilitate the reuse of the metals. Sorting scrap metals reuses metals that may otherwise be sent to landfills. In addition, the use of separated scrap metals leads to reduced pollution and emissions compared to refining raw materials from their ores. If the quality of the separated metals meets certain standards, manufacturers can use scrap metals in place of raw materials. Scrap metals can include ferrous and non-ferrous metals, heavy metals, valuable metals such as nickel and titanium, cast or wrought metals, and various other alloys.
[0007] An estimated 15 million vehicles (referred to as end-of-life vehicles) are scrapped each year in the United States. Each vehicle may contain multiple (e.g., 6-15) airbag modules, meaning that over 90 million airbag modules could enter the automotive recycling stream each year. An airbag module typically consists of three main parts enclosed within some sort of container or canister: the airbag, the inflator, and the propellant.
[0008] The problem that arises with all these airbag modules is that they contain toxic sodium azide, which is used for inflation. Furthermore, not all airbag modules inflate / explode as they pass through the vehicle shredder. As a result, these airbag modules may inflate / explode in different locations with different results; i.e., on the conveyor system, they can damage the conveyor belt, if handled by humans, they can cause serious injury or loss of limb, and once sold to customers from the recycling facility, they can damage the customer's equipment.
[0009] There are technical challenges that must be overcome to ensure satisfactory removal of such airbag modules from vehicle scrap: airbag modules can be small (e.g., airbag modules often have a cylindrical form factor of 1 inch (2.54 cm) in diameter and 1 inch (2.54 cm) in height), airbag modules in the mixed scrap metal stream after shredding appear similar to other scrap metal, shredded airbag modules can be difficult to identify when mixed with other scrap pieces, airbag modules can become mixed and partially occluded with other scrap pieces during transport on a conveyor belt, and airbag modules come in a variety of shapes, sizes, and colors. [Prior art documents] [Patent documents]
[0010] [Patent Document 1] US Patent Application Publication No. 2022 / 0016675 [Non-patent literature]
[0011] [Non-Patent Document 1] Krizhevsky et al., "ImageNet Classification with Deep Convolutional Networks," Proceedings of the 25th International Conference on Neural Information Processing Systems, December 3-6, 2012, Lake Tahoe, NV [Non-Patent Document 2] LeCun et al., "Gradient-Based Learning Applied to Document Recognition," IEEE Proceedings, Institute of Electrical and Electronics Engineers (IEEE), November 1998. [Brief description of the drawings]
[0012] [Figure 1] 1 is a schematic diagram of a material handling system constructed in accordance with an embodiment of the present disclosure. [Figure 2A] FIG. 2 illustrates an exemplary representation of a control set for an airbag module used during the training phase. [Figure 2B] FIG. 2 illustrates an exemplary representation of a control set for an airbag module used during the training phase. [Figure 2C] FIG. 1 illustrates an exemplary representation of a control set of an airbag module used during the training phase in which the algorithm identified and classified the airbag modules. [Diagram 3] FIG. 1 illustrates a flow chart configured in accordance with an embodiment of the present disclosure. [Figure 4] FIG. 1 illustrates a flow chart configured in accordance with an embodiment of the present disclosure. [Diagram 5] FIG. 1 is a block diagram of a data processing system configured in accordance with an embodiment of the present disclosure. [Figure 6A] FIG. 1 illustrates an exemplary representation of a heterogeneous mixture of material pieces that comprise an airbag module. [Figure 6B]FIG. 1 illustrates an exemplary representation of a heterogeneous mixture of material pieces that comprise an airbag module. [Figure 7] FIG. 1 illustrates an exemplary representation of a heterogeneous mixture of material pieces including airbag modules where an artificial intelligence algorithm has identified and / or classified the airbag modules. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Various detailed embodiments of the present disclosure are disclosed herein. However, it should be understood that the disclosed embodiments are merely exemplary of the present disclosure, which may be embodied in various alternative forms. The numerical values are not necessarily to scale, and some features may be exaggerated or minimized to show the details of specific components. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to use various embodiments of the present disclosure.
[0014] Embodiments of the present disclosure utilize artificial intelligence techniques for identification / classification of airbag modules in the scrap stream. According to certain embodiments of the present disclosure, pieces of material can be separated on a conveyor belt with spaces between pieces using any standard computer vision method. According to certain embodiments of the present disclosure, a region proposal neural network can be utilized for airbag module detection and / or a deep neural network can be utilized for airbag module classification. According to certain embodiments of the present disclosure, these two neural networks for detection and classification can be combined. Embodiments of the present disclosure can use semantic segmentation or object detection / localization. Alternatively, instance segmentation or panoptic segmentation can be utilized. Embodiments of the present disclosure can use pixel-level, neighborhood, region, and / or whole image classification.
[0015] As used herein, "material" includes metals (ferrous and non-ferrous), alloys, pieces of metal embedded in another, different material, plastics (including, but not limited to, those disclosed herein, known in the industry, or hereafter created), rubber, foam, glass (including, but not limited to, borosilicate glass or soda lime glass, and various colored glasses), ceramics, paper, cardboard, Teflon, PE, bundled wire, insulated wire, rare earth elements, leaves, wood, plants, plant parts, fibers, bio-waste, packaging, electronic waste, batteries and accumulators, automotive scrap from shredded vehicles, mining, construction and demolition waste, agricultural crop waste, forest residues. , cultivated grasses, woody energy crops, microalgae, urban food waste, food waste, hazardous chemical and biomedical waste, construction debris, farm waste, biogenic items, non-biogenic items, objects with a particular carbon content, other objects that may be found in municipal solid waste, and any other objects, articles, or materials disclosed herein, including further types or classes of any of the foregoing that can be distinguished from one another by one or more sensor systems (including but not limited to), including but not limited to any of the sensor technologies disclosed herein.
[0016] In a more general sense, "material" may include any item or object composed of a chemical element, a compound or mixture of one or more chemical elements, or a compound or mixture of compounds or mixtures of chemical elements, the complexity of which may range from simple to complex (all of which may also be referred to herein as materials having a particular "chemical composition"). "Chemical element" means a chemical element of the periodic table of chemical elements, including chemical elements that may be discovered after the filing date of this application. Within this disclosure, the terms "scrap," "scrap piece," "material," and "material piece" may be used interchangeably. As used herein, material piece or scrap piece referred to as having a metal alloy composition is a metal alloy having a particular chemical composition that is distinct from other metal alloys.
[0017] As used herein, the term "predetermined" refers to something that is established or determined in advance, including but not limited to, by a user or operator of the sorting system disclosed herein.
[0018] As used herein, "spectral imaging" is imaging using multiple bands across the electromagnetic spectrum. A typical camera captures light across three wavelength bands, red, green, and blue ("RGB") in the visible spectrum, but spectral imaging can include a variety of techniques including and beyond RGB. For example, spectral imaging can use infrared, visible, ultraviolet, and / or X-ray spectrum, or combinations of the above. Spectral data, or spectral image data, is a digital data representation of a spectral image. Spectral imaging can include simultaneous acquisition of spectral data in visible and non-visible bands, illumination from outside the visible range, or use of optical filters to capture specific spectral ranges. It is also possible to capture hundreds of wavelength bands for each pixel of a spectral image.
[0019] As used herein, the term "image data packet" refers to a packet of digital data relating to a captured spectral image of an individual piece of material.
[0020] As used herein, the terms "identify" and "classify", "identification" and "classification", and their derivatives, may be used interchangeably. As used herein, "classifying" a piece of material means determining (i.e., identifying) the type or class of material to which the piece of material belongs. For example, according to certain embodiments of the present disclosure, a sensor system (described further herein) may be configured to collect and analyze any type of information to classify materials, and classification may be utilized within a sorting system to selectively separate pieces of material according to a set of one or more physical and / or chemical properties (e.g., user definable), including, but not limited to, color, texture, hue, shape, brightness, weight, density, chemical composition, size, uniformity, manufacturing type, chemical characteristics, predetermined fractions, radioactive characteristics, transmittance of light, sound, or other signals, and response to stimuli such as various fields, including radiative and / or reflected electromagnetic radiation ("EM") of the piece of material.
[0021] The material types or classes (i.e., classifications) are user definable and are not limited to known material classifications. The granularity of the types or classes can range from very coarse to very fine. For example, the types or classes can include relatively coarse types or classes of plastics, ceramics, glasses, metals, and other materials, finer types or classes of various metals and metal alloys, such as zinc, copper, brass, chrome plate, aluminum, or relatively fine types or classes of plastics of a particular type. Thus, the types or classes can be configured to distinguish between materials of significantly different chemical compositions, such as plastics and metal alloys, or between materials of nearly identical chemical compositions, such as different types of metal alloys. It should be understood that the methods and systems discussed herein can be applied to accurately identify / classify materials whose chemical composition is completely unknown before being classified.
[0022] A "conveyor system" as referred to herein may refer to any known mechanical handling device that moves material from one location to another, including, but not limited to, aero-mechanical conveyors, automotive conveyors, belt conveyors, belt driven live roller conveyors, bucket conveyors, chain conveyors, chain driven live roller conveyors, drag conveyors, dust-tight conveyors, motorized tracked vehicle systems, flexible conveyors, gravity conveyors, gravity skate wheel conveyors, line shaft roller conveyors, motorized driven roller conveyors, overhead I-beam conveyors, overland conveyors, pharmaceutical conveyors, plastic belt conveyors, pneumatic conveyors, screw or auger conveyors, spiral conveyors, tubular gallery conveyors, vertical conveyors, free fall conveyors, vibrating conveyors, wire mesh conveyors, robotic arm manipulators.
[0023] The systems and methods described herein according to certain embodiments of the present disclosure receive a heterogeneous mixture of multiple pieces of material, where at least one piece of material in the heterogeneous mixture comprises a different elemental composition than one or more other pieces of material, and / or at least one piece of material in the heterogeneous mixture is physically distinguishable from the other pieces of material, and / or at least one piece of material in the heterogeneous mixture is a different class or type of material from the other pieces of material in the mixture, and the systems and methods are configured to identify / classify / separate / sort the one piece of material into a separate group from the other pieces of material. The embodiments of the present disclosure may be utilized to classify any type or class of material defined herein. In contrast, a set or group of homogeneous materials are all classified into an identifiable class or type of material (or a specified number of identifiable classes or types of material), such as a live airbag module.
[0024] Embodiments of the present disclosure may be described herein as sorting pieces of material into separate groups according to user-defined groupings (e.g., material type, classification, etc.) by physically depositing (e.g., diverting or discharging) the pieces into separate containers or bins. As an example, in certain embodiments of the present disclosure, pieces of material may be sorted into separate containers to separate pieces of material classified as belonging to a particular class or type of material (e.g., live airbag modules) that are distinguishable from other pieces of material (e.g., those classified as belonging to a different class or type of material).
[0025] It should be noted that the material to be separated may be of irregular sizes and shapes. For example, such material may have been pre-run through some type of shredding mechanism that chops the material into pieces of such irregular shapes and sizes (producing scrap pieces) and then fed or redirected to a conveyor system. According to an embodiment of the present disclosure, the material pieces include automotive scrap pieces of vehicles that have been run through some type of shredding mechanism, and the automotive scrap pieces include non-detonated (i.e., non-inflated or non-exploded) airbag modules, also referred to herein as "live airbag modules."
[0026] FIG. 1 illustrates an example of a system 100 configured in accordance with various embodiments of the present disclosure. A conveyor system 103 may be implemented to transport the individual pieces of material 101 through the system 100, so that each of the individual pieces of material 101 can be tracked, classified, differentiated, and separated into predetermined desired groups. Such a conveyor system 103 may be implemented with one or more conveyor belts along which the pieces of material 101 typically move at a predetermined constant speed. However, certain embodiments of the present disclosure may be implemented with other types of conveyor systems, including systems in which the pieces of material free-fall through various components of the system 100 (or any other type of vertical separator), or vibrating conveyor systems. Hereinafter, where applicable, the conveyor system 103 may also be referred to as a conveyor belt 103. In one or more embodiments, some or all of the operations or functions of communicating, capturing, stimulating, detecting, classifying, differentiating, and separating may be performed automatically, i.e., without human intervention. For example, in system 100, one or more cameras, one or more stimulus sources, one or more radiation detectors, classification modules, sorting devices / apparatus, and / or other system components may be configured to perform these and other operations automatically.
[0027] Additionally, while FIG. 1 illustrates a single stream of pieces of material 101 on the conveyor system 103, embodiments of the present disclosure may be implemented such that multiple such streams of pieces of material pass through various components of the system 100 in parallel with one another. For example, as further described in U.S. Pat. No. 10,207,296, the pieces of material may be distributed into two or more parallel, singulated streams moving on a single conveyor belt, or a set of parallel conveyor belts. Thus, certain embodiments of the present disclosure may simultaneously track, sort, and separate multiple such parallel moving streams of pieces of material. In accordance with certain embodiments of the present disclosure, the incorporation or use of a singulator is not required. Instead, the conveyor system (e.g., conveyor belt 103) may simply transport a collection of pieces of material deposited on the conveyor system 103 in a random manner.
[0028] According to certain embodiments of the present disclosure, some suitable feeder mechanism (e.g., another conveyor system or hopper 102) may be utilized to feed the pieces of material 101 onto the conveyor system 103, which may then transport the pieces of material 101 through various components within the system 100. After the pieces of material 101 are received by the conveyor system 103, an optional tumbler / vibrator / singulator 106 may be utilized to separate individual pieces of material from the mass of pieces of material. In certain embodiments of the present disclosure, the conveyor system 103 is operated to move at a predetermined speed by a conveyor system motor 104. This predetermined speed may be programmable and / or adjustable by an operator in any known manner. Alternatively, monitoring of the predetermined speed of the conveyor system 103 may be performed using a position detector 105. In certain embodiments of the present disclosure, control of the conveyor system motor 104 and / or the position detector 105 may be performed by an automatic control system 108. Such an automated control system 108 may operate under the control of a computer system 107 and / or the functions for performing the automated control may be implemented in software within the computer system 107 .
[0029] The conveyor system 103 may be a conventional endless belt conveyor that uses a conventional drive motor 104 suitable for moving the belt conveyor at a predetermined speed. The position detector 105 may be a conventional encoder and may be operatively coupled to the conveyor system 103 and the automatic control system 108 to provide information corresponding to the movement (e.g., speed) of the conveyor belt. Thus, through the utilization of control over the conveyor system drive motor 104 and / or the automatic control system 108 (and alternatively including the position detector 105), as each of the pieces of material 101 moving on the conveyor system 103 are identified, they may be tracked by position and time (relative to the various components of the system 100) so that various components of the system 100 may be activated / deactivated as each piece of material 101 passes nearby. As a result, the automatic control system 108 may track the position of each of the pieces of material 101 as they move along the conveyor system 103.
[0030] Referring again to FIG. 1 , certain embodiments of the present disclosure may utilize a vision or optical recognition system 110 and / or a piece of material tracker 111 as a means of tracking each of the pieces of material 101 moving on the conveyor system 103. The vision system 110 may utilize one or more still or live action cameras 109 to record the position (i.e., location and timing) of each of the pieces of material 101 on the moving conveyor system 103. The vision system 110 may additionally or alternatively be configured to perform a particular type of identification (e.g., classification) of all or a portion of the pieces of material 101, as described further herein. For example, such a vision system 110 may be utilized to capture or obtain information about each of the pieces of material 101. For example, the vision system 110 may be configured (e.g., using an artificial intelligence ("AI") system) to capture or gather any type of information from the pieces of material available within the system 100 in order to classify / distinguish and selectively separate the pieces of material 101 according to a set of one or more characteristics (e.g., physical and / or chemical and / or radioactive materials, etc.) as described herein. According to certain embodiments of the present disclosure, the vision system 110 may be configured to capture visual images (including one-dimensional, two-dimensional, three-dimensional, or holographic images) of each of the pieces of material 101, for example, by using optical sensors used in common digital cameras and video equipment. Such visual images captured by the optical sensors are stored in a memory device as spectral image data (e.g., initialized as image data packets). According to certain embodiments of the present disclosure, such spectral image data may represent images captured within optical wavelengths of light (i.e., wavelengths of light observable by a typical human eye). However, alternative embodiments of the present disclosure may utilize sensor systems configured to capture images of materials comprised of wavelengths of light outside the visual wavelengths of the human eye.
[0031] According to alternative embodiments of the present disclosure, the system 100 may be implemented with one or more sensor systems 120 that may be utilized alone or in combination with the vision system 110 to classify / identify / differentiate the pieces of material 101. The sensor system 120 may be configured with any type of sensor technology, including sensors that utilize emitted or reflected electromagnetic radiation (e.g., infrared ("IR"), Fourier transform IR ("FTIR"), forward infrared ("FLIR"), very near infrared ("VNIR"), near infrared ("NIR"), short wavelength infrared ("SWIR"), long wavelength infrared ("LWIR"), mid wavelength infrared ("MWIR" or "MIR"), x-ray transmission ("XRT"), gamma ray, ultraviolet ("UV"), x-ray fluorescence ("XRF"), laser induced breakdown spectroscopy ("LIBS"), Raman spectroscopy, anti-Stokes Raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., beyond visible wavelengths), acoustic spectroscopy, NMR spectroscopy, microwave spectroscopy, terahertz spectroscopy, including one-dimensional, two-dimensional, or three-dimensional imaging using any of the foregoing), or with other types of sensor technologies, including, but not limited to, chemical or radioactive materials. Implementation of XRF systems (e.g., for use as sensor system 120 herein) is further described in U.S. Patent No. 10,207,296. Accordingly, it should be noted that in certain contexts of the description herein, references to sensor systems may refer to vision systems. Nonetheless, any of the vision and sensor systems disclosed herein may be configured to collect or capture information (e.g., characteristics) specifically related to each of the pieces of material, such that the captured information may be utilized to identify / classify / distinguish particular ones of the pieces of material.
[0032] According to certain embodiments of the present disclosure, multiple optical cameras and / or sensor systems, including at different angles, can be used to help identify live airbag modules that are partially obstructed or substantially or completely obstructed by other material on the conveyor system. According to certain embodiments of the present disclosure, multiple cameras and / or sensor systems can be used to create 3D information to generate more useful information than is possible with 2D data. The 2D or 3D data can be used in AI systems to gather data.
[0033] According to certain embodiments of the present disclosure, a Lidar system ("Light Detection and Ranging" or "Laser Imaging, Detection, and Ranging") can be used instead of a camera and / or sensor system. According to certain embodiments of the present disclosure, a scanning laser can be used to collect 3D data of the scrap stream. The laser-based 3D data is used in a neural network to identify live airbag modules.
[0034] It should be noted that while FIG. 1 is shown with a combination of a vision system 110 and one or more sensor systems 120, embodiments of the present disclosure may be implemented with any combination of sensor systems utilizing any of the sensor technologies disclosed herein or other sensor technologies currently available or developed in the future. Although FIG. 1 is shown including one or more sensor systems 120, such sensor system implementation is optional within certain embodiments of the present disclosure. In certain embodiments of the present disclosure, a combination of both a vision system 110 and one or more sensor systems 120 may be used to classify the piece of material 101. In certain embodiments of the present disclosure, any combination of one or more different sensor technologies disclosed herein may be used to classify the piece of material 101 without utilizing a vision system 110. Additionally, embodiments of the present disclosure may include any combination of one or more sensor systems and / or vision systems where the output of the sensor / vision system may be processed within an AI system (as further disclosed herein) to classify / identify materials from a heterogeneous mixture of materials, which may then be separated from one another.
[0035] In certain embodiments of the present disclosure, the material piece tracker 111 and associated control system 112 may be utilized and configured to measure the position (i.e., location and timing) of each material piece 101 on the moving conveyor system 103 as well as the size and / or shape of each of the material pieces 101 passing in proximity to the material piece tracker 111. Exemplary operation of such material piece tracker 111 and control system 112 is further described in U.S. Pat. No. 10,207,296. Alternatively, as previously disclosed, the vision system 110 may be utilized to track the position (i.e., location and timing) of each of the material pieces 101 transported by the conveyor system 103. Accordingly, certain embodiments of the present disclosure may be implemented without a material piece tracker (e.g., material piece tracker 111) for tracking the material pieces.
[0036] Such a distance measuring device 111 may be implemented using well-known visible light (e.g., laser light) systems that continuously measure the distance light travels before being reflected to a detector of the laser light system. Thus, as each piece of material 101 passes in the vicinity of the device 111, the device 111 outputs a signal indicative of such distance measurement to the control system 112. Such a signal may thus represent a substantially intermittent series of pulses, whereby a baseline of the signal is generated as a result of measuring the distance between the distance measuring device 111 and the conveyor belt 103 when no piece of material 101 is in the vicinity of the device 111, with each pulse providing a measurement of the distance between the distance measuring device 111 and the piece of material 101 passing on the conveyor belt 103.
[0037] In certain embodiments of the present disclosure implementing one or more sensor systems 120, the sensor systems 120 may be configured to assist the vision system 110 in identifying the chemical composition, relative chemical composition, and / or manufacturing type of each of the pieces of material 101 as they pass in the vicinity of the sensor system 120. The sensor system 120 may include, for example, an energy emitting source 121 that may be powered by a power source 122 to stimulate a response from each of the pieces of material 101.
[0038] In certain embodiments of the present disclosure, the sensor system 120 can emit an appropriate sensing signal towards the piece of material 101 as each piece of material 101 passes in the vicinity of the radiation source 121. The one or more detectors 124 can be arranged and configured to sense / detect one or more characteristics from the piece of material 101 in a format appropriate for the type of sensor technology employed. The one or more detectors 124 and associated detector electronics 125 capture these received sensed characteristics and perform signal processing to generate digitized information (e.g., spectral data) representative of the sensed characteristics, which can then be analyzed and used to classify each of the pieces of material 101 according to certain embodiments of the present disclosure. This classification can be performed within the computer system 107 and can then be utilized by the automatic control system 108 to operate one of the N (N>1) sorting devices 126...129 of the sorting device to sort (e.g., remove / redirect / eject) the piece of material 101 into one or more N (N>1) sorting containers 136...139 according to the determined classification. Four sorting devices 126...129 and four sorting bins 136...139 associated with these sorting devices are shown in FIG. 1 by way of non-limiting example only.
[0039] As described herein, embodiments of the present disclosure are configured to identify live airbag modules within a moving stream of scrap pieces (e.g., to distinguish live airbag modules from other automotive scrap pieces) and separate these live airbag modules for removal / redirection / discharge from the conveyor system.
[0040] The sorting device may include any known sorting mechanism for removing / redirecting / discharging selected pieces of material 101 identified as live airbag modules toward a desired location, including but not limited to diverting the pieces of material 101 from the belt conveyor system to one or more sorting bins. According to certain embodiments of the present disclosure, since it may be more important to remove / redirect / discharge the live airbag module even if it means losing one or more other pieces of material from the remaining scrap stream, the sorting mechanism for removing / redirecting / discharging the live airbag module from the conveyor system may be configured to remove / redirect / discharge the airbag from the conveyor system regardless of whether other pieces of material in the vicinity of the live airbag module are also removed, redirected, or discharged from the conveyor system along with the live airbag module. For example, with reference to any of Figures 6A, 6B, or 7, it is readily apparent that there are other scrap pieces in the vicinity of the live airbag module. In such cases, other scrap pieces in the vicinity of the classified live airbag module may be diverted from the conveyor belt into a designated bin along with the classified live airbag module, as it is more important to remove the live airbag module from the scrap piece stream than to risk the sorting mechanism not being sufficiently accurate in its diverting action, thereby attempting to divert only the classified live airbag module, which would result in the classified live airbag module not being removed from the scrap piece stream.
[0041] Mechanisms that can be used to remove / redirect / eject pieces of material include robotically removing the pieces of material from the conveyor belt, pushing the pieces of material off the conveyor belt (e.g., using a paintbrush type plunger), creating an opening in the conveyor system 103 through which the pieces of material can fall (e.g., a trap door), or using air jets to separate the pieces of material into separate bins as they fall off the end of the conveyor belt. The term pusher device, as used herein, can refer to any form of device that can be actuated to dynamically move objects on or off the conveyor system / apparatus using pneumatic, mechanical, hydraulic, vacuum actuators, or other means, such as any suitable type of mechanical pushing mechanism (e.g., ACME screw drive), pneumatic pushing mechanism, or air jet pushing mechanism.
[0042] According to certain embodiments of the present disclosure, the live airbag module may need to be removed / redirected / ejected from the conveyor system in a relatively "gentle" manner so that the live airbag module is not activated and inflated / exploded. According to certain embodiments of the present disclosure, any technique for removing / redirecting / ejecting the live airbag module from the conveyor system may be utilized, where the force with which the removal / redirecting / ejection is performed is configured to not result in activation of the live airbag module, as the live airbag module would inflate or explode. For example, sorting may be performed by a sorting mechanism that redirects the live airbag module into a container using a redirecting force configured to not activate the live airbag module. Thus, the sorting mechanism may be configured to utilize less force than is known to activate such live airbag module, while diverting the live airbag module from the conveyor belt with sufficient force to move the live airbag module. Of course, this may be determined using trial and error. According to certain embodiments of the present disclosure, such a sorting mechanism may be a paintbrush type plunger.
[0043] Robotic removal can be performed by any suitable robotic arm, such as a Stewart platform, a delta robot, a multi-jaw gripper, or the like.
[0044] In addition to the N sorting bins 136...139 from which the material pieces 101 (e.g., live airbag modules) are removed / diverted / discharged, the system 100 may also include a bin 140 that receives material pieces 101 (e.g., remaining automobile scrap pieces) that are not diverted / discharged from the conveyor system 103 into any of the aforementioned sorting bins 136...139.
[0045] Depending on the various classifications of the desired pieces of material, multiple classifications may be mapped to a single sorting device and associated sorting bin. In other words, there need not be a one-to-one correlation between classifications and sorting bins. For example, a user may desire to separate certain classifications of material (e.g., live airbag modules and other types of material) into the same sorting bin. To accomplish this separation, when pieces of material 101 are classified as falling into a given classification group, the same sorting device may operate to separate them into the same sorting bin. Such combination separations may be applied to generate any desired combination of separated pieces of material. The classification mapping may be programmed by the user (e.g., using an algorithm operated by the computer system 107) to generate such desired combinations. Furthermore, the classifications of the pieces of material are user definable and are not limited to specific known classifications of the pieces of material.
[0046] The conveyor system 103 may include a carousel (not shown) such that unsorted pieces of material are returned to the beginning of the system 100 and passed through the system 100 again. Additionally, because the system 100 can specifically track each piece of material 101 moving on the conveyor system 103, some sorting device (e.g., sorting device 129) may be implemented to remove / direct / eject the pieces of material 101 so that the system 100 does not fail to sort after a predetermined number of cycles by the system 100 (or the pieces of material 101 are collected in the container 140) (e.g., pieces of material that were not classified as live airbag modules by a predetermined threshold, but the user wants to classify as live airbag modules pieces of material that have been assigned a particular value for the live airbag module classification below a predetermined threshold to increase the likelihood that all or substantially all of the live airbag modules will be removed / directed / ejected).
[0047] As illustrated in FIGS. 2A-2C, the systems and methods described herein may be applied to sort and / or separate individual airbag modules having any of a variety of sizes.
[0048] As previously mentioned, certain embodiments of the present disclosure may implement one or more vision systems (e.g., vision system 110) to identify, track, and / or sort pieces of material. According to embodiments of the present disclosure, such vision systems may operate alone to identify and / or sort and separate pieces of material, or may operate in combination with a sensor system (e.g., sensor system 120) to identify and / or sort and separate pieces of material. When a system (e.g., system 100) is configured to operate with only such vision system 110, sensor system 120 may be omitted from system 100 (or may simply be turned off).
[0049] Such a vision system may be configured with one or more devices for capturing or acquiring images of the pieces of material passing on the conveyor system. The devices may be configured to capture or acquire any desired wavelength range emitted or reflected by the pieces of material, including but not limited to visible, infrared ("IR"), and ultraviolet ("UV") light. For example, the vision system may be configured with one or more cameras (which may be configured to capture still and / or video, both of which may be configured to capture two-dimensional, three-dimensional, and / or holographic images) positioned near (e.g., on) the conveyor system such that images of the pieces of material are captured as they pass through the sensor system. According to alternative embodiments of the present disclosure, data captured by the sensor system 120 may be processed (converted) into data that is utilized (either alone or in combination with image data captured by the vision system 110) for classification / segregation of the pieces of material. Such implementations may be in lieu of or in combination with the utilization of the sensor system 120 for classifying the pieces of material.
[0050] Regardless of the type of sensed feature / information of the piece of material, the information may then be transmitted to a computer system (e.g., computer system 107) and processed (e.g., by an AI system) to identify and / or classify the piece of material. The AI system may be any known AI system (e.g., artificial intelligence ("ANI"), artificial general intelligence ("AGI"), and artificial superintelligence ("ASI")), or derivatives thereof yet to be developed, machine learning systems including those implementing neural networks (e.g., artificial neural networks, deep neural networks, convolutional neural networks, recurrent neural networks, autoencoders, reinforcement learning, etc.), supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, self-learning, feature learning, sparse dictionary learning, anomaly detection, robotic learning, correlation, etc. Machine learning systems may implement rule learning, fuzzy logic, deep learning algorithms, hierarchical learning algorithms for deep structural learning, extreme learning machines, support vector machines ("SVM") (e.g., linear SVM, non-linear SVM, SVM regression), decision tree learning (e.g., classification and regression trees ("CART")), ensemble techniques (e.g., ensemble learning, random forests, bagging and pasting, patch and subspace, boosting, stacking), dimensionality reduction (e.g., projection, manifold learning, principal component analysis, etc.), and / or deep machine learning algorithms.Non-limiting examples of publicly available machine learning software and libraries that can be utilized within embodiments of the present disclosure include Python, OpenCV, Inception, Theano, Torch, PyTorch, Pylearn2, Numpy, Blocks, TensorFlow, MXNet, Caffe, Lasagne, Keras, Chainer, Matlab Deep Learning, CNTK, MatConvNet (a MATLAB toolbox implementing convolutional neural networks for computer vision applications), DeepLearnToolbox (a Matlab toolbox for deep learning (by Rasmus Berg Palm)), BigDL, Cuda-Convnet (a fast C++ / CUDA implementation of convolutional (more generally feedforward) neural networks), Deep Belief Networks, RNNLM, RNNLIB-RNNLIB, matrbm, deeplearning4j, Eblearn.lsh, deepmat, MShadow, Matplotlib, SciPy, CXXNET, Nengo-Nengo, Eblearn, cudamat, Gnumpy, 3-Way Factorized RBM and mcRBM, mPoT (Python code to train a model for natural images using CUDAMat and Gnumpy), ConvNet, Elektronn, OpenNN, NeuralDesigner, Theano Generalized Hebbian Learning, Apache Singa, Lightnet, and SimpleDNN.
[0051] According to an embodiment of the present disclosure, the identification and / or classification of each of the pieces of material 101 may be performed by an AI system implementing semantic segmentation, although other implementations may be utilized, such as image segmentation such as Mask R-CNN (e.g., Python code), panoptic segmentation, instance segmentation, block segmentation, bounding box algorithms, etc.
[0052] Image segmentation can identify / classify pieces of material that are partially occluded by other pieces of material. Figures 6A and 7 show example images of overlapping pieces of material in which one or more live airbag modules are partially occluded but can be identified / classified by an embodiment of the present disclosure as a live airbag module (as shown in Figure 7) and thus distinguished from other automotive scrap pieces, such as if the AI system implemented some form of image segmentation algorithm.
[0053] The configuration of an AI system often occurs in multiple stages. For example, training occurs first, which may be performed offline in that the system 100 is not utilized to perform the actual classification / segregation of the material pieces. The system 100 may be utilized (e.g., by conveyor system 103) to train the AI system in that a homogenous set (also referred to herein as a control sample) of material pieces (i.e., having the same type or class of material) may be passed through the system 100, and all such material pieces may not be separated or may be collected in a common container (e.g., container 140). Alternatively, training may be performed at another location remote from the system 100, including using some other mechanism to collect sensory information (characteristics) of a control set of material pieces. During this training phase, algorithms within the AI system extract features from the captured information (e.g., using image processing techniques well known in the art). Non-limiting examples of training algorithms include, but are not limited to, linear regression, gradient descent, feedforward, polynomial regression, learning curves, regularized learning models, and logistic regression. Additionally, training may include data curation, data organization, data labeling, constructing semi-synthetic data, generating synthetic data, data augmentation, and other activities related to preparing a "curriculum" (e.g., training or control set) to be taught to the AI system (e.g., off-machine training using separate equipment designed for the purpose, or "instrument-free" training (simulation, augmentation, etc.) conducted entirely within the computer's memory). It is during this training phase that algorithms within the AI system learn relationships between materials and their features / characteristics (e.g., as captured by the vision and / or sensor systems) to create a knowledge base for later classifying heterogeneous mixtures of material pieces received by the system 100 that can then be classified according to the desired classification.Such a knowledge base may include one or more libraries, each library including parameters (e.g., neural network parameters) utilized by the AI system in classifying material pieces. For example, a particular library may include parameters configured during a training phase to recognize and classify airbag modules. According to certain embodiments of the present disclosure, such libraries may be input to the AI system, and a user of the system 100 may be able to adjust certain of the parameters to adjust the operation of the system 100 (e.g., to adjust a threshold effectiveness of how well the AI system identifies / classifies and distinguishes live airbag modules from a mixture of materials (e.g., a moving stream of automotive scrap pieces)).
[0054] As shown in the exemplary images of FIGS. 2A-2B, during a training phase, one or more example live airbag modules, which may be referred to herein as one or more sets of control samples, may be delivered (e.g., by a conveyor system) past the vision system and / or one or more sensor systems to allow the algorithms in the AI system to detect, extract, and learn features representative of such type or class of material. For example, each of the live airbag modules may be passed through such a training phase so that the algorithms in the AI system "learn" (are trained) how to detect, recognize, and classify the live airbag modules (see FIG. 2C). When training a vision system (e.g., vision system 110), it is trained to visually identify pieces of material. This creates a library of parameters specific to the live airbag module. The same process may then be performed, for example, on a particular class or type of metal alloy (or a mixture of automotive scrap pieces that are not live airbag modules), creating a library of parameters specific to that class or type of metal alloy, etc. For each class or type of material to be classified by the system, any number of example pieces of material of that class or type may be passed through the vision system and / or one or more sensor systems. Given a captured image or other captured characteristics as input data, the AI algorithm might use N classifiers, each testing one of N different material classes or types.
[0055] After the algorithms are established and the AI system has sufficiently learned (trained) (e.g., within a user-defined statistical confidence level) the differences (e.g., visually discernible differences) in the material classifications, the library of different material classifications is then implemented into a material sorting / separation system (e.g., system 100) that is used to identify and / or classify material pieces (e.g., live airbag modules) from a heterogeneous mixture of material pieces (e.g., a stream of automotive scrap pieces) and then separate such classified material pieces.
[0056] Techniques for building, optimizing, and utilizing AI systems are known to those skilled in the art as described in the relevant literature, examples of which include "Intelligent Machine Learning: A Practical Guide to Machine Learning," in ...
[0057] In an exemplary technique, data captured by a vision or sensor system regarding a particular piece of material (e.g., an actual airbag module) may be processed (within a data processing system (e.g., data processing system 3400 of FIG. 10 ) that implements an AI system) as an array of data values. For example, the data may be spectral data captured by a digital camera or other type of sensor system regarding a particular piece of material and processed as an array of data values (e.g., image data packets). Each data value may be represented by a single number or as a series of numbers that represent a value. These values may be multiplied by weight parameters of a neuron (e.g., using a neural network) and a bias may be added. This may affect the nonlinearity of the neuron. The resulting number output by a neuron may be treated the same as a value by multiplying this output by the weight value of the subsequent neuron, optionally adding a bias, again affecting the nonlinearity of the neuron. Each iteration of such processing is known as a “layer” of the neural network. The final output of the final layer may be interpreted as a probability that material is present or absent in the captured data associated with the piece of material. Examples of such processing are described in detail in both the aforementioned references, "ImageNet Classification with Deep Convolutional Networks" and "Gradient-Based Learning Applied to Document Recognition." According to embodiments of the present disclosure, a bias can be configured such that the AI system classifies the automobile scrap piece as a live airbag module because the captured visual image of the automobile scrap contains visual features that make the captured visual image similar to a live airbag module. The bias may be configured such that false positives outnumber false negatives by a percentage greater than a predetermined threshold (e.g., 95%). A false positive is an instance where the classification results in the automobile scrap piece being identified as a live airbag module when in fact it is not (e.g., the automobile scrap piece physically resembles a live airbag module).A false negative is an instance where the classification results in the failure to identify a live airbag module. Because it is critical to remove live airbag modules from the automotive scrap stream, a predefined very high rate of false positives and false negatives configured into the classification algorithm of the AI system may be acceptable even if it results in other automotive scrap being removed from the scrap stream.
[0058] According to certain embodiments of the present disclosure where a neural network is implemented as the final layer ("classification layer"), a final set of neuron outputs is trained to represent the likelihood that a piece of material (such as an "airbag module") is associated with the captured data. In operation, a particular piece of material is determined to be indeed associated with the captured data if the likelihood that the piece of material is associated with the captured data exceeds a user-specified threshold. These techniques can be extended to determine not only the presence of a type of material associated with the particular captured data, but also whether a sub-region of the particular captured data belongs to one type of material or another. This process is known as segmentation, and there are techniques in the literature that use neural networks, such as neural networks known as "fully convolutional" neural networks and networks that are not fully convolutional but include convolutional parts (i.e., are partially convolutional). This allows the location and size of the material to be determined. Examples include Mask R-CNN, which implements image segmentation.
[0059] It should be understood that the present disclosure is not limited to AI techniques only. Other common techniques for material classification / identification can also be used. For example, a sensor system may utilize spectroscopy techniques using multispectral or hyperspectral cameras to provide a signal indicative of the presence or absence of a type of material (e.g., containing one or more specific elements) by examining the material's spectral emission (i.e., spectral imaging). Spectral images of a piece of material (e.g., an airbag module, etc.) may also be used in a template matching algorithm in which a database of spectral images is compared to the captured spectral images to detect the presence or absence of a particular type of material from the database. A histogram of a captured spectral image may also be compared to a database of histograms. Similarly, a bag-of-words model may be used in conjunction with feature extraction techniques such as scale-invariant feature transform ("SIFT") to compare the captured image to those in the database. According to certain embodiments of the present disclosure, instead of utilizing a training phase in which control samples of the pieces of material are passed by a vision and / or sensor system, training of the machine learning system can be performed utilizing labeling / annotation techniques (or other supervised learning techniques) whereby as data / information of the pieces of material is captured by the vision / sensor system, a user inputs labels or annotations that identify each piece of material (such as a live airbag module) and that are used to create a library for the machine learning system to use when classifying pieces of material within a heterogeneous mix of pieces of material. In other words, a pre-generated knowledge base of properties captured from one or more samples of a class of material can be achieved by any of the techniques disclosed herein, whereby such knowledge base is utilized to automatically classify materials.
[0060] Thus, as disclosed herein, certain embodiments of the present disclosure provide for identification / classification of one or more different types or classes of material to determine which pieces of material (e.g., live airbag modules) should be diverted from the conveyor system within a defined group. According to certain embodiments, AI techniques are utilized to train (i.e., configure) a neural network to identify various one or more different classes or types of material. Spectral images or other types of sensory information are captured from the material (e.g., moving on the conveyor system), and based on the identification / classification of such material, the system described herein can determine which pieces of material should remain on the conveyor system and which pieces of material should be diverted / removed from the conveyor system (e.g., either placed in a collection bin or diverted to another conveyor system).
[0061] One point to mention here is that according to certain embodiments of the present disclosure, the collected / captured / detected / extracted features / characteristics (e.g., spectral images) of the pieces of material do not necessarily have to be simply particularly identifiable or identifiable physical characteristics, they may be abstract formulas that can only be expressed mathematically, or may not be expressible mathematically at all, and nevertheless, the AI system may be configured to analyze the spectral data to look for patterns that can classify the control samples during the training phase. Furthermore, the machine learning system may take subsections of the captured information (e.g., spectral images, etc.) of the pieces of material and try to find correlations between predefined classifications.
[0062] According to certain embodiments of the present disclosure, instead of utilizing a training phase in which control samples of pieces of material are passed by a vision and / or sensor system, training of the AI system can be performed using labeling / annotation techniques (or other supervised learning techniques) whereby data / information about the pieces of material (e.g., a live airbag module) is captured by the vision / sensor system and a user inputs labels or annotations that identify each piece of material and are used to create a library for use by the AI system in classifying pieces of material within a heterogeneous mix of pieces of material.
[0063] According to certain embodiments of the present disclosure, any sensed property output by any of the sensor systems 120 disclosed herein may be input to an AI system to classify and / or separate materials. For example, in an AI system implementing supervised learning, the output of a sensor system 120 that uniquely characterizes a particular type or composition of material (e.g., a live airbag module) may be used to train the AI system.
[0064] FIG. 3 illustrates a flow chart diagram illustrating an exemplary embodiment of a process 3500 for sorting / separating material pieces utilizing a vision system and / or one or more sensor systems, according to certain embodiments of the disclosure. Process 3500 may be configured to operate within any embodiment of the disclosure described herein, including system 100 of FIG. 1. The operations of process 3500 may be performed by hardware and / or software, including within a computer system (e.g., computer system 3400 of FIG. 5) that controls the system (e.g., computer system 107, vision system 110, and / or sensor system 120 of FIG. 1). In process block 3501, material pieces (e.g., a mixture of automotive scrap pieces) may be deposited on a conveyor system, such as that shown in FIG. 6A and FIG. 6B. In process block 3502, the position of each material piece on the conveyor system is detected to track each material piece as it moves through system 100. This may be performed by the vision system 110 (e.g., by distinguishing the piece of material from the underlying conveyor system material while communicating with the conveyor system position detector (e.g., position detector 105)). Alternatively, the piece of material tracker 111 may be used to track the piece of material. Or, a system may be used that can generate a light source (including but not limited to visible light, UV, and IR) and has a detector that can be used to identify the location of the piece of material. In processing block 3503, sensed information / characteristics of the piece of material are captured / obtained as the piece of material moves near one or more of the vision system and / or sensor system. In processing block 3504, the vision system as described above (e.g., implemented in the computer system 107) may perform pre-processing of the captured information, and the captured information may be utilized to detect (extract) information of each of the pieces of material (e.g., from the background (e.g., conveyor belt); in other words, pre-processing may be utilized to identify the difference between the piece of material and the background). Well-known image processing techniques such as dilation, thresholding, contouring, etc. may be utilized to identify the piece of material as distinguishable from the background. At processing block 3505, segmentation may be performed.For example, the captured information may include information about one or more pieces of material. Furthermore, when the image is captured, a particular piece of material may be located at a seam of a conveyor belt. Therefore, in such cases, it may be desirable to separate the image of the individual pieces of material from the background of the image. In an exemplary technique of processing block 3505, the first step is to apply a high contrast to the image, in this way the background pixels are reduced to substantially all black pixels and at least some pixels related to the piece of material are lightened to substantially all white pixels. The white image pixels of the piece of material are expanded to cover the size of the entire piece of material. Upon completing this step, the location of the piece of material is a high contrast image of all white pixels placed on a black background. A contour algorithm may then be utilized to detect the boundaries of the piece of material. The boundary information is saved and the boundary location is transferred to the original image. A segmentation is then performed on an area of the original image that is larger than the previously defined boundary. In this way, the piece of material is identified and separated from the background.
[0065] According to an embodiment of the present disclosure, processing block 3505 may implement a semantic segmentation process to identify airbag modules within a heterogeneous mixture of material pieces, as shown in Figure 7. Alternatively, instance segmentation, such as Mask R-CNN, or panoptic segmentation may be utilized.
[0066] In optional processing block 3506, the pieces of material may be transported along a conveyor system near a piece of material tracking device and / or sensor system to track each piece of material and / or determine the size and / or shape of the piece of material, which may be useful if an XRF system or other spectroscopic sensor is also implemented in the sorting system. In processing block 3507, post-processing may be performed. Post-processing may include resizing the captured information / data in preparation for use with the neural network. This post-processing may also include modifying certain characteristics (e.g., enhancing image contrast, changing the background of the image, applying filters, etc.) in a manner that enhances the AI system's ability to classify and differentiate the pieces of material. In processing block 3509, the data may be resized. Under certain circumstances, it may be desirable to resize the data to fit the data input requirements of a particular AI system, such as a neural network. For example, a neural network may require an image size (e.g., 225×255 pixels or 299×299 pixels) that is much smaller than the size of an image captured by a typical digital camera. Additionally, the smaller the input data size, the less processing time required to perform the classification, which may ultimately increase the throughput of the system 100 and increase its value.
[0067] In processing blocks 3510 and 3511, each piece of material is identified / classified based on the sensed / detected features. For example, processing block 3510 may consist of a neural network using one or more algorithms that compare the extracted features to features stored in a previously generated knowledge base (e.g., generated during a training phase) and, based on such comparison, assigns the most matching classification to each piece of material. The algorithm may process the captured information / data hierarchically using automatically trained filters. The filter responses are successfully combined at the next level of the algorithm until a probability is obtained in a final step. In processing block 3511, these probabilities can be used for each of the N classifications to determine which of the N sorting bins the respective piece of material should be sorted into. For example, each of the N classifications can be assigned to one sorting bin, and the piece of material under consideration is sorted into that bin corresponding to the classification that returns a probability greater than a predefined threshold. In an embodiment of the present disclosure, such a predetermined threshold may be pre-set by a user (e.g., to ensure that the number of false positive classifications is substantially greater than the number of false negative classifications). If neither probability is greater than a predetermined threshold, the particular piece of material may be classified as an outlier bin (eg, sorting bin 140).
[0068] Next, in process block 3512, the sorting device is activated (e.g., a command is sent to the sorting device to sort) corresponding to the classification of the piece of material. Between the time the image of the piece of material is captured and the time the sorting device is activated, the piece of material has moved from near the vision system and / or sensor system to a position downstream of the conveyor system (e.g., at the conveying speed of the conveyor system). In an embodiment of the present disclosure, activation of the sorting device is timed such that the sorting device is activated when the piece of material passes a sorting device mapped to the classification of the piece of material, and the piece of material is removed / redirected / ejected from the conveyor system (e.g., into an associated sorting bin). In an embodiment of the present disclosure, activation of the sorting device may be timed by a respective position detector that detects when the piece of material passes in front of the sorting device and sends a signal to enable activation of the sorting device. In process block 3513, the sorting bin corresponding to the activated sorting device receives the removed / redirected / ejected piece of material.
[0069] 4 illustrates a flow chart diagram illustrating an example embodiment of a process 400 for separating pieces of material, according to certain embodiments of the present disclosure. Process 400 may be configured to operate within any embodiment of the present disclosure described herein, including system 100 of FIG. 1. Process 400 may be configured to operate in conjunction with process 3500. For example, according to certain embodiments of the present disclosure, process blocks 403 and 404 may be incorporated into process 3500 (e.g., operating in series or parallel with process blocks 3503-3510) to combine the efforts of vision system 110 implemented in conjunction with an AI system with a sensor system (e.g., sensor system 120) that is not implemented in conjunction with an AI system to sort and / or separate pieces of material.
[0070] The operations of process 400 may be performed by hardware and / or software, including within a computer system (e.g., computer system 3400 of FIG. 5) that controls a system (e.g., computer system 107 of FIG. 1). In process block 401, pieces of material may be deposited on a conveyor system. Then, in optional process block 402, the pieces of material may be transported along the conveyor system within the vicinity of a piece of material tracker and / or an optical imaging system to track each piece of material and / or determine the size and / or shape of the piece of material. In process block 403, as the pieces of material move near the sensor system, they may be interrogated or stimulated with EM energy (waves) or other types of stimuli appropriate for the particular type of sensor technology utilized in the sensor system. In process block 404, physical properties of the pieces of material are sensed / detected and captured by the sensor system. In process block 405, for at least some of the pieces of material, the type of material may be identified / classified based (at least in part) on the captured properties and combined with classification by an AI system in conjunction with the vision system 110.
[0071] Next, in process block 406, if separation of the material pieces is to be performed, a separation device corresponding to the classification of the material pieces is activated. Between sensing the material piece and activating the separation device, the material piece has moved from near the sensor system to a position downstream of the conveyor system at the conveying speed of the conveyor system. In certain embodiments of the present disclosure, activation of the separation device is timed such that when the material piece passes a separation device mapped to the classification of the material piece, the separation device is activated and the material piece is removed / redirected / discharged from the conveyor system to its associated separation bin. In certain embodiments of the present disclosure, activation of the separation device may be timed by a respective position detector that detects when the material piece passes in front of the separation device and transmits a signal enabling activation of the separation device. In process block 407, a separation bin corresponding to the activated separation device receives the removed / redirected / discharged material piece.
[0072] According to certain embodiments of the present disclosure, at least a portion of the systems 100 may be linked together in series to perform multiple iterations or layers of sorting. For example, when two or more systems 100 are coordinated in this manner, the conveyor system may be implemented with a single conveyor belt or multiple conveyor belts to transport the material pieces past a first vision system (and according to certain embodiments, a sensor system) configured to separate a first set of material pieces of a heterogeneous mixture of materials into a first set of one or more bins (e.g., sorting bins 136...139) by a sorting device (e.g., a first automated control system 108 and associated one or more sorting devices 126...129) and to transport the material pieces past a second vision system (and according to certain embodiments, another sensor system) configured to separate a second set of material pieces of a heterogeneous mixture of materials into a second set of one or more sorting bins by a second sorting device. For example, a first sorting system may separate live airbag modules for safe removal from the automotive scrap stream before a second sorting system separates two or more metal alloys. Further discussion of such multi-stage sorting is provided in U.S. Patent No. 6,399,633, which is incorporated herein by reference.
[0073] Such a series of systems 100 may include any number of such systems linked together in such a manner, and according to certain embodiments of the present disclosure, each successive vision system may be configured to separate a different category or type of material than the previous system.
[0074] According to various embodiments of the present disclosure, different types or classes of material may each be classified by different types of sensors for use in an AI system and combined to classify pieces of material in a scrap or waste stream.
[0075] According to various embodiments of the present disclosure, data (e.g., spectral data) from two or more sensors can be combined using a single or multiple AI systems to perform classification of the pieces of material.
[0076] According to various embodiments of the present disclosure, multiple sensor systems can be mounted on a single conveyor system, and each sensor system can utilize a different AI system. According to various embodiments of the present disclosure, multiple sensor systems can be mounted on different conveyor systems, and each sensor system can utilize a different AI system.
[0077] 5, a block diagram is shown illustrating a data processing ("computer") system 3400 in which aspects of embodiments of the present disclosure may be implemented. (The terms "computer", "system", "computer system", and "data processing system" may be used interchangeably herein). The computer system 107, the automatic control system 108, aspects of the sensor system 120, and / or the vision system 110 may be configured similarly to the computer system 3400. The computer system 3400 may use a local bus 3405 (e.g., a peripheral component interconnect ("PCI") local bus architecture). Any suitable bus architecture may be utilized, such as Accelerated Graphics Port ("AGP") or Industry Standard Architecture ("ISA"), among others. One or more processors 3415, volatile memory 3420, and non-volatile memory 3435 may be connected to the local bus 3405 (e.g., via a PCI bridge (not shown)). An integrated memory controller and cache memory may be coupled to the one or more processors 3415. The one or more processors 3415 may include one or more central processor units and / or one or more graphics processor units and / or one or more tensor processing units. Additional connections to the local bus 3405 may be made through direct component interconnections or add-in boards. In the illustrated example, a communications (e.g., network (LAN)) adapter 3425, an I / O (e.g., small computer system interface ("SCSI") host bus) adapter 3430, and an expansion bus interface (not shown) may be connected to the local bus 3405 by component direct connections. An audio adapter (not shown), a graphics adapter (not shown), and a display adapter 3416 (coupled to a display 3440) may be connected to the local bus 3405 (e.g., by an add-in board inserted into an expansion slot).
[0078] User interface adapter 3412 may provide connections for a keyboard 3413 and mouse 3414, a modem / router (not shown), and additional memory (not shown). I / O adapter 3430 may provide connections for a hard disk drive 3431, a tape drive 3432, and a CD-ROM drive (not shown).
[0079] One or more operating systems may run on one or more processors 3415 and be used to coordinate and control various components within the computer system 3400. In FIG. 5, the operating system may be a commercially available operating system. An object-oriented programming system (e.g., Java, Python, etc.) may run in conjunction with the operating system and provide calls to the operating system from programs (e.g., Java, Python, etc.) running on the system 3400. Instructions for the operating system, object-oriented operating system, and programs may be located on a non-volatile memory 3435 storage device, such as a hard disk drive 3431, and may be loaded into the volatile memory 3420 for execution by the processor 3415.
[0080] Those skilled in the art will appreciate that the hardware in FIG. 5 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash ROM (or equivalent non-volatile memory) or optical disk drives, may be used in addition to or in place of the hardware depicted in FIG. 5. Also, any of the processes disclosed herein may be applied to a multi-processor computer system or performed by multiple such systems 3400. For example, training of the vision system 110 may be performed by a first computer system 3400, while operation of the vision system 110 for classification may be performed by a second computer system 3400.
[0081] As another example, computer system 3400 may be a stand-alone system configured to be bootable independent of some type of network communication interface, regardless of whether computer system 3400 includes any type of network communication interface. As a further example, computer system 3400 may be an embedded controller configured with ROM and / or flash ROM that provides non-volatile memory for storing operating system files or user generated data.
[0082] 5 and above-described examples are not meant to imply architectural limitations. Moreover, the computer program format of the aspects of the present disclosure may reside on any computer readable storage medium (i.e., floppy disk, compact disk, hard disk, tape, ROM, RAM, etc.) for use by a computer system.
[0083] As described herein, embodiments of the present disclosure may be implemented to perform various functions described for identifying, tracking, sorting, and / or separating pieces of material. Such functions may be implemented in hardware and / or software, such as in one or more data processing systems (e.g., data processing system 3400 of FIG. 5), such as aspects of computer system 107, vision system 110, sensor system 120, and / or automated control system 108 described above. However, the functions described herein are not limited to implementation on any particular hardware / software platform.
[0084] As will be appreciated by those skilled in the art, aspects of the present disclosure may be embodied as a system, process, method, and / or program product. Thus, various aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, generally referred to herein as a "circuit," "circuitry," "module," or "system." Additionally, aspects of the present disclosure may take the form of a program product embodied in one or more computer-readable storage media having computer-readable program code embodied therein. (However, any combination of one or more computer-readable media may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.)
[0085] A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, biological, atomic, or semiconductor system, apparatus, controller, or device, or any suitable combination thereof, and the computer-readable storage medium itself is not a transitory signal. More specific examples (non-exhaustive list) of computer-readable storage media may include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory ("RAM") (e.g., RAM 3420 of FIG. 5), a read-only memory ("ROM") (e.g., ROM 3435 of FIG. 5), an erasable programmable read-only memory ("EPROM" or flash memory), an optical fiber, a portable compact disk read-only memory ("CD-ROM"), an optical storage device, a magnetic storage device (e.g., hard drive 3431 of FIG. 5), or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, controller, or device. The program code embodied on a computer readable signal medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
[0086] A computer-readable signal medium may include a propagated data signal having computer-readable program code embodied therein, for example as part of a baseband or carrier wave. Such a propagated signal may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium is not a computer-readable storage medium, but may be any computer-readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, controller, or device.
[0087] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, processes, and program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of code, including one or more executable program instructions for implementing a specified logical function. It should also be noted that, depending on the implementation, the functions shown in the blocks may be executed in a different order than the order shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may be executed in the reverse order, depending on the functionality involved.
[0088] Modules implemented in software executed by various types of processors (e.g., GPU 3401, CPU 3415) may include one or more physical or logical blocks of computer instructions that may be organized as, for example, an object, procedure, or function. However, executable files of identified modules need not be physically located together and may include disparate instructions stored in different locations that, when logically combined, incorporate the module and achieve the specified purpose of the module. In fact, a module of executable code may be a single instruction or many instructions and may be distributed across multiple different code segments, different programs, and multiple memory devices. Similarly, operational data (e.g., a material classification library described herein) may be identified and illustrated in modules herein and may be embodied in any suitable form and organized in any suitable type of data structure. The operational data may be collected as a single data set or distributed in different locations, including different storage devices. The data may provide electronic signals over a system or network.
[0089] These program instructions can be provided to one or more processors and / or controllers of a general purpose computer, special purpose computer, or other programmable data processing apparatus (e.g., a controller) such that the instructions, which execute via a processor (e.g., GPU 3401, CPU 3415) of the computer or other programmable data processing apparatus, create circuits or means for implementing the functions / acts specified in the blocks of the flowcharts and / or block diagrams to produce a machine.
[0090] It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems (which may include, for example, one or more graphics processing units (such as GPU3401)) that perform the specified functions or operations, or by a combination of special purpose hardware and computer instructions. For example, a module can be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, controllers, or other discrete components. A module may also be implemented with programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, etc.
[0091] In the description herein, the flow charted techniques may be described as a series of sequential operations. The order of operations and the elements performing the operations may be freely modified without departing from the scope of the teachings. Operations may be added, removed, or modified in some manner. Similarly, the order of operations may be changed or looped. Furthermore, while processes, methods, algorithms, etc. may be described in a sequential order, such processes, methods, algorithms, or any combination thereof, may be operable to be performed in another order. Furthermore, some operations within a process, method, or algorithm may be performed simultaneously at least at some point (e.g., operations performed in parallel), and may also be performed in whole, in part, or in any combination thereof.
[0092] As used herein, reference may be made to a device, circuit, circuitry, system, or module being "configured" to perform a particular function. It should be understood that this may involve selecting and logically associating predefined logic blocks to provide a particular logical function, including a monitoring or control function. It may also involve programming computer software-based logic, wiring together discrete hardware components, or a combination of any or all of the above.
[0093] Unless otherwise described herein, many of the details regarding specific materials, process acts, and circuits are conventional and can be found in computing, electronics, and software technology textbooks and other sources.
[0094] Computer program code, i.e., instructions, for carrying out the operations of aspects of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, Python, C++, the "C" programming language or similar programming languages, programming languages such as MATLAB or LabVIEW, or traditional procedural programming languages such as any of the AI software disclosed herein. The program code may run entirely on the user's computer system as a standalone software package, partially on the user's computer system, partially on the user's computer system (e.g., the computer system used for sorting), partially on a remote computer system (e.g., the computer system used for training the AI system, etc.), or entirely on a remote computer system or server. In the latter scenario, the remote computer system may be connected to the user's computer system via any type of network, including a local area network ("LAN") or wide area network ("WAN"), or may be connected to an external computer system (e.g., via the Internet using an Internet Service Provider). As examples of the foregoing, various aspects of the disclosure may be configured to execute on one or more of aspects of computer system 107, automatic control system 108, vision system 110, and sensor system 120.
[0095] These program instructions may also be stored on a computer-readable storage medium that can cause a computer system, other programmable data processing device, controller, or other device to function in a particular manner, such that the instructions stored on the computer-readable medium produce an article of manufacture that includes instructions that implement the functions / acts specified in the flowchart and / or block diagram blocks.
[0096] The program instructions may also be loaded into a computer, other programmable data processing device, controller, or other device to cause the computer, other programmable device, or other device to perform a series of operational steps to generate a computer-implemented process, such that the instructions, which execute on the computer or other programmable device, provide a process for implementing the functions / operations specified in the flowchart and / or block diagram blocks.
[0097] A host may include one or more databases to store and provide access to data for various implementations. Those skilled in the art will also appreciate that for security reasons, any database, system, or component of the present disclosure may include any combination of databases or components in a single location or multiple locations, and each database or system may include any of a variety of suitable security features, such as firewalls, access codes, encryption, decryption, etc. The databases may be any type of database, such as relational, hierarchical, object-oriented, etc. Common database products that may be used to implement the databases include IBM's DB2, database products available from Oracle Corporation, Microsoft Access from Microsoft Corporation, or other database products. The databases may be organized in any suitable manner, such as data tables and lookup tables.
[0098] Association of specific data (e.g., for each piece of material processed by the material handling system described herein) may be accomplished through any data association technique known and practiced in the art. For example, association may be performed manually or automatically. Automatic association techniques may include, for example, database lookup, database merge, GREP, AGREP, SQL, and the like. The association step may be accomplished by a database merge function that uses key fields of the respective data tables of the manufacturer and retailer. The key fields divide the database according to high level classes of objects defined in the key fields. For example, a particular class may be designated as a key field in both the first data table and the second data table, and the two data tables may be merged based on the class data in the key field. In these embodiments, the data corresponding to the key fields in each of the merged data tables is preferably the same. However, data tables that are not identical but have similar data in the key fields may also be merged using, for example, AGREP.
[0099] Aspects of the present disclosure provide a method for separating live airbag modules from a moving stream of automotive scrap pieces, the method including the steps of conveying automotive scrap pieces through a vision system, the automotive scrap pieces including a live airbag module, capturing visual images of the automotive scrap pieces, processing the captured visual images of the automotive scrap pieces through an artificial intelligence system to distinguish the live airbag module from other automotive scrap pieces, and separating the live airbag module from the moving stream of automotive scrap pieces. The separating may include redirecting the live airbag module into a container along with other automotive scrap pieces in the vicinity of the live airbag module. The separating may be performed without activating the live airbag module. The separating may be performed by a separating mechanism that redirects the live airbag module using a redirecting force configured to not activate the live airbag module. The separating mechanism may be a paintbrush type plunger. The live airbag module may be partially obstructed by at least one other automotive scrap piece, preventing the vision system from acquiring spectral image data of the entire live airbag module. The artificial intelligence system may be configured to identify partially occluded live airbag modules. The artificial intelligence system may be configured with a semantic segmentation algorithm to distinguish between live airbag modules and other automotive scrap pieces. The method may further include separating the live airbag modules from the stream of automotive scrap pieces, followed by separating the automotive scrap pieces into distinct metal alloys. The artificial intelligence system may be configured to classify certain automotive scrap pieces as live airbag modules with a ratio of false positives to false negatives that exceeds a predetermined threshold.
[0100] Aspects of the present disclosure provide a system for separating live airbag modules from a moving stream of automotive scrap pieces, the system comprising: a conveyor system for conveying automotive scrap pieces past a vision system, the automotive scrap pieces including live airbag modules; a vision system configured to capture visual images of the automotive scrap pieces; a data processing system configured with an artificial intelligence system, the data processing system configured to process the captured visual images of the automotive scrap pieces through the artificial intelligence system to distinguish the live airbag modules from other automotive scrap pieces; and a sorting device for separating the live airbag modules from the moving stream of automotive scrap pieces. The sorting may include diverting the live airbag modules into a receptacle along with automotive scrap pieces in proximity to the live airbag modules. The sorting may be performed without actuating the live airbag modules. The sorting device may include a sorting mechanism that redirects the live airbag modules using a redirecting force configured to not activate the live airbag modules. The sorting mechanism may be a paintbrush type plunger. The live airbag module may be partially occluded by at least one other automotive scrap piece, preventing the vision system from acquiring spectral image data of the entire live airbag module. The artificial intelligence system may be configured to identify the partially occluded live airbag module and distinguish the partially occluded live airbag module from other automotive scrap pieces. The artificial intelligence system may be configured using a Mask R-CNN algorithm to distinguish between the live airbag module and other automotive scrap pieces. The artificial intelligence system may be configured to classify the particular automotive scrap piece as a live airbag module if the particular automotive scrap is sufficiently similar to the live airbag module. The artificial intelligence system may be configured to classify the particular automotive scrap piece as a live airbag module with a ratio of false positives to false negatives that exceeds a predetermined threshold.
[0101] In the description herein, numerous specific details are provided, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, controllers, etc., to provide a thorough understanding of embodiments of the present disclosure. However, those skilled in the art will recognize that the present disclosure may be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other cases, well-known structures, materials, or operations may not be shown or described in detail to avoid obscuring aspects of the present disclosure.
[0102] Reference throughout this specification to "one embodiment," "embodiment," or similar terms means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances throughout this specification of "one embodiment," "in an embodiment," "embodiment," "particular embodiment," "various embodiments," and similar terms may all refer to the same embodiment, but not necessarily the same. Furthermore, the described features, structures, aspects, and / or characteristics of the present disclosure may be combined in any suitable manner in one or more embodiments. Accordingly, in some cases, one or more features may be deleted from a combination described in a claim, even if the features are initially claimed to function in a particular combination, and the combination described in the claims may be subject to subcombinations or variations of the subcombinations.
[0103] Benefits, advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and elements by which the benefits, advantages, solutions occur or become more pronounced may not be construed as critical, necessary, or essential features or elements of some or all of the claims. Moreover, no element described herein is required for the practice of the disclosure unless expressly described as essential or critical.
[0104] As used herein, the term "or" is intended to be inclusive, such that "A or B" includes A or B, and also includes both A and B. As used herein, the term "and / or," when used in the context of a list of entities, refers to the entities present alone or in combination. Thus, for example, the phrase "A, B, C, and / or D" includes A, B, C, and D individually, but also all combinations and subcombinations of A, B, C, and D.
[0105] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" may be intended to include the plural forms as well, unless the context clearly indicates otherwise.
[0106] Corresponding structures, materials, acts, and equivalents of all means- or step-plus-function elements in the following claims may be intended to include any structure, material, or act for performing a function in combination with other claim elements that are specifically claimed.
[0107] As used herein with respect to a specified characteristic or condition, "substantially" refers to a degree of deviation that is small enough so as not to measurably impair the specified characteristic or condition. The exact degree of deviation that is permitted may depend on the particular situation in some cases.
[0108] As used herein, for convenience, a plurality of items, structural elements, components, and / or materials may be presented in common lists. However, these lists should be construed as if each member of the list were individually identified as a separate and unique member. Thus, the individual members of such lists should not be construed as being effectively equivalent to other members of the same list solely based on their appearance within a common grouping, absent any indication to the contrary.
[0109] Unless otherwise defined, all technical and scientific terms used herein (such as acronyms used for chemical elements in the periodic table) have the same meaning as commonly understood by one of ordinary skill in the art to which the presently disclosed subject matter belongs. Although any methods, devices, and materials similar or equivalent to those described herein can be used in the practice or testing of the presently disclosed subject matter, representative methods, devices, and materials are described herein.
[0110] The term "coupled" as used herein is not intended to be limited to a direct or mechanical coupling. Unless otherwise specified, terms such as "first" and "second" are used to arbitrarily distinguish between the elements that such terms describe. As such, these terms are not necessarily intended to indicate a temporal or other priority of such elements. [Explanation of symbols]
[0111] 100 Systems 101 Material pieces 103 Conveyor System 104 Conveyor system motor 105 Position detector 106 Tumbler / Vibrator / Singulator 107, 3400 Computer Systems 108 Automatic Control System 109 Live-action camera 110 Optical Recognition System 111 Material piece tracking device 112 Control System 120 Sensor System 121 Radiation source 122 Power supply 124 Detector 125 Detector Electronics 126…129 Sorting device 140 Separation container 3401 GPU 3405 Local Bus 3412 User Interface Adapter 3413 Keyboard 3414 Mouse 3415 Processor (CPU) 3416 Display Adapter 3420 Volatile Memory (RAM) 3425 Network (LAN) Adapter 3430 I / O Adapter 3431 Hard Disk Drive 3432 Tape Drive 3435 Non-volatile memory (ROM) 3440 Display
Claims
1. 1. A method for separating live airbag modules from a moving stream of automotive scrap material, comprising: conveying a piece of automotive scrap past a vision system, the piece of automotive scrap including a live airbag module; capturing a visual image of the automobile scrap piece; processing the captured visual images of the automotive scrap pieces through an artificial intelligence system to distinguish the live airbag module from other automotive scrap pieces; separating said live airbag modules from said moving stream of automotive scrap; A method comprising:
2. 10. The method of claim 1, wherein the separating step includes diverting the live airbag module to a container along with other automotive scrap pieces in the vicinity of the live airbag module.
3. The method of claim 1 , wherein the separating step is performed without activating the live airbag module.
4. 2. The method of claim 1, wherein the step of separating is performed by a separation mechanism that redirects the live airbag module using a redirecting force configured to deactivate the live airbag module.
5. The method of claim 4 , wherein the fractionating mechanism is a paintbrush type plunger.
6. The method of claim 2, wherein the artificial intelligence system is configured to identify the live airbag module that is partially obstructed when the live airbag module is partially obstructed by at least one other automotive scrap piece and the vision system is unable to acquire spectral image data of the entire live airbag module.
7. 7. The method of claim 6, wherein the artificial intelligence system is configured with a semantic segmentation algorithm to distinguish between live airbag modules and other automotive scrap pieces.
8. 10. The method of claim 1, further comprising the step of separating said live airbag modules from said stream of automotive scrap, followed by separating said automotive scrap into separate metal alloys.
9. The method of claim 1 , wherein the artificial intelligence system is configured to classify a particular automotive scrap piece as a live airbag module with a ratio of false positives to false negatives that exceeds a predetermined threshold.
10. 1. A system for separating live airbag modules from a moving stream of automotive scrap material, comprising: a conveyor system for transporting automotive scrap pieces past a vision system, the automotive scrap pieces including live airbag modules; the vision system configured to capture a visual image of the automobile scrap piece; a data processing system configured to process the captured visual images of the automotive scrap pieces through an artificial intelligence system to distinguish the live airbag module from other automotive scrap pieces; and a sorting device for sorting said live airbag modules from said moving stream of automotive scrap; A system comprising:
11. 11. The system of claim 10, wherein the separating includes diverting the live airbag module to a container along with pieces of automotive scrap proximate the live airbag module.
12. The system of claim 10 , wherein the separation is performed without activating the live airbag module.
13. 11. The system of claim 10, wherein the sorting device includes a sorting mechanism that redirects the live airbag module using a redirecting force configured to deactivate the live airbag module.
14. The system of claim 13 , wherein the fractionating mechanism is a paintbrush type plunger.
15. The system of claim 11, wherein the artificial intelligence system is configured to identify the partially obstructed live airbag module and distinguish the partially obstructed live airbag module from other automotive scrap pieces when the live airbag module is partially obstructed by at least one other automotive scrap piece and the vision system is unable to acquire spectral image data for the entire live airbag module.
16. 16. The system of claim 15, wherein the artificial intelligence system is configured with a semantic segmentation algorithm to distinguish between live airbag modules and other automotive scrap pieces.
17. 11. The system of claim 10, wherein the artificial intelligence system is configured to classify a particular automotive scrap piece as a live airbag module if the particular automotive scrap piece is sufficiently similar to a live airbag module.
18. 11. The system of claim 10, wherein the artificial intelligence system is configured to classify a particular automotive scrap piece as a live airbag module with a ratio of false positives to false negatives that exceeds a predetermined threshold.
Citation Information
Patent Citations
Fail-safe device and method for disposing of pyrotechnic safety devices in automobiles
JP2011520702A
JPP3288048B
Sorting between metal alloys
US20210229133A1
Multiple stage sorting
US20220016675A1
