Machine learning model for plastic type classification

Abstract

A method of training a machine learning model for performing plastic type classification is provided. The method includes: pre-training the machine learning model based on unlabeled spectral data of plastic samples using self-supervised learning with spectra masking to obtain a pre-trained machine learning model; and fine-tuning the pre-trained machine learning model based on labeled spectral data of plastic samples for performing plastic type classification. There is also provided a corresponding system for training a machine learning model for performing plastic type classification. A method of classifying a plastic type of a plastic object is also provided. The method includes: obtaining spectral data of the plastic object; and classifying the plastic type of the plastic object based on the spectral data of the plastic object using a machine learning model trained according to the above-mentioned method of training a machine learning model for performing plastic type classification. There is also provided a corresponding system for classifying a plastic type of a plastic object.

Description

MACHINE LEARNING MODEL FOR PLASTIC TYPE CLASSIFICATIONCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of priority of Singapore Patent Application No. 10202403031Y filed on 30 September 2024, the content of which being hereby incorporated by reference in its entirety for all purposes.TECHNICAL FIELD

[0002] The present invention generally relates to a method and a system for training a machine learning model (or an artificial intelligence (Al) model) for performing plastic type classification, as well as a method and a system for classifying a plastic type of a plastic object using the trained machine learning model.BACKGROUND

[0003] Since the early 2000s, surging plastic production and mismanagement have become major environmental threats With only a tiny fraction, around 9%, being recycled, the rest of this plastic pollutes our world, ending up in landfills or worse. Plastic bottles are among the top five most-produced plastics, highlighting the urgent need for better recycling solutions.

[0004] Widely used near-infrared (NIR) spectroscopy and chemometric analysis play an important role in accurately detecting polymer composition. The literature reveals advanced methods for plastic waste classification, some leveraged NIR spectroscopy with machine learning algorithms, such as support vector machines (SVMs) and principal component analysis (PC A), to enhance the classification accuracy of household plastics by analyzing their unique spectral signatures, while a previous work employed LightGBM, a tree-based machine learning technique, and N-BEATS, a deep learning approach for hyperspectral image analysis to classify difficult plastics like black polymers effectively. Another previous work takes it a step further by developing a robotic system that uses RGB cameras and a NIR hyperspectral line-scanning camera, processed with YOLO (You Only Look Once) neural network architecture for realtime object detection and k-nearest neighbors (k-NN) for classification, automating the sorting process. However, when new spectral features are introduced by a different type of spectrometer in practical applications, the predictive model often suffers significant degradation due to variations in illumination source, spectral range, and hyperspectral resolution. This can result in poor quantification accuracy if the established NIR-based model from an olderinstrument is directly applied to spectra acquired by a new instrument. Moreover, the diversity of plastic spectra data is limited and may not represent the heterogeneity of plastic samples encountered at material recovery facilities (MRFs).

[0005] Unfortunately, collecting such diverse and accurately annotating the large-scale data is an extremely laborious and expensive procedure. Moreover, the reliance on the labeled data limits the generalization of the model when dealing with new plastic resin types or unseen data, which can be a challenge for plastic waste sorting at industrial MRFs with a variety of collected household and industrial plastics wastes with different colors, thickness, and light reflectance levels.

[0006] A need therefore exists to provide a method of training a machine learning model for performing plastic type classification, as well as a system thereof, that seeks to overcome, or at least ameliorate, one or more deficiencies in conventional methods of training a machine learning model for performing plastic type classification, and more particularly, with improved or enhanced efficiency and effectiveness. It is against this background that the present invention has been developed.SUMMARY

[0007] According to a first aspect of the present invention, there is provided a method of training a machine learning model for performing plastic type classification, the method comprising: pre-training the machine learning model based on unlabeled spectral data of plastic samples using self-supervised learning with spectra masking to obtain a pre-trained machine learning model; and fine-tuning the pre-trained machine learning model based on labeled spectral data of plastic samples for performing plastic type classification.

[0008] According to a second aspect of the present invention, there is provided a system for training a machine learning model for performing plastic type classification, the system comprising: at least one memory; and at least one processor communicatively coupled to the at least one memory and configured to:pre-train the machine learning model based on unlabeled spectral data of plastic samples using self-supervised learning with spectra masking to obtain a pre-trained machine learning model; and fine-tune the pre-trained machine learning model based on labeled spectral data of plastic samples for performing plastic type cl ssification.

[0009] According to a third aspect of the present invention, there is provided a method of classifying a plastic type of a plastic object, the method comprising: obtaining spectral data of the plastic object; and classifying the plastic type of the plastic object based on the spectral data of the plastic object using a machine learning model trained according to the method according to the above- mentioned first aspect of the present invention.

[0010] According to a fourth aspect of the present invention, there is provided a system for classifying a plastic type of a plastic object, the system comprising: at least one memory; and at least one processor communicatively coupled to the at least one memory and configured to: obtain spectral data of the plastic object; and classify the plastic type of the plastic object based on the spectral data of the plastic object using a machine learning model trained according to the method according to the above- mentioned first aspect of the present invention or the system according to the above-mentioned second aspect of the present invention.

[0011] According to a fifth aspect of the present invention, there is provided a computer program product, embodied in one or more non-transitory computer-readable storage mediums, comprising instructions executable by at least one processor to perform the method of training a machine learning model for performing plastic type classification according to the above- mentioned first aspect of the present invention.

[0012] According to a sixth aspect of the present invention, there is provided a computer program product, embodied in one or more non-transitory computer-readable storage mediums, comprising instructions executable by at least one processor to perform the method of classifying a plastic type of a plastic object according to the above-mentioned third aspect of the present invention.BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Embodiments of the present invention will be better understood and readily apparent to one of ordinary skill in the art from the following written description, by way of example only, and in conjunction with the drawings, in which:FIG. 1 depicts a schematic diagram of a method of training a machine learning model for performing plastic type classification, according to various embodiments of the present invention;FIG. 2 depicts a schematic diagram of a method of classifying a plastic type of a plastic object, according to various embodiments of the present invention, according to various embodiments of the present invention;FIG. 3 depicts a schematic block diagram of a system for training a machine learning model for performing plastic type classification, according to various embodiments of the present invention;FIG. 4 depicts a schematic block diagram of a system for classifying a plastic type of a plastic object, according to various embodiments of the present invention;FIG. 5 shows Table 1 presenting the collected dataset’s number of plastic samples, with a total of 744 waste household plastic bottles and 6552 acquired spectra in the AIRP dataset (Al Recycling Plastics dataset - a diverse plastic dataset collected according to various example embodiments of the present invention);FIG. 6 shows a method or procedure of spectra sample extraction in the AIRP dataset, according to various example embodiments of the present invention;FIG. 7 depicts a schematic diagram of an example method of training a machine learning model for performing plastic type classification, according to various example embodiments of the present invention;FIG. 8 depicts images (denoted by (a) and (b)) showing illumination light changes on a HDPE plastic bottle, an image (denoted by (c)) showing a crushed HDPE plastic bottle, and an image (denoted by (d)) showing a HDPE plastic bottle with a label, according to various example embodiments of the present invention;FIG. 9 depicts a plot showing the diversity of acquired spectra from the center ROI of HDPE plastic samples, according to various example embodiments of the present invention;FIG. 10 depicts an image (denoted by (a)) showing a PET plastic bottle with a distorted shape, an image (denoted by (b)) showing a PET plastic bottle with fluid contamination, animage (denoted by (c)) showing a PET bottle with a label, and an image (denoted by (d)) showing a crushed bottle, according to various example embodiments of the present invention;FIG. 11 depicts a plot showing the diversity of acquired spectra from PET plastic samples, according to various example embodiments of the present invention;FIG. 12 shows Table 2 presenting the classification accuracy of each of five models evaluated on the test dataset for two different plastic resin types;FIGs. 13 A and 13B each depicts a plot showing a reconstruction of a plastic spectra signal from the AIRP dataset by the masked SSL framework with 70% of masked frequency segments, compared with the original plastic spectra signal, according to various example embodiments of the present invention;FIG. 14A and 14B visualize results obtained during the inference for plastic wastes (plastic bottle classification as HDPE and PET bottles) by the plastic classification model trained using the method described with reference to FIG. 7 according to various example embodiments of the present invention using different ROls within the same image; andFIG. 15 shows the results of masked signal modeling (MSM) with different masking ratios, according to various example embodiments of the present invention.DETAILED DESCRIPTION

[0014] Various embodiments of the present invention provide a method and a system for training a machine learning model (or an Al model) for performing plastic type classification (e.g., plastic resin (e.g., plastic waste resin) type). In addition, various embodiments of the present invention provide a method and a system for classifying a plastic type (e.g., a plastic resin type) of a plastic object using the trained machine learning model.

[0015] As discussed in the background, various conventional methods of performing plastic type classification, or various conventional methods of training a machine learning model for performing plastic type classification, are prone to significant performance degradation for various reasons, and in particular, poor input variance tolerance (machine learning model’s robustness against variations in its inputs). Therefore, for example, such trained machine learning models often suffer significant degradation when features of a plastic object (being classified with respect to its plastic type) are captured under different conditions (e.g., variations in illumination conditions) or when the plastic object is not part of the labeled training datasets used to train the machine learning models. This performance degradation may be reduced by collecting a significantly more diverse plastic dataset to better represent the heterogeneity ofplastic samples. However, collecting such diverse and accurately annotating large-scale data is an extremely laborious and expensive procedure. Therefore, various embodiments of the present invention seek to provide a method of training a machine learning model for performing plastic type classification, as well as a system thereof, that seeks to overcome, or at least ameliorate, one or more deficiencies in conventional methods of training a machine learning model for performing plastic type classification, and more particularly, with improved or enhanced efficiency and effectiveness.

[0016] FIG. 1 depicts a schematic diagram of a method 100 of training a machine learning model for performing plastic type classification, according to various embodiments of the present invention. The method 100 comprises: pre-training (at 106) the machine learning model based on unlabeled spectral data of plastic samples using self-supervised learning with spectra masking to obtain a pre-trained machine learning model; and fine-tuning (at 108) the pre-trained machine learning model based on labeled spectral data of plastic samples for performing plastic type classification.

[0017] The method 100 of training a machine learning model for performing plastic type classification has improved or enhanced efficiency and effectiveness. Firstly, the method 100 advantageously includes a pre-training stage whereby the machine learning model for performing plastic type classification is pre-trained based on unlabeled spectral data of plastic samples using self-supervised learning (SSL). Therefore, the machine learning model can first be pre-trained based on unlabeled spectral data of a diverse and / or large amount of different plastic samples (e.g., a diverse plastic dataset), thereby advantageously avoiding the need to annotate a large-scale training dataset which is extremely laborious and expensive. The method 100 further includes a subsequent fine-tuning stage whereby the pre-trained machine learning model is fine-tuned based on labeled spectral data of plastic samples. In this regard, as the machine learning model has already been pre-trained to learn rich features that generalize well from the diverse and / or large amount of different plastic samples, the pre-trained machine learning model can be fine-tuned using a small amount of labeled spectral data to produce an effective machine learning model trained for performing plastic type classification. Furthermore, during the pre-training stage, the pre-training is performed using SSL with spectra masking. This has been found to improve the input variance tolerance of the machine learning model, thereby further improving the effectiveness of the machine learning model in performing plastic type classification. Therefore, the method 100 of training a machine learning model for performing plastic type classification is not only effective but is also efficient. Theseadvantages or technical effects, and / or other advantages or technical effects, will become more apparent to a person skilled in the art as the method 100 of (and the corresponding system for) training a machine learning model for performing plastic type classification, as well as the corresponding method of (and the corresponding system for) classifying a plastic type of a plastic object using the trained machine learning model, is described in more detail according to various embodiments and example embodiments of the present invention.[0018| In various embodiments, the unlabeled spectral data of the plastic samples are obtained from unlabeled spectral images of the plastic samples. In this regard, for each of the unlabeled spectral images, the unlabeled spectral data of the unlabeled spectral image comprises a plurality of spectral reflectance values for a plurality of spectral bands, respectively.

[0019] In various embodiments, the unlabeled spectral images of the plastic samples are unlabeled hyperspectral images of the plastic samples. Furthermore, for each of the unlabeled spectral images, the unlabeled spectral data of the unlabeled spectral image is in the form of a 1-dimensional (ID) vector comprising the plurality of spectral reflectance values for the plurality of spectral bands, respectively. Accordingly, in various embodiments, the unlabeled spectral data of the plastic samples is represented or configured in such a manner for facilitating or enabling the pre-training of the machine learning model using self-supervised learning with spectra masking.

[0020] In various embodiments, wherein the unlabeled spectral images of the plastic samples are obtained in the near-infrared (NIR) field.

[0021] In various embodiments, the method 100 further comprises, for each of the unlabeled spectral images and for each spectral band of the plurality of spectral bands of the unlabeled spectral image, determining the spectral reflectance value for the spectral band of the unlabeled spectral image based on averaging pixel values of pixels of the unlabeled spectral image at the spectral band. Accordingly, in various embodiments, for configuring the unlabeled spectral data of the unlabeled spectral image in the form of a ID vector comprising the plurality of spectral reflectance values for the plurality of spectral bands, respectively, for each spectral band, the pixel values of pixels of the unlabeled spectral image at the spectral band are averaged.

[0022] In various embodiments, the above-mentioned spectra masking comprises, for each of the unlabeled spectral images: masking a plurality of frequency segments of the unlabeled spectral data of the unlabeled spectral image to obtain partially masked spectral data of the unlabeled spectral image; and inputting the partially masked spectral data of the unlabeled spectral image to the machine learning model to produce a feature vector thereof.

[0023] In various embodiments, the above-mentioned masking the plurality of frequency segments of the unlabeled spectral data of the unlabeled spectral image comprises masking a majority (i.e., more than half) of the frequency segments of the unlabeled spectral data.

[0024] In various embodiments, preferably, more than 50% to about 90% of the frequency segments of the unlabeled spectral data are masked (e g., which has been found to achieve about 90% classification accuracy or higher). In various embodiments, more preferably, from about 60% to about 80% of the frequency segments of the unlabeled spectral data are masked (e.g., which has been found to achieve about 92% classification accuracy or higher). In various embodiments, still more preferably, about 70% of the frequency segments of the unlabeled spectral data are masked (e.g., which has been found to achieve an optimal classification accuracy of about 96%).

[0025] In various embodiments, the plurality of frequency segments are randomly masked amongst the frequency segments of the unlabeled spectral data.

[0026] In various embodiments, the above-mentioned pre-training (at 106) the machine learning model comprises pre-training the machine learning model with a prediction head based on the unlabeled spectral data of the plastic samples using the above-mentioned self-supervised learning with the above-mentioned spectra masking to obtain the pre-trained machine learning model. Furthermore, for the above-mentioned each of the unlabeled spectral images, the above- mentioned self-supervised learning comprises inputting the feature vector produced based on the partially masked spectral data of the unlabeled spectral image to the prediction head to produce a reconstructed spectral data of the partially masked spectral data of the unlabeled spectral image based on the feature vector.

[0027] In various embodiments, the machine learning model comprises an encoder and the prediction head comprises a decoder. Accordingly, the pre-trained machine learning model comprises a pre-trained encoder.

[0028] In various embodiments, the above-mentioned fine-tuning the pre-trained machine learning model comprises fine-tuning the pre-trained machine learning model with a classification head based on the labeled spectral data of plastic samples. In this regard, the classification head being configured to perform a plastic type classification task.

[0029] FIG. 2 depicts a schematic diagram of a method 200 of classifying a plastic type (e.g., a plastic resin type) of a plastic object, according to various embodiments of the present invention. The method 200 comprises: obtaining (at 206) spectral data of the plastic object; and classifying (at 208) the plastic type of the plastic object based on the spectral data of the plasticobject using a machine learning model trained according to the method 100 as described hereinbefore according to various embodiments of the present invention.

[0030] In various embodiments, same as or similar to the method 100, the spectral data of the plastic object is obtained from a spectral image of the plastic object, and the spectral data of the spectral image comprises a plurality of spectral reflectance values for a plurality of spectral bands, respectively.[00311 In various embodiments, same as or similar to the method 100, the spectral image of the plastic object is a hyperspectral image of the plastic object, and the spectral data of the spectral image is in the form of a ID vector comprising the plurality of spectral reflectance values for the plurality of spectral bands, respectively.

[0032] In various embodiments, same as or similar to the method 100, the spectral image of the plastic sample is obtained in the NIR field.

[0033] In various embodiments, same as or similar to the method 100, the method 200 further comprises, for each spectral band of the plurality of spectral bands of the spectral image, determining the spectral reflectance value for the spectral band of the spectral image based on averaging pixel values of pixels of the spectral image at the spectral band.

[0034] In various embodiments, the above-mentioned classifying (at 208) the plastic type of the plastic obj ect based on the spectral data of the plastic obj ect further uses the classification head configured to perform the plastic type classification task.

[0035] FIG. 3 depicts a schematic block diagram of a system 300 for training a machine learning model for performing plastic type classification, according to various embodiments of the present invention, corresponding to the above-mentioned method 100 of training a machine learning model for performing plastic type classification as described hereinbefore according with reference to FIG. 1 according to various embodiments of the present invention The system 300 comprises: at least one memory 302; and at least one processor 304 communicatively coupled to the at least one memory 302 and configured to perform the method 100 of training a machine learning model for performing plastic type classification as described hereinbefore according to various embodiments of the present invention. Accordingly, the at least one processor 304 is configured to: pre-train the machine learning model based on unlabeled spectral data of plastic samples using self-supervised learning with spectra masking to obtain a pre-trained machine learning model; and fine-tune the pre-trained machine learning model based on labeled spectral data of plastic samples for performing plastic type classification.

[0036] It will be appreciated by a person skilled in the art that the at least one processor 304 may be configured to perform various functions or operations through set(s) of instructions (e.g., software modules) executable by the at least one processor 304 to perform various functions or operations. Accordingly, as shown in FIG. 3, the system 300 may comprise: a pretraining module (or a pre-training circuit) 306 configured to pre-train the machine learning model based on unlabeled spectral data of plastic samples using self-supervised learning with spectra masking to obtain a pre-trained machine learning model; and a fine-tuning module (a fine-tuning circuit) 308 configured to fine-tune the pre-trained machine learning model based on labeled spectral data of plastic samples for performing plastic type classification.

[0037] It will be appreciated by a person skilled in the art that the above-mentioned modules of the system 300 are not necessarily separate modules, and two or more modules may be realized by or implemented as one functional module (e g., a circuit or a software program) as desired or as appropriate without deviating from the scope of the present invention. For example, the pre-training module 306 and the fine-tuning module 308 may be realized (e.g., compiled together) as one executable software program (e.g., software application), which for example may be stored in the at least one memory 302 and executable by the at least one processor 304 to perform the corresponding functions or operations as described herein according to various embodiments of the present invention.

[0038] In various embodiments, the system 300 for training a machine learning model corresponds to the method 100 of for training a machine learning model as described hereinbefore with reference to FIG. 1, therefore, various operations, functions or steps configured to be performed by the at least one processor 304 may correspond to various operations, functions or steps of the method 100 of training a machine learning model as described hereinbefore according to various embodiments, and thus need not be repeated with respect to the system 300 for training a machine learning model for clarity and conciseness. In other words, various embodiments described herein in context of methods (e g., the method 100 of training a machine learning model) are analogously valid for the corresponding systems or devices (e.g., the system 300 for training a machine learning model), and vice versa. For example, in various embodiments, the at least one memory 302 may have stored therein the pre-training module 306 and / or the fine-tuning module 308, which respectively correspond to various operations, functions or steps of the method 100 of training a machine learning model as described hereinbefore according to various embodiments, which are executable by the atleast one processor 304 to perform the corresponding operations, functions or steps as described herein.

[0039] FIG. 4 depicts a schematic block diagram of a system 400 for classifying a plastic type of a plastic object, according to various embodiments of the present invention, corresponding to the above-mentioned method 200 of classifying a plastic type of a plastic object as described hereinbefore with reference to FIG. 2 according to various embodiments of the present invention. The system 400 comprises: at least one memory 402; and at least one processor 404 communicatively coupled to the at least one memory 402 and configured to perform the method 200 of classifying a plastic type of a plastic object as described hereinbefore according to various embodiments of the present invention. Accordingly, the at least one processor 404 is configured to: obtain spectral data of the plastic object; and classify the plastic type of the plastic object based on the spectral data of the plastic object using a machine learning model trained according to the method 100 or using the system 300 as described hereinbefore according to various embodiments of the present invention.

[0040] Similar to the system 300, it will be appreciated by a person skilled in the art that the at least one processor 404 may be configured to perform various functions or operations through set(s) of instructions (e.g., software modules) executable by the at least one processor 404 to perform various functions or operations. Accordingly, as shown in FIG. 4, the system 400 may comprise: a spectral data obtaining module (or a spectral data obtaining circuit) 406 configured to obtain spectral data of the plastic object, and a plastic type classifying module (or a plastic type classifying circuit) 408 configured to classify the plastic type of the plastic object based on the spectral data of the plastic object using a machine learning model trained according to the method 100 or using the system 300 as described hereinbefore according to various embodiments of the present invention.

[0041] Similar to the system 300, it will be appreciated by a person skilled in the art that the above-mentioned modules of the system 400 are not necessarily separate modules, and the modules may be realized by or implemented as one functional module (e g., a circuit or a software program) as desired or as appropriate without deviating from the scope of the present invention. For example, the spectral data obtaining module 406 and the plastic type classifying module 408 may be realized (e.g., compiled together) as one executable software program (e.g., software application), which for example may be stored in the at least one memory 402 and executable by the at least one processor 404 to perform the corresponding functions or operations as described herein according to various embodiments of the present invention.

[0042] In various embodiments, the system 400 for classifying a plastic type of a corresponds to the method 200 of classifying a plastic type of a plastic object as described hereinbefore with reference to FIG. 2, therefore, various operations, functions or steps configured to be performed by the at least one processor 404 may correspond to various operations, functions or steps of the method 200 of classifying a plastic type of a plastic object as described hereinbefore according to various embodiments, and thus need not be repeated with respect to the system 400 for classifying a plastic type of a plastic object for clarity and conciseness. In other words, various embodiments described herein in context of methods (e.g., the method 200 of classifying a plastic type of a plastic object) are analogously valid for the corresponding systems or devices (e g., the system 400 classifying a plastic type of a plastic object), and vice versa. For example, in various embodiments, the at least one memory 402 may have stored therein the spectral data obtaining module 406 and / or the plastic type classifying module 408, which respectively correspond to various operations, functions or steps of the method 200 of classifying a plastic type of a plastic object as described hereinbefore according to various embodiments, which are executable by the at least one processor 404 to perform the corresponding operations, functions or steps as described herein.

[0043] A computing system, a controller, a microcontroller or any other system providing a processing capability may be provided according to various embodiments in the present invention. Such a system may be taken to include one or more processors and one or more computer-readable storage mediums. For example, the system 300 for training a machine learning model and the system 400 for classifying a plastic type of a plastic object described hereinbefore may each include at least one processor (or controller) and at least one computer- readable storage medium (or memory) which are for example used in various processing carried out therein as described herein. A memory or computer-readable storage medium used in various embodiments may be a volatile memory, for example a DRAM (Dynamic Random Access Memory) or a non-volatile memory, for example a PROM (Programmable Read Only Memory), an EPROM (Erasable PROM), EEPROM (Electrically Erasable PROM), or a flash memory, e g., a floating gate memory, a charge trapping memory, an MRAM (Magnetoresistive Random Access Memory) or a PCRAM (Phase Change Random Access Memory). Furthermore, it will be appreciated by a person skilled in the art that the system 300 for training a machine learning model may be implemented by a high-performance computer system, or a network of high-performance computer systems, known in the art for performing training, especially when a large-scale training is performed. For example, the high-performancecomputer system may include multiple processors including GPUs (graphics processing units) and CPUs (central processing units) optimized for advanced computing tasks, such as execution of machine learning algorithms. For example, the high-performance computer system may comprise an array of GPUs (Graphics Processing Units) dedicated to handling parallel processing tasks to facilitate the rapid execution of machine learning algorithms. Tn the field of machine learning, it will be understood by a person skilled in the art that the system 300 for training a machine learning model may be implemented as a server (e.g., centralized or distributed server (centralized or decentralized machine learning)), such as a cloud server, including specialized hardware (e.g., high performance graphics processing unit(s) (GPU(s))) designed for handling resource-intensive tasks associated with machine learning.

[0044] In various embodiments, a “circuit” may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing software stored in a memory, firmware, or any combination thereof. Thus, in an embodiment, a “circuit” may be a hard-wired logic circuit or a programmable logic circuit such as a programmable processor, e.g., a microprocessor (e.g., a Complex Instruction Set Computer (CISC) processor or a Reduced Instruction Set Computer (RISC) processor). A “circuit” may also be a processor executing software, e.g., any kind of computer program, e.g., a computer program using a virtual machine code, e.g., Java. Any other kind of implementation of various functions or operations may also be understood as a “circuit” in accordance with various other embodiments. Similarly, a “module” may be a portion of a system according to various embodiments in the present invention and may encompass a “circuit” as above, or may be understood to be any kind of a logic-implementing entity therefrom.

[0045] Some portions of the present disclosure may be explicitly or implicitly presented in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the data processing arts to convey most effectively the substance of their work to others skilled in the art. An algorithm may be, and generally, conceived to be a self-consi stent sequence of steps leading to a desired result.

[0046] The present specification also discloses systems (e.g., which may also be embodied as devices or apparatuses), such as the systems 300, 400, for performing various operations, functions or steps of various methods described herein. Such systems may each be specially constructed for the required purposes or may comprise a general purpose computer system selectively activated or reconfigured by a computer program stored in the computer system Ingeneral, various algorithms that may be presented herein are not limited to being implemented or executed by any particular computer system. Alternatively, the construction of more specialized computer system (e.g., a high-performance computer system as described hereinbefore) to perform various operations, functions or steps of various methods described herein may be provided as desired or as appropriate without going beyond the scope of the present invention.|0047| In addition, the present specification also at least implicitly discloses computer program(s) or software / functional module(s), in that it would be apparent to a person skilled in the art that various operations, functions or steps of various methods described herein may be put into effect by computer code The computer program(s) is not intended to be limited to any particular programming language and implementation thereof, and it will be appreciated by a person skilled in the art that a variety of programming languages and coding thereof may be used to implement the computer program(s). Moreover, the computer program(s) is not intended to be limited to any particular control flow as there are a variety of programming languages which can use different control flows. It will be appreciated by a person skilled in the art that a computer program may be stored on any computer-readable storage medium (non- transitory computer-readable storage medium), such as but not limited to, a magnetic disk, an optical disk or a memory chip. For example, a computer program stored on a computer-readable storage medium may be loaded and executed on a computer system to implement various operations, functions or steps of various methods described herein according to various embodiments of the present invention.

[0048] Accordingly, in various embodiments, there is provided a computer program product, embodied in one or more computer-readable storage mediums (non-transitory computer-readable storage medium), comprising instructions (e g., the pre-training module 306 and / or the fine-tuning module 308) executable by one or more computer processors to perform the method 100 of training a machine learning model as described hereinbefore with reference to FIG. 1 according to various embodiments of the present invention. In various embodiments, there is provided a computer program product, embodied in one or more computer-readable storage mediums (non-transitory computer-readable storage medium), comprising instructions (e.g., the spectral data obtaining module 406 and / or the plastic type classifying module 408) executable by one or more computer processors to perform the method 200 of classifying a plastic type of a plastic object as described hereinbefore with reference to FIG. 2 according to various embodiments of the present invention. Accordingly, various computer programs orsoftware modules described herein may be stored in a computer program product receivable by a system therein, such as the system 300 for classifying a plastic type of a plastic object as shown in FIG. 3, for execution by at least one processor 304 of the system 300 to perform various operations, functions or steps of various methods described herein according to various embodiments of the present invention.

[0049] It will be appreciated by a person skilled in the art that various modules of systems described herein (e.g., the pre-training module 306 and / or the fine-tuning module 308 of the system 300 or the spectral data obtaining module 406 and / or the plastic type classifying module 408 of the system 400) may be software module(s) realized by computer program(s) or set(s) of instructions executable by a computer processor to perform various functions or operations. Various modules of systems described herein may also be implemented as hardware module(s) being functional hardware unit(s) designed to perform various functions or operations. More particularly, in the hardware sense, a module is a functional hardware unit designed for use with other components or modules. For example, a module may be implemented using discrete electronic components, or it can form a portion of an entire electronic circuit such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA). Numerous other possibilities exist. It will also be appreciated by a person skilled in the art that a combination of hardware and software modules may be implemented. Furthermore, various operations, functions or steps of various methods described herein may be performed in parallel rather than sequentially as desired or as appropriate (e.g., as long as it does not render the method(s) inoperable or unsatisfactory for its intended purpose).

[0050] It will be appreciated by a person skilled in the art that the terminology used herein is for the purpose of describing various embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0051] Any reference to an element or a feature herein using a designation such as “first”, “second” and so forth does not limit the quantity or order of such elements or features, unless stated or the context requires otherwise For example, such designations may be used herein asa convenient way of distinguishing between two or more elements or instances of an element. Thus, a reference to first and second elements does not necessarily mean that only two elements can be employed, or that the first element must precede the second element, unless stated or the context requires otherwise. In addition, a phrase referring to “at least one of’ a list of items refers to any single item therein or any combination of two or more items therein.

[0052] In order that the present invention may be readily understood and put into practical effect, various example embodiments of the present invention will be described hereinafter by way of examples only and not limitations. It will be appreciated by a person skilled in the art that the present invention may, however, be embodied in various different forms or configurations and should not be construed as limited to the example embodiments set forth hereinafter. Rather, these example embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present invention to those skilled in the art.

[0053] In particular, for better understanding of the present invention and without limitation or loss of generality, various example embodiments of the present invention will now be described with respect to the machine learning model (or the Al model) being an encoder and the prediction head being a decoder. However, it will be understood by a person skilled in the art that the machine learning model and the prediction head are not limited to having an encoder-decoder architecture or any specific configuration or architecture and may have any configuration or architecture as desired or as appropriate without going beyond the scope of the present invention, as long as the machine learning model (or the Al model) can be pre-trained (with the prediction head) using self-supervised learning and then fine-tuned (with a classification head) for performing plastic type classification. Similarly, it will be understood by a person skilled in the art that the classification head is not limited to any specific configuration or architecture and may have any configuration or architecture as desired or as appropriate without going beyond the scope of the present invention, as long as the classification head is configured to perform a plastic type classification task and is operable with the pre-trained machine learning model for fine-tuning the pre-trained machine learning model for performing plastic type classification.

[0054] According to various example embodiments, a unique approach is developed for plastic type classification (or plastic spectra signal classification) using self-supervised learning (SSL) based on near-infrared (NIR) spectroscopy. A key challenge in training a machine learning model for performing plastic spectra signal classification is the representation of plasticspectra signals, which relies on spectrum data of limited variety collected, for example, in a lab. According to various example embodiments, to overcome these challenges, SSL is adopted to learn general representations from large amounts of unlabeled spectral data of plastic samples (e.g., hyperspectral image (HSI) samples) during the pre-training stage. Various example embodiments found that the representation power of SSL enables higher classification performance to be achieved even with limited labeled spectral data, thereby reducing the necessity for large-scale annotated data in NIR-based signal analysis.

[0055] In various example embodiments, a diverse spectra reflectance library of plastic samples was collected to be utilized for unknown plastic sample classification. Furthermore, an SSL framework using masked signal modeling (MSM) is presented that employs a masking strategy to learn general representations for spectra signals during the pre-training stage. To the best of knowledge of the inventors, this is the first attempt to classify the plastic resin type using SSL pre-training. The enhanced representations obtained through SSL enable the fine-tuning of the machine learning model with a small amount of labeled spectral data for spectra signal classification, for example, of less common plastic types. The method of training a machine learning model for performing plastic type classification, as well as the method of classifying a plastic type of a plastic object, according to various example embodiments will be evaluated and discussed later below through experiments, demonstrating its efficiency and effectiveness in plastic type classification (e g., plastic spectra signal classification).Plastic Samples

[0056] According to various example embodiments, target plastic resin types were collected and a diverse spectra reflectance library of plastic samples was developed to be utilized for unknown plastic sample classification Household plastic waste is the most prevalent plastic types in daily life and 60% of the total plastic waste. Specifically, plastic waste samples, including the production of bottles and caps of detergents and cleaning products, were collected from local supermarkets, and the plastic wastes were collected at a material recovery facility (MRF), which are common household items in daily life. As an illustrative example and without limitations, two common types of household plastic waste, PET and HDPE, were investigated as they are among the main polymers used for household items in daily life. In this regard, plastic bottles with varying optical properties (color, brightness, transmissivity) were collected. In experimental studies according to various example embodiments of the present invention, a plastic spectra dataset was built with different compositions of virgin (i.e., clean)and contaminated plastics. The contamination commonly includes the plastic being distorted in shape and / or being filled or smeared with contaminants (e.g., fluids and stains) internally or externally. Table 1 shown in FIG. 5 presents the collected dataset’s number of plastic samples, with a total of 744 waste household plastic bottles and 6552 acquired spectra in the AIRP dataset (Al Recycling Plastics dataset - a diverse plastic dataset collected according to various example embodiments of the present invention). In particular, in total, 744 plastic bottles, 397 and 347 samples of each HDPE and PET were used as standard samples for training, validating, and testing the model. Of these, 114 and 102 samples from HDPE and PET, respectively, were used as test samples to verify the accuracy of the downstream task.

[0057] As an example, a Fabry-Perot Interferometer (FPI) hyperspectral camera (900-1700 nanometers wavelength) was used for the imaging of plastic items using a single camera setup, resulting in a 3D spatial -spectral hypercube data, for example, of 640 (pixels) by 512 (lines) by 92 (bands). In various example embodiments, a pipeline that leverages foundation models (FMs) was investigated and developed to facilitate the creation of the plastic samples dataset. In various example embodiments, to reduce the influence of unrelated areas, a segmentation model (e.g., the Segment Anything Model (SAM) model disclosed in Kirillov et al., “Segment anything”, arVix:2304.02643, 2023) was used to automatically annotate the plastic bottle boundaries from other objects and backgrounds. Only samples within the region-of-interest (ROT) mask generated by the segmentation model were used to build the datasets. SAM is a promptable foundation model (FM) with a variety of prompts, such as bounding boxes, polygons, or points. In various example embodiments, the SAM foundation model was employed with text prompt, also known as Language-Segment-Anything (Lang-SAM).

[0058] FIG. 6 shows a method or procedure of spectra sample extraction in the AIRP dataset, according to various example embodiments of the present invention. As an illustrative example, FIG. 6 shows an example text prompt for guiding the GroundingDINO and SAM model and acquired spectra in the NIR field (900-1700 nm) obtained from 64 X 64 X 92 3D hypercube samples. According to various example embodiments, given the large size of HSI and 92 spectral bands covering the visible near-infrared spectrum (spectral range 900-1700 nm), 64 X 64 X 92 3D hypercube samples are extracted. Table 1 shows the number of samples for each plastic resin type.Self-Supervised Representation Learning of Spectra Data

[0059] FIG. 7 depicts a schematic diagram of an example method 700 of training a machine learning model for performing plastic type classification, according to various example embodiments of the present invention. The method 700 comprises a pre-training stage 710 configured to pre-train the machine learning model (e.g., an encoder) 712 based on unlabeled spectral data of plastic samples using self-supervised learning (SSL) with spectra masking to obtain a pre-trained machine learning model (e.g., a pre-trained encoder) 722; and a fine-tuning stage 720 (or a downstream fine-tuning stage) configured to fine-tune the pre-trained machine learning model 722 based on labeled spectral data of plastic samples for performing plastic type classification.

[0060] Accordingly, as shown in FIG. 7, the pre-training stage 710 performs selfsupervised pre-training using masked signal modeling (MSM), which may also be referred to as pretext self-supervised training. For example, the MSM framework may comprise an encoder 712 and a decoder 716. During the pre-training, the raw input spectra signal, where parts of it are randomly masked (spectra masking), is fed into the encoder 712 to extract the relevant features. During the fine-tuning, the pre-trained encoder 722 and a classification head 726 are leveraged for the downstream task, such as plastic spectra signal classification.

[0061] Patchification. Given a HSI sample I E ]Rnkxnwxn'i, where nh, nw= 64 represents the height and width of sample and n denotes the number of spectral bands, according to various example embodiments of the present invention, raw spectra data is extracted from HSI sample by averaging the pixels values at each spectral band. In various example embodiments, the raw spectral data is represented as a ID vector of f E IRn^, f —spectra frequencies (e.g., 92 in the HSI data), in which each value represents the mean reflectance value at a specific frequency. As described above, the example model architecture of the masked SSL method in the pre-training stage is shown in FIG. 7. For example, the signal of length 92 may be divided into 92 segments, each of length 1 as the patch size.

[0062] Accordingly, in various example embodiments, the unlabeled spectral data of the plastic samples may be obtained from unlabeled spectral images of the plastic samples. In this regard, for each of the unlabeled spectral images, the unlabeled spectral data of the unlabeled spectral image comprises a plurality of spectral reflectance values for a plurality of spectral bands, respectively. Furthermore, in various example embodiments, the unlabeled spectral images of the plastic samples are unlabeled hyperspectral images of the plastic samples. In this regard, for each of the unlabeled spectral images, the unlabeled spectral data of the unlabeledspectral image are in the form of a ID vector comprising the plurality of spectral reflectance values for the plurality of spectral bands, respectively. Still further, in various example embodiments, the method 700 (or more particularly, the pre-training stage 710) further comprises, for each of the unlabeled spectral images and for each spectral band of the plurality of spectral bands of the unlabeled spectral image, determining the spectral reflectance value for the spectral band of the unlabeled spectral image based on averaging pixel values of pixels of the unlabeled spectral image at the spectral band.

[0063] Spectra Masking’. There are different masking strategies, where the masked parts can be initialized by learnable random values or can be set to zero. In various example embodiments, a masking strategy is applied where the input spectra drawn from the samples are randomly masked out in specific frequencies and set to zero. In various example embodiments, a majority of these frequency segments are randomly masked. In various example embodiments, more than 50% to about 90%, from about 60% to about 80%, or about 70%, of the frequency segments are masked. In this regard, various example embodiments found that a high ratio of about 70% was found to produce an optimal classification performance. The masked segments may be different for each spectra, and the position of the masked segments may be randomly chosen for the signals in a data batch. As shown in FIG. 7, according to various example embodiments, a simple encoder-masking SSL framework may be provided where the masked segments and visible segments are put into the encoder 712, and then, in the reconstruction phase, a lightweight prediction head (such as, but not limited to, a linear layer) 716 reconstructs the spectral data. An illustrative example of the masking strategy is shown in FIG. 7. In this regard, each of the input spectra is masked at a high ratio, and both masked and visible segments are input to the encoder 712.

[0064] Accordingly, in various example embodiments, the above-mentioned spectra masking comprises, for each of the unlabeled spectral images: masking a plurality of frequency segments of the unlabeled spectral data of the unlabeled spectral image to obtain partially masked spectral data of the unlabeled spectral image; and inputting the partially masked spectral data of the unlabeled spectral image to the machine learning model 712 to produce a feature vector thereof. Furthermore, in various example embodiments, the above-mentioned masking the plurality of frequency segments of the unlabeled spectral data of the unlabeled spectral image comprises masking a majority of the frequency segments of the unlabeled spectral data. Still further, in various example embodiments, about 60% to about 80% of the frequency segments of the unlabeled spectral data are masked

[0065] Pre-training Stage'. During the pre-training stage 710, the pretext task involves masking and reconstructing input data using a masking strategy on the spectra data. As shown in FIG. 7, the partially masked input signal, is fed into the encoder 712. For example, the encoder 712 may include a 3-layer multilayer perceptron (MLP) followed by a rectified linear unit (ReLU) activation function. For example, the encoder 712 takes a 1 X 92 vector as input and finally outputs a 150-dimensional feature vector. In the MSM, according to various example embodiments, one prediction head 716 may be utilized as a decoder to reconstruct the masked frequency segments of the NIR spectrum. This reconstruction task allows the machine learning model 712 to learn more meaningful representations from the information of the partially masked frequency signals. The raw reflectance values of the masked segments may be regressed using fl loss. The loss may be calculated by comparing the true reflectance value and the predicted value of all masked segments of the signal.

[0066] The above-mentioned pre-training the machine learning model 712 comprises pretraining the machine learning model 712 with a prediction head 716 based on the unlabeled spectral data of the plastic samples using the above-mentioned self-supervised learning with the above-mentioned spectra masking to obtain the pre-trained machine learning model 722. Furthermore, for each of the unlabeled spectral images, the self-supervised learning comprises inputting the feature vector produced based on the partially masked spectral data of the unlabeled spectral image to the prediction head 716 to produce a reconstructed spectral data of the partially masked spectral data of the unlabeled spectral image based on the feature vector.

[0067] Fine-tuning Stage : During the fine-tuning stage 720, the pre-trained encoder model 722 is leveraged for the downstream task, and a classification head 726 (e.g., a fully connected layer) is added to the encoder 722 to detect the class (e.g., plastic resin type). In this regard, fine-tuning adjusts all the parameters of the pre-trained model 722. As for the training and evaluation of the downstream classification task, a binary cross-entropy loss may be employed as the loss function.

[0068] Accordingly, in various example embodiments, the above-mentioned fine-tuning the pre-trained machine learning model 722 comprises fine-tuning the pre-trained machine learning model 722 with a classification head 726 based on the labeled spectral data of plastic samples. In this regard, the classification head 726 is configured to perform a plastic type classification task.Experimental Results

[0069] Experimental results will now be presented, including a description of the dataset utilized, baselines, an evaluation of the performance of an example method of classifying a plastic type of a plastic object using the machine learning model trained according to the method 700, and discussion of the performance measure of the example method of classifying a plastic type of a plastic object.|0070| Dataset. In this study, the AIRP training dataset was employed for pre-training, without utilizing any annotations. Meanwhile, the AIRP Test was utilized as the evaluation dataset. In the former setting, the labels of the AIRP training dataset were removed during the pre-training stage, and 6502 samples were used for the self-supervised pre-training The model was fine-tuned for example, on only 50 labeled samples chosen from the AIRP Training dataset for both self-supervised and baseline settings.

[0071] Experimental Setup. Two experiments were conducted: one with the self-supervised pre-training approach using MSM as described in the method 700 and one without selfsupervised pre-training as the baseline. First, a baseline 3-layer MLP model was trained on the labeled AIRP training dataset using supervised training, i.e., omitting the self-supervised pretraining step and fine-tuning the model on the downstream task. Also, the MLP model was trained from scratch on only 50 samples. Second, self-supervised pre-training was conducted on the simple MLP-based encoder for the 80% of the gathered AIRP training spectra dataset, while 20% of the signals were reserved for validation. Accuracy is the more common evaluation metric. The models fine-tuned in both settings were evaluated on AIRP Test dataset. The model weights with the lowest loss were saved for the evaluation on the test samples extracted from the center ROI (see FIGs. 8 to 11). In FIG. 8, the images denoted by (a) and (b) show illumination light changes on a HOPE plastic bottle, the image denoted by (c) shows a crushed HDPE plastic bottle, and the image denoted by (d) shows a HDPE plastic bottle with a label. FIG. 9 depicts a plot showing the diversity of acquired spectra from the center ROI of HDPE plastic samples. In FIG. 10, the image denoted by (a) shows a PET plastic bottle with a distorted shape, the image denoted by (b) shows a PET plastic bottle with fluid contamination, the image denoted by (c) shows a PET bottle with a label, and the image denoted by (d) shows a crushed bottle. FIG. 11 depicts a plot showing the diversity of acquired spectra from PET plastic samples.

[0072] Training. The experiments were based on the PyTorch framework. The total number of pre-training stages was 300 epochs, and the learning rate was le-3. In both the pre-trainingand fine-tuning stages, Adam was invoked as an optimizer with a decay strategy of learning rate when validation loss stopped improving and a batch size of 64. The total number of training epochs in the fine-tuning stage was 30, and a learning rate of le-3. The experiments were performed on a single RTX 6000 Ada GPU.

[0073] Results and Discussion'. Table 2 shown in FIG. 12 reports the classification accuracy of each model evaluated the test dataset for two different plastic resin types. The method 700 was used to pre-train an MLP encoder on the AIRP Pre-training dataset. A 3-layerMLP model was used to train the 50 samples from scratch (Baseline 1) and supervised pre-training on the AIRP Pre-training dataset and then fine-tuned on 50 samples (Baseline 2) as the baseline models. The mean and standard deviation of accuracy were reported over five seeds. It was investigated whether fine-tuning from a self-supervised representation learning could possibly enhance results in the downstream plastic resin type classification task compared to using only supervised pre-training and fine-tuning. Besides the previously mentioned techniques, certain machine learning algorithms, such as the SVM (described in Duan et aL, “Classification of common household plastic wastes combining multiple methods based on near-infrared spectroscopy,” ACS ES&T Engineering, vol. 1, pp. 1065 - 1073, 7 2021) and random forest (RF) (Chen etal., “Enabling mechanical recycling of plastic bottles with shrink sleeves through near-infrared spectroscopy and machine learning algorithms,” Resources, Conservation and Recycling, vol. 188, pp. 106719, Jan 2023), are also extensively utilized in NIR applications. Therefore, in this study, the effectiveness of SVM, RF, and the three aforementioned methods were evaluated and compared for classifying two different types of plastic waste, namely, HDPE and PET. For the model fine-tuned on the supervised pre-trained model on the labelled AIRP dataset, the model performed better, with an accuracy result of 82.42% compared to the baseline 3-layerMLP model trained from scratch, which obtained an accuracy result of 46.28% for the AIRP Test dataset. Moreover, the model trained according to the method 700 obtained the highest accuracy of 96.06%. The model trained according to the method 700 also performs better than two baselines (i.e., training from scratch and supervised pre-training), indicating that the simple MLP encoder 712 can learn feature representations that are efficient and effective in detecting plastic resin types through self-supervised pretraining. To obtain a qualitative sense of the pre-training and the spectral signature reconstruction according to various example embodiments of the present invention, see FIGs. 13 A and 13B. In particular, FIGs. 13 A and 13B each depicts a plot showing a reconstruction of a plastic spectra signal from AIRP dataset by the masked SSL framework with 70% of masked frequency segments, compared with theoriginal plastic spectra signal, according to various example embodiments of the present invention. It can be seen that there is little difference between the original and reconstructed signals of different plastic categories.

[0074] FIG. 14A and 14B visualize results obtained during the inference for plastic wastes (plastic bottle classification as HDPE and PET bottles) by the plastic classification model trained according to the method 700 using different ROIs within the same image. In particular, in FIG. 14A, the input data cube samples are from the center of detected object boundaries using the object detection module. Each colored box has a 32X 32 spatial resolution. In FIG. 14B, plastic resin type classification using the plastic classification model, and the numbers show the confidence of the detected class.

[0075] To find a suitable masking ratio (or masking rate) for MSM, ablation experiments was conducted using various masking ratios, as shown in FIG. 15. In the pretraining phase 710, masking ratios ranging from 10% to 90% were established or tested. Throughout this process, the model was fine-tuned, and the accuracy metric was evaluated using the AIRP test set. According to the experimental results, as shown in FIG. 15, masking ratios from more than 50% to about 90% were found to achieve about 90% classification accuracy or higher on the AIRP test set. Moreover, masking ratios from about 60% to about 80% were found to achieve about 92% classification accuracy or higher. Furthermore, a masking ratio of about 70% was found to achieve an optimal classification accuracy of about 96%. Accordingly, it can be seen that masking about 70% of frequency segments is an optimal masking ratio due to its superior performance in accuracy. In particular, FIG. 15 shows the results of MSM with different masking ratios. A higher masking ratio brings a higher risk of training failure, where models have difficulty in finding good representations with limited information. In this regard, according to various example embodiments, a masking ratio of about 70% is considered to be the best masking ratio due to its good performance in fine-tuning, and the results are consistent with the experimental results of MSM.

[0076] Accordingly, near-infrared (NIR) spectroscopy is a viable option for plastic type classification (e.g., plastic material detection). The lack of diversity and scale of such data makes it difficult to translate the research outcomes into industrial settings. According to various example embodiments, a diverse plastic dataset (which may be referred to herein as Al Recycling Plastics (AIRP)) is collected, with different compositions of clean and contaminated plastics. Moreover, self-supervised pre-training with masked signal modeling (MSM) was applied to improve the performance and sample efficiency of deep learning models in detectingthe plastic resin types from the spectral signature data. Experimental results, as discussed above, demonstrate that the self-supervised pre-training with MSM, as a pre-training task, teaches the machine learning model to learn rich features that generalize well from unlabeled spectra data. Furthermore, fine-tuning performance in the detection of plastic resin type outperforms the baseline supervised techniques. Accordingly, the method 700 of training a machine learning model for performing plastic type classification, as well as the corresponding method of classifying a plastic type of a plastic object, shows superior results (e.g., improved efficiency and effectiveness) that can be applied industrially, such as at material recovery facilities to sort recyclable plastics and less common polymer types (e.g., integrated in a robot platform for automatic waste recycling). In contrast, conventional workflows rely on humans for manual plastic sorting, which not only is slow and labour-intensive but also can achieve only substandard purity, resulting in degraded socioeconomical outcomes. Tn this regard, the lack of effective plastic waste sorting technologies is one key barrier that inhibits plastic recycling rate. Accordingly, various example embodiments advantageously enable the transformation of plastic waste sorting into an automated and effective procedure via multispectral Al technologies. In this regard, various example embodiments advantageously developed multispectral self-supervised learning to exploit underlying structure of unlabeled spectral data in the spectral dimension. For example, the method 700 of training a machine learning model for performing plastic type classification, as well as the corresponding method of classifying a plastic type of a plastic object, may be applied for large-scale waste recycling at processing facility and large-scale waste profiling / auditing for large corporations / institutions.

[0077] As an illustrative example and without limitation, the classification of two example common household plastic types using self-supervised masked signal modeling (MSM) has been described above. It will be understood by a person skilled in the art that the method of classifying a plastic type of a plastic object is not limited to classifying two plastic types (or not limited to classifying HDPE and PET) and the classification head 726 may be configured and trained with the machine learning model 722 to classify any number of plastic types (or plastic resin types) as desired or as appropriate without going beyond the scope of the present invention. According to various example embodiments, a large and diverse NIR-based dataset of plastic spectra is collected with careful sample pretreatment (e.g., with a variety of contamination, including crushed / di started shapes and fluid contamination). The method 700 of training a machine learning model for performing plastic type classification does not rely on a large number of labeled plastic samples, but utilizes a simple MLP architecture to design thepretext task and learn a good representation of the plastic spectral representations. Moreover, the diversity of collected datasets of NIR-based spectra makes the pre-trained model learn better representation. Accordingly, the method 700 of training a machine learning model for performing plastic type classification, as well as the corresponding method of classifying a plastic type of a plastic object, advantageously widens the application of self-supervised pretrained models in the practical application of waste management, which for example may also be applied to NIR-based plastic polymer identification. Various example embodiments demonstrate that HSI is a promising technology for real-time plastic waste sorting due to its high accuracy and environmentally friendly operation.

[0078] While embodiments of the invention have been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the scope of the invention as defined by the appended claims. The scope of the invention is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced.

Claims

CLAIMS1. A method of training a machine learning model for performing plastic type classification, the method comprising: pre-training the machine learning model based on unlabeled spectral data of plastic samples using self-supervised learning with spectra masking to obtain a pre-trained machine learning model; and fine-tuning the pre-trained machine learning model based on labeled spectral data of plastic samples for performing plastic type classification.

2. The method according to claim 1, wherein the unlabeled spectral data of the plastic samples are obtained from unlabeled spectral images of the plastic samples, and for each of the unlabeled spectral images, the unlabeled spectral data of the unlabeled spectral image comprises a plurality of spectral reflectance values for a plurality of spectral bands, respectively.

3. The method according to claim 2, wherein the unlabeled spectral images of the plastic samples are unlabeled hyperspectral images of the plastic samples, and for each of the unlabeled spectral images, the unlabeled spectral data of the unlabeled spectral image is in the form of a 1-dimensional vector comprising the plurality of spectral reflectance values for the plurality of spectral bands, respectively.

4. The method according to claim 2 or 3, wherein the unlabeled spectral images of the plastic samples are obtained in the near-infrared (NIR) field.

5. The method according to any one of claims 2 to 4, further comprising, for each of the unlabeled spectral images and for each spectral band of the plurality of spectral bands of the unlabeled spectral image, determining the spectral reflectance value for the spectral band of the unlabeled spectral image based on averaging pixel values of pixels of the unlabeled spectral image at the spectral band.

6. The method according to any one of claims 2 to 5, wherein said spectra masking comprises, for each of the unlabeled spectral images: masking a plurality of frequency segments of the unlabeled spectral data of the unlabeled spectral image to obtain partially masked spectral data of the unlabeled spectral image; and inputting the partially masked spectral data of the unlabeled spectral image to the machine learning model to produce a feature vector thereof.

7. The method according to claim 6, wherein said masking the plurality of frequency segments of the unlabeled spectral data of the unlabeled spectral image comprises masking a majority of the frequency segments of the unlabeled spectral data.

8. The method according to claim 7, wherein about 60% to about 80% of the frequency segments of the unlabeled spectral data are masked.

9. The method according to any one of claims 6 to 8, wherein the plurality of frequency segments are randomly masked amongst the frequency segments of the unlabeled spectral data.

10. The method according to any one of claims 6 to 9, wherein said pre-training the machine learning model comprises pre-training the machine learning model with a prediction head based on the unlabeled spectral data of the plastic samples using said self-supervised learning with said spectra masking to obtain the pre-trained machine learning model; and for said each of the unlabeled spectral images, said self-supervised learning comprises inputting the feature vector produced based on the partially masked spectral data of the unlabeled spectral image to the prediction head to produce a reconstructed spectral data of the partially masked spectral data of the unlabeled spectral image based on the feature vector.

11. The method according to claim 10, wherein the machine learning model comprises an encoder, the pre-trained machine learning model comprises a pre-trained encoder, and the prediction head comprises a decoder.

12. The method according to any one of claims 6 to 11, wherein said fine-tuning the pretrained machine learning model comprises fine-tuning the pre-trained machine learning model with a classification head based on the labeled spectral data of plastic samples, the classification head being configured to perform a plastic type classification task.

13. A system for training a machine learning model for performing plastic type classification, the system comprising: at least one memory; and at least one processor communicatively coupled to the at least one memory and configured to: pre-train the machine learning model based on unlabeled spectral data of plastic samples using self- supervised learning with spectra masking to obtain a pre-trained machine learning model; and fine-tune the pre-trained machine learning model based on labeled spectral data of plastic samples for performing plastic type classification.

14. The system according to claim 13, wherein the unlabeled spectral data of the plastic samples are obtained from unlabeled spectral images of the plastic samples, and for each of the unlabeled spectral images, the unlabeled spectral data of the unlabeled spectral image comprises a plurality of spectral reflectance values for a plurality of spectral bands, respectively.

15. The system according to claim 14, wherein the unlabeled spectral images of the plastic samples are unlabeled hyperspectral images of the plastic samples, and for each of the unlabeled spectral images, the unlabeled spectral data of the unlabeled spectral image is in the form of a 1-dimensional vector comprising the plurality of spectral reflectance values for the plurality of spectral bands, respectively.

16. The system according to claim 14 or 15, wherein the unlabeled spectral images of the plastic samples are obtained in the near-infrared (NIR) field.

17. The system according to any one of claims 14 to 16, wherein the at least one processor is further configured to, for each of the unlabeled spectral images and for each spectral band of the plurality of spectral bands of the unlabeled spectral image, determine the spectral reflectance value for the spectral band of the unlabeled spectral image based on averaging pixel values of pixels of the unlabeled spectral image at the spectral band.

18. The system according to any one of claims 14 to 17, wherein said spectra masking comprises, for each of the unlabeled spectral images: masking a plurality of frequency segments of the unlabeled spectral data of the unlabeled spectral image to obtain partially masked spectral data of the unlabeled spectral image; and inputting the partially masked spectral data of the unlabeled spectral image to the machine learning model to produce a feature vector thereof.

19. The system according to claim 18, wherein said masking the plurality of frequency segments of the unlabeled spectral data of the unlabeled spectral image comprises masking a majority of the frequency segments of the unlabeled spectral data.

20. The system according to claim 19, wherein about 60% to about 80% of the frequency segments of the unlabeled spectral data are masked.

21. The system according to any one of claims 18 to 20, wherein the plurality of frequency segments are randomly masked amongst the frequency segments of the unlabeled spectral data.

22. The system according to any one of claims 18 to 21, wherein said pre-train the machine learning model comprises pre-training the machine learning model with a prediction head based on the unlabeled spectral data of the plastic samples using said self-supervised learning with said spectra masking to obtain the pre-trained machine learning model; and for said each of the unlabeled spectral images, said self-supervised learning comprises inputting the feature vector produced based on the partially masked spectral data of the unlabeled spectral image to the prediction head to produce a reconstructed spectral data of the partially masked spectral data of the unlabeled spectral image based on the feature vector.

23. The system according to claim 22, wherein the machine learning model comprises an encoder, the pre-trained machine learning model comprises a pre-trained encoder, and the prediction head comprises a decoder.

24. The system according to any one of claims 18 to 23, wherein said fine-tune the pretrained machine learning model comprises fine-tuning the pre-trained machine learning model with a classification head based on the labeled spectral data of plastic samples, the classification head being configured to perform a plastic type classification task.

25. A method of classifying a plastic type of a plastic object, the method comprising: obtaining spectral data of the plastic object; and classifying the plastic type of the plastic object based on the spectral data of the plastic object using a machine learning model trained according to the method of any one of claims 1 to 12.

26. The method according to claim 25, wherein the spectral data of the plastic object is obtained from a spectral image of the plastic object, and the spectral data of the spectral image comprises a plurality of spectral reflectance values for a plurality of spectral bands, respectively.

27. The method according to claim 26, wherein the spectral image of the plastic object is a hyperspectral image of the plastic object, and the spectral data of the spectral image is in the form of a 1 -dimensional vector comprising the plurality of spectral reflectance values for the plurality of spectral bands, respectively.

28. The method according to claim 26 or 27, wherein the spectral image of the plastic sample is obtained in the near-infrared (NIR) field.

29. The method according to any one of claims 26 to 28, further comprising, for each spectral band of the plurality of spectral bands of the spectral image, determining the spectral reflectance value for the spectral band of the spectral image based on averaging pixel values of pixels of the spectral image at the spectral band.

30. The method according to any one of claims 25 to 29, when dependent on claim 24, wherein said classifying the plastic type of the plastic object based on the spectral data of the plastic object further uses the classification head configured to perform the plastic type classification task.

31. A system for classifying a plastic type of a plastic object, the system comprising: at least one memory; and at least one processor communicatively coupled to the at least one memory and configured to: obtain spectral data of the plastic object; and classify the plastic type of the plastic object based on the spectral data of the plastic object using a machine learning model trained according to the method of any one of claims 1 to 12 or using the system of any one of claims 13 to 24.

32. The system according to claim 31, wherein the spectral data of the plastic object is obtained from a spectral image of the plastic object, and the spectral data of the spectral image comprises a plurality of spectral reflectance values for a plurality of spectral bands, respectively.

33. The system according to claim 32, wherein the spectral image of the plastic object is a hyperspectral image of the plastic object, and the spectral data of the spectral image is in the form of a 1 -dimensional vector comprising the plurality of spectral reflectance values for the plurality of spectral bands, respectively.

34. The system according to claim 32 or 33, wherein the spectral image of the plastic sample is obtained in the near-infrared (NTR) field.

35. The system according to any one of claims 32 to 34, wherein the at least one processor is further configured to, for each spectral band of the plurality of spectral bands of the spectral image, determine the spectral reflectance value for the spectral band of the spectral image based on averaging pixel values of pixels of the spectral image at the spectral band.

36. The system according to any one of claims 31 to 35, when dependent on claim 30, wherein said classify the plastic type of the plastic object based on the spectral data of the plastic object further uses the classification head configured to perform the plastic type classification task.

37. A computer program product, embodied in one or more non -transitory computer- readable storage mediums, comprising instructions executable by at least one processor to perform the method of training a machine learning model for performing plastic type classification according to any one of claims 1 to 12.

38. A computer program product, embodied in one or more non-transitory computer- readable storage mediums, comprising instructions executable by at least one processor to perform the method of classifying a plastic type of a plastic object according to any one of claims 25 to 30.

Images