Computer-implemented method and system for labeling objects

Abstract

Aspects concern a computer-implemented method and a system for linking an object to an object type. The computer-implemented method / system uses various image sensors to capture RGB images and power intensity heatmap images of an object on a moving platform, to synchronize the RGB images with the power intensity heatmap images, to detect and track the object as it moves on the platform, so that when, based on the tracking of the RGB image, it is determined that an object is being picked up for sorting by a sorter assigned to a specific object type, the corresponding power intensity heatmap image is automatically labeled with the determined object type.

Description

Technical area

[0001] The disclosure relates to a computer-implemented method and a computer-implemented system for labeling objects for a variety of purposes, such as subsequent training for machine learning. background

[0002] As global waste generation increases, the need to recycle or reuse used items such as plastic packaging or bottles grows to reduce CO2 emissions. Used items are typically sorted at collection points or in special sorting facilities before recycling.

[0003] One solution for sorting used objects relies on manual labor in the form of human workers or sorters. The workers are trained to recognize different objects based on, for example, plastic types. In a sorting area, which may include a conveyor belt, each sorter along the conveyor belt is assigned to pick up a specific type of plastic waste and transport it to an appropriate container or bin for sorting.

[0004] Artificial intelligence has been used in combination with various types of image capture devices to identify objects. However, artificial intelligence-based methods require enormous amounts of data to prepare a dataset for effective training and / or testing. Such preparation involves manual labeling, which can be time-consuming.

[0005] There is a need to provide a solution to alleviate at least one of the above problems. Brief description

[0006] A technical solution is provided in the form of a computer-implemented method and system for labeling objects prior to sorting. Sorting can be performed by a processor or an artificial intelligence (AI) module, such as a deep neural network (DNN) module. Such labeling involves associating an object with a specific object type and can then be used to train and / or test the AI ​​processor / module. The labeling process is independent of manual sorting.

[0007] The solution includes relatively inexpensive image capture devices such as cameras, video cameras, and camcorders to detect and track moving objects (such as plastic objects of at least two different plastic types) on a sorting area such as a conveyor belt and to monitor the sorters to determine whether they have picked up or removed the respective assigned plastic type.

[0008] The technical solution provides an automated solution for labeling the power intensity heatmap images (also known as THz images) generated by an electromagnetic radiation source while sorters perform the sorting work manually. The solution incorporates relatively inexpensive RGB cameras for locating and tracking moving plastic parts on a moving platform such as a conveyor belt and for monitoring the hand gestures of sorters. With gesture recognition and object detection, a cost-effective solution is provided that can reduce the hurdle of data collection and annotation when using THz or hyperspectral technologies. Furthermore, the technical solution minimizes disruption to existing sorting operations at a physical site.

[0009] According to the present disclosure, claim 1 provides a method for linking an object to an object type. Claim 9 provides a computer-implemented system for linking an object to an object type. Claim 11 defines a computer program comprising instructions for carrying out the computer-implemented method.

[0010] The dependent claims define some examples associated with the method. Short description of the drawings

[0011] A better understanding of the disclosure will be obtained by reference to the detailed description taken in conjunction with the non-limiting examples and the accompanying drawings, in which: - Fig. 1 is a flowchart of a computer-implemented method for associating an object with an object type according to some embodiments; - Fig. 2A shows a system architecture of a system for sorting objects according to some embodiments, FIG. 2B shows a design of a processor including various modules to implement the method for linking an object to an object type, Fig. 2C shows an intermediate and final output associated with the modules; - Fig. 3 shows an embodiment of time-based synchronization of RGB images and THz frames; - Fig. 4A and Fig. 4B show specific examples of a deep learning framework related to the detection and tracking of one or more objects; - Fig. Figure 5 shows an object alignment between a pair of an RGB image (single image) and a THz image (single image) - Fig. 6A to 6C illustrate hand landmarks, keypoint detection on a left human hand with a single and multiple fingers according to some embodiments of the gesture recognition module; - Fig. 7A to Fig. 7E illustrate the combined hand gesture recognition and object detection to form a hybrid model for determining whether an object is picked up; and - Fig. 8 shows a schematic diagram of a processor 210 for sorting objects according to some embodiments. Detailed description

[0012] The following detailed description refers to the accompanying drawings, which illustrate specific details and embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure. Other embodiments may be utilized, and structural and logical changes may be made without departing from the scope of the disclosure. The various embodiments are not necessarily mutually exclusive, as some embodiments may be combined with one or more other embodiments to form new embodiments.

[0013] Embodiments described in the context of one of the systems or methods apply analogously to the other systems or methods.

[0014] Features described in connection with one embodiment may accordingly be applicable to the same or similar features in the other embodiments. Features described in connection with one embodiment may accordingly be applicable to the other embodiments, even if they are not explicitly described in those other embodiments. Furthermore, additions and / or combinations and / or alternatives as described for a feature in connection with one embodiment may accordingly be applicable to the same or similar feature in the other embodiments.

[0015] In the context of some embodiments, the articles “a,” “an,” and “the,” as used with respect to a feature or element, include a reference to one or more of the features or elements.

[0016] As used herein, the term “and / or” includes any and all combinations of one or more of the related listed elements.

[0017] As used herein, the term "object" includes any object, especially recyclable or reusable objects, that can be sorted by one or more object types. For example, plastic objects can be sorted by whether they are made of high-density polyethylene (HDPE), polypropylene (PP), polystyrene (PS), low-density polyethylene (LDPE), polyvinyl chloride (PVC), or polyethylene terephthalate (PET). These objects include bottles, jars, containers, plates, bowls, etc., in various shapes, sizes, and configurations (e.g., partially compressed, deformed).

[0018] As used herein, the terms "link," "linked," "linking," and "linking" indicate a defined relationship (or cross-reference) between two or more elements. For example, a sorter may include a mechanical or human hand, or part thereof, used to pick up an object for sorting. A captured image associated with an object may include a defined region of interest focused on the object for further processing via one or more object detection algorithms.

[0019] As used herein, the term “sorting” broadly includes classification and / or categorization and / or arrangement.

[0020] As used herein, the term "sorter" includes a human tasked with sorting an object according to an object type assigned to the human. For example, a first worker may be assigned to sort HDPE plastic bottles, and a second worker may be assigned to sort PET plastic bottles. Accordingly, a sorter may include a hand or one or more other parts for retrieving the object.

[0021] As used herein, the term "module" refers to, forms part of, or includes an application-specific integrated circuit (ASIC); an electronic circuit; a combinational logic circuit; a field-programmable gate array (FPGA); a processor (shared, dedicated, or group) that executes code; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system-on-chip. The term "module" may include memory (shared, dedicated, or group) that stores code executed by a processor.

[0022] The disclosure uses electromagnetic radiation in a terahertz (THz) frequency range to capture image signals or data of an object. The THz frequency range can operate from 0.1 THz. The image data or signal may include the penetration intensity of the electromagnetic radiation, which is why the term "power intensity" is consistently interpreted throughout the description to include the power penetration intensity. It follows that image data or a signal based on the "power intensity" may include heat maps that show or represent the power penetration distribution.

[0023] As used here, the term “RGB image” refers broadly to images that include image data whose colors are encoded as three numbers associated with the colors “red,” “green,” and “blue.”

[0024] Fig. 1 shows a flowchart of a computer-implemented method for associating an object with an object type according to one embodiment. The method 100 includes the following steps: obtaining a plurality of RGB images and a plurality of power intensity heatmap images associated with the object positioned on a movable platform (step S102); correlating an RGB image from the plurality of RGB images with a corresponding power intensity heatmap image from the plurality of power intensity heatmap images (step S104); identifying a first region of interest on the RGB image and a second region of interest on the corresponding power intensity heatmap image, wherein the first region of interest and the second region of interest correspond to a boundary around the object (step S106);Determining, based on the first region of interest, whether the object is picked up by a sorter associated with an object type (step S108); wherein, if it is determined that the object is picked up by the sorter, associating the RGB image and the power intensity heatmap image with the object type (step S110).

[0025] The step of obtaining a plurality of RGB images and a plurality of power intensity heatmap images associated with the object positioned on a movable platform (step S102) may include receiving a video data file comprising a plurality of individual images of one or more objects moving on the platform over a predetermined time. Each object moving on the movable platform may be scanned or recorded by one or more first image sensors arranged to obtain RGB images and one or more second image sensors that obtain power intensity heatmap images.

[0026] The step of correlating an RGB image from the plurality of RGB images with a corresponding power intensity heatmap image from the plurality of power intensity heatmap images (step S104) may include a time-based synchronization step comprising substeps of (i.) providing a first timestamp on the RGB image and a second timestamp on the power intensity heatmap image and (ii.) comparing the first timestamp and / or the second timestamp with an expected timestamp, wherein the expected timestamp is derived from a speed of the moving platform and a distance on the moving sorting platform between a first image sensor for obtaining the RGB image and a second image sensor for obtaining the power intensity heatmap image.

[0027] The step of identifying a first region of interest on the RGB image and a second region of interest on the corresponding power intensity heatmap image (step S106) may include a step of detecting and tracking the object on the moving platform and assigning a tracking identifier to the RGB image. In some embodiments, the detected object in the RGB image and the corresponding power intensity heatmap image may be aligned based on calculating a Euclidean distance from a reference point of the RGB image to the center of the boundary around the object. In some embodiments, the reference point may originate from an origin coordinate (0,0) of the RGB image frame, and the boundary may correspond to a bounding box surrounding the object.

[0028] In step S108, the step of determining, based on the first region of interest, whether the object is being picked up by a sorter associated with an object type may include a step of determining whether the sorter's hand is in an open or closed position. If it is determined that the hand is in the closed position and it is determined that the bounding box and tracking identifier are not present compared to a previously existing linked prior image, it is determined that the object is being picked up.

[0029] In step S110, linking the RGB image and the power intensity heatmap image with the object type includes labeling the power intensity heatmap image with the object type.

[0030] In some embodiments, the labeled RGB image and power intensity heatmap image may form an input set, or a portion thereof, for training an artificial intelligence module. The artificial intelligence module is configured to sort a plurality of plastic types.

[0031] It is contemplated that steps S106 and / or S108 may include analytical and predictive computer algorithms such as artificial intelligence (AI) machine learning (ML) algorithms.

[0032] Steps S102 to S110 are shown in a specific order. However, it is contemplated that other arrangements are possible. In some cases, steps may also be combined. For example, steps S106 and S108 may be combined.

[0033] The method 100 can be implemented at least as part of a sorting system 200 for sorting plastic objects. Fig. Figure 2A illustrates such a sorting system 200 with a movable platform in the form of a conveyor belt 202 and Fig. Figure 2C shows the corresponding intermediate and final output at different stages. Objects 204 to be sorted are placed on the conveyor belt 202, and these objects are labeled A, B. In Fig. 2A shows two sorters in the form of workers S1 and S2. Worker S1 may be assigned to pick up and sort an object A, and worker S2 may be assigned to pick up and sort an object B. Object A may belong to one class of plastic (e.g., PET) and object B may belong to another class of plastic (e.g., HDPE).

[0034] Two first image sensors 206 may be positioned at suitable locations near the sorting area to record RGB videos (plurality of images) associated with each worker S1, S2, and the objects 204. A first image sensor 206a may be assigned to a sorting area 208a associated with the sorter S1, and another first image sensor 206b may be assigned to another sorting area 208b associated with the sorter S2. Although the illustrated embodiment shows one image capture device 206 associated with each of the sorters S1 and S2, it should be understood that one image capture device 206 may be assigned to two or more sorters, or two or more image capture devices 206 may be assigned to one sorter. A second image sensor 209 may comprise an electromagnetic radiation source and scanner assembly operating in a terahertz (THz) frequency range (e.g.,0.1 THz) and arranged such that an object moving on the conveyor belt 202 and irradiated by the electromagnetic radiation source generates a power intensity heatmap that is received by the scanner positioned beneath the conveyor belt. The power intensity heatmap can then be stored. The conveyor belt can be configured to move at a predefined speed, e.g., 0.45 meters per second (m / s).

[0035] In some embodiments, the system includes a processor 210 (see Fig. 2B) with various modules for performing steps S102 to S110. The modules may include a time synchronization module 212, an object detection / tracking module 214, a gesture recognition module 216, and a label module 218. The above-mentioned modules may include a respective input interface configured to receive and parse image data. The output of the time synchronization module 212 includes a pair of an RGB image and a THz image that are time-synchronized and associated with each object of interest. The output of the object detection / tracking module 214 may include a boundary around the object of interest, unique tracking identifiers associated with each detected and tracked object, and whether or not the boundary and / or unique tracking identifiers are present while the object is being detected and tracked by the module 214.The output of gesture recognition module 216 may be in the form of whether the part of a sorter associated with picking up the object (e.g., a human hand) is in an open state or a closed state. Processor 210 may be a single server or a distributed computing network across different computing resources / platforms.

[0036] The label module 218 is operable to receive the output from the gesture recognition module 216 and the object detection / tracking module 214 as inputs to provide a final output that associates the object A, B captured by the assigned sorter S1, S2 with an object type, and then label the THz image accordingly.

[0037] The first image sensor 206a, 206b and the second image sensor 209 may be operable to capture images in the form of a video file comprising multiple individual images, continuously or at predetermined intervals. Data from the image sensors 206, 209 may then be extracted and sent to the processor 210 for processing.

[0038] In some embodiments, processing by processor 210 may be performed in real time.

[0039] Fig. 3 shows an embodiment 300 of time-based synchronization of RGB images and THz frames. The RGB cameras 206 and the THz image sensor 209 are connected to the same computer / processor with synchronized global time. Each RGB image and each THz image can each be timestamped using the computer time format, e.g., hh.mm.ss:ssss, while an object moves within the scan range 302 of the RGB cameras 206 and the scan range 304 of the THz image sensor 209. An expected time parameter in the form of a variable travel time parameter can be derived by dividing the travel distance by the conveyor belt speed, as shown. For example, assume that the travel time is approximately 1080 milliseconds (00.00.0.1080). The RGB frame is timestamped at 13.04.5.3425 (time 1) and the THz frame is timestamped at 13.04.5.2345 (time 2).To correlate and localize a pair of THz and RGB frames, the timestamp of the THz frame is combined with travel time to find the nearest timestamp of an RGB frame, (time 1 ~ time 2 + travel time).

[0040] The time-synchronized RGB and power intensity heatmap images may be sent to the object detection / tracking module 214 for further processing. In some embodiments, the object detection / tracking module 214 may include hardware and / or software components to implement a machine learning (ML) and / or artificial intelligence (AI) algorithm to perform a method for detecting and tracking an object on the conveyor belt. Such a method may include generating bounding boxes and tracking identifiers. In some embodiments, detecting and tracking may include comparing the relevant features of an object in a first image with the features of the object in subsequent images, e.g., a second, third, fourth image, etc., and determining whether an object is present or absent.The object detection / tracking module 214 may be tested and / or trained prior to actual operation using techniques such as YOLOv4 and the DeepSORT model, which are known to those skilled in the art. In some embodiments, such training may involve placing an RGB camera on the conveyor belt, and continuously recording data of the plastics on the conveyor belt using the RGB camera and the THz image sensor. The recorded dataset may be divided into a training and validation dataset. The training dataset contained 85% of the total data, and the test dataset contained the remaining 15%. To generalize the deep learning model, augmentation may be applied to the training dataset. Examples of such augmentation include 90-degree rotations, 5% cropping, 2px blur, and horizontal and vertical flipping.

[0041] In some embodiments, the AI ​​model can be trained with 2000 epochs with a batch size of 64. The training process can be monitored by evaluating the images of the validation dataset every 100 epochs. One metric used to calculate the accuracy of the predictions is based on MAP (Mean Average Precision). It was found that the model achieved a MAP of 94.8% after training.

[0042] After training the object detection model, the DeepSORT tracking algorithm is used to track the detected objects across multiple RGB and THz frames. DeepSORT uses feature matching, examining the relevant features of an object in the first frame and comparing them with the features of the objects in the following frames. The algorithm tracks the objects if the features in the following continuous frames match within a threshold. During tracking, the algorithm generates a unique tracking identifier for each object. An example of the architecture combining the object detection and tracking models is shown in Fig. 4A and Fig. 4B shown.

[0043] After applying object detection and tracking, each object has a bounding box that includes parameters such as width, height, coordinates of the bounding box's center point, and a tracking identifier, which is a unique identifier. A step for aligning the detected object in the RGB image and the corresponding power intensity heatmap image involves calculating a Euclidean distance from a reference point of the RGB image to the center of the boundary around the object, as shown in Fig. 5, which shows the origin coordinate as the reference point, and calculates the Euclidean distance from the origin coordinate (0, 0) of the frame to the center of the bounding box, e.g., d1, d2 in the THz frame; d'1, d'2 in the RGB frame. The distances in both frames can then be sorted in ascending order, e.g., {1: d1, 2: d2} for the THz frame and {1: d'1, 2: d'2}, and the detected objects in both frames can be aligned using the sorting list.

[0044] The gesture recognition module 216 may include hardware and / or software components for implementing machine learning (ML) and / or artificial intelligence (AI), such as a known MediaPipe™ model for recognizing or predicting a hand gesture to detect a part of the sorter, e.g., a hand (or a part thereof) using an object detection technique and analyzing the landmark model to determine whether the hand is in an open or closed position. In some embodiments, the part of the hand may include a palm and / or the back of a hand. It is understood that the gesture recognition module 216 may perform hand gestures even if the hand (or a part thereof) of the sorter is covered by a glove. The MediaPipe™ framework may be used for building multimodal pipelines for applied machine learning. The framework may be used for building multimodal (e.g.,Video, audio, any time series data), cross-platform (i.e., Android, iOS, web, edge devices) pipelines for applied ML.

[0045] Fig. Figures 6A to 6C show various illustrations of the detection of hand landmarks and keypoints on a left human hand with a single and multiple fingers. It should be understood that the detection of hand landmarks and keypoints can be applied to a right human hand.

[0046] It is contemplated that other computer vision / image processing techniques known to those skilled in the art may be combined / supplemented to form further embodiments that complement and / or replace the ML / AI algorithms. For example, instead of a single-stage feature detector, a multi-stage feature detector may be considered.

[0047] Fig. Figure 7A shows a state diagram associated with predicting whether the sorter's hand is open or closed based on the output provided by gesture recognition module 216. The distance between keypoints is considered to predict whether the hand is open (state 702) or closed (state 704). The hand is considered closed if the distance between all 10 keypoints (five fingers) is less than or equal to a threshold specified for each finger. Transitioning between state 702 and state 704 is based on whether the distance between the keypoints is above or below the keypoint distance, which can be recalculated at every predetermined interval.

[0048] Fig. Figure 7B illustrates an open position and a closed position on a human hand with key points on each finger.

[0049] Fig. 7C shows a state diagram associated with predicting whether an object is determined to be present or absent based on the output provided by the object detection and tracking module 214. An object is determined to be present and in state 706 when both the bounding box and the tracking label are present. An object is determined to be absent and in state 708 when the bounding box and / or the tracking label disappear (i.e., during the course of tracking, the label was previously present in some frames and then disappears in subsequent frames).

[0050] The edition of both in Fig. 7A and Fig. The state diagrams shown in Figure 7C can be combined to determine whether a sorter picks up the object. Fig. Figure 7D shows the state diagram for this combination state diagram. An object is determined to be not picked up and to state 710, regardless of whether the hand is open or closed (state 702 or 704), as long as the tracking identifier is present (state 706). An object is also determined to be not picked up and to state 710 if the hand is open (state 702) and the tracking identifier is not present (state 708). An object is determined to be picked up and to state 712 if the hand is closed (state 704) and the tracking identifier is not present (state 708).

[0051] Fig. 7E illustrates image or video capture of an object being picked up by a sorter for sorting based on the detection of a back of a hand, corresponding to a state 710 (left frame) and a state 712 (right frame).

[0052] Once the object is determined to have been picked up by a sorter assigned to an object type, the labeling module 218 labels the power intensity heatmap image as the object type. For example, if the sorter S1 assigned to sort PET plastic objects picks up an object, the labeling module 218 automatically labels the power intensity heatmap image with a "PET object" label based on the detection of the RGB image picked up or retrieved for sorting.

[0053] It is contemplated that the object detection and tracking identifier is used to check whether each of the objects disappears over the predetermined period of time specified by the individual images and to output the class of the object. This is based on the assumption that the hand covers part or all of the object while capturing an object, and therefore the object detection / tracking module 214 can no longer detect the object. In other words, the bounding box and the tracking identifier of the object disappear, and the data is used to predict the disappearing object and its class, as shown in Fig. 7E shown.

[0054] In some embodiments, processor 210 may include hardware components such as one or more server computers arranged in a distributed or non-distributed configuration to implement characterization databases. The hardware components may be supplemented by a database management system configured to compile one or more industry-specific feature databases. In some embodiments, the industry-specific feature databases may include analysis modules for correlating one or more data sets with an industry. Such analysis modules may include an expert rule database, a fuzzy logic system, or any other artificial intelligence module.

[0055] In some embodiments, after an object is picked up by a sorter, the processor 210 may include an additional module for auditing the sorted objects and subsequently generating one or more reports.

[0056] Fig. 8 shows a server computer system 800 according to one embodiment. The server computer system 800 includes a communication interface 802 (e.g., configured to receive captured images from the image capture device(s) 806). The server computer 800 further includes a processing unit 804 and a memory 806. The memory 806 may be used by the processing unit 804 to store, for example, data to be processed, such as data associated with the captured images and intermediate results output by the modules 212, 214, 216 and / or final results output by the module 218. The server computer is configured to perform the method of Fig.1. It should be noted that the server computer system 800 may be a distributed system having a plurality of computers.

[0057] In some embodiments, a computer-readable medium is provided that includes program instructions that, when executed by one or more processors, cause the one or more processors to perform the methods according to the embodiments described above. The computer-readable medium may include a non-transitory computer-readable medium.

[0058] In some embodiments, the ML / AI algorithms may be trained using supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning techniques. In some embodiments, the ML / AI algorithms may include algorithms such as neural networks, fuzzy logic, evolutionary algorithms, etc.

[0059] In some embodiments, the described method for linking an object to an object type and generating resulting labels can be part of a semi-automated or hybrid real-time sorting process. For example, if the two sorters S1 and S2 have missed some objects due to object overlaps and / or a higher belt speed, the first image sensors 206 in the form of an RGB camera and AI can be used to provide predictions with a robotic sorter toward the end of the conveyor belt of the moving platform.

[0060] It is contemplated that the method for associating an object with an object type and generating resulting labels can be continuously executed by a processor during an actual sorting process performed by the manual sorters S1 and S2, so that a simultaneous increase in the size of the training dataset can be achieved, thereby strengthening and updating an AI prediction model that can be applied to train one or more AI robots for an entire real-time waste management facility, without requiring additional time for offline training and labeling.

[0061] Although the disclosure has been particularly shown and described with reference to particular embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the disclosure as defined by the appended claims. The scope of the disclosure is thus indicated by the appended claims, and all changes which come within the meaning and range of equivalence of the claims are therefore intended to be embraced.

Claims

[1] A computer-implemented method for associating an object with an object type, comprising the following steps: (a) obtaining a plurality of RGB images and a plurality of power intensity heatmap images associated with the object positioned on a movable platform; (b) correlating an RGB image from the plurality of RGB images with a corresponding power intensity heatmap image from the plurality of power intensity heatmap images; (c) identifying a first region of interest on the RGB image and a second region of interest on the corresponding power intensity heatmap image, wherein the first region of interest and the second region of interest correspond to a boundary around the object; (d) determining, based on the first region of interest, whether the object is picked up by a sorter associated with an object type; wherein, if it is determined that the object is picked up by the sorter, associating the RGB image and the power intensity heatmap image with the object type. [2] The method of claim 1, wherein the step of correlating the RGB image with the corresponding power intensity heatmap image includes: (i.) providing a first timestamp on the RGB image and a second timestamp on the power intensity heatmap image; (ii.) Comparing the first timestamp and / or the second timestamp with an expected timestamp, wherein the expected timestamp is derived from a speed of the moving platform and a distance on the moving sorting platform between a first image sensor for obtaining the RGB image and a second image sensor for obtaining the power intensity heatmap image. [3] The method of claim 1 or 2, wherein the step of identifying a first region of interest on the RGB image includes a step of detecting and tracking the object on the movable platform and assigning a tracking identifier to the RGB image. [4] The method of claim 3, further comprising a step of aligning the detected object within the RGB image and the corresponding power intensity heatmap image. [5] The method of claim 4, wherein the step of aligning includes calculating a Euclidean distance from a reference point of the RGB image to the center of the boundary around the object. [6] A method according to any one of the preceding claims, wherein the sorter comprises a hand and the step of determining whether the object is received for sorting includes the step of determining whether the hand is in an open position or a closed position. [7] The method of claim 6, wherein if it is determined that the hand is in the closed position and it is determined that the bounding box and the tracking identifier are not present compared to a previously existing linked prior image, it is determined that the object is captured. [8] A method according to any one of the preceding claims, wherein the step of associating the RGB image and the power intensity heatmap image with the object type includes labeling the RGB image and the power intensity heatmap image with the object type. [9] The method of claim 8, wherein the labeled RGB image and the power intensity heatmap image form an input set or a portion thereof for training an artificial intelligence module, the artificial intelligence module being configured to sort a plurality of plastic types. [10] Sorting system comprising: a first image sensor configured to obtain a plurality of RGB images of one or more objects positioned on a movable platform; a second image sensor configured to obtain a plurality of power intensity heatmap images of the one or more objects positioned on the movable platform; a processor arranged in data or signal communication with the first image sensor and the second image sensor to receive the plurality of RGB images and power intensity heatmap images, the processor operable to: Correlating an RGB image with a corresponding power intensity heatmap image; Identifying a first area of ​​interest on the RGB image and a second region of interest on the corresponding power intensity heatmap image, wherein the first region of interest and the second region of interest correspond to a boundary around the object; Determining, based on the first area of ​​interest, whether the object is picked up by a sorter associated with an object type; where, if it is determined that the object is picked up by the sorter, linking the RGB image and the power intensity heatmap image to the object type. [11] A computer program comprising instructions which, when executed by a computer, cause the computer to perform the following steps: (a) obtaining a plurality of RGB images and a plurality of power intensity heatmap images associated with the object positioned on a movable platform; (b) correlating an RGB image with a corresponding power intensity heatmap image; (c) identifying a first region of interest on the RGB image and a second region of interest on the corresponding power intensity heatmap image, wherein the first region of interest and the second region of interest correspond to a boundary around the object; (d) determining, based on the first region of interest, whether the object is picked up by a sorter associated with an object type; wherein, if it is determined that the object is picked up by the sorter, associating the RGB image and the power intensity heatmap image with the object type.

Images