Method and electronic device for automated waste management

Abstract

[0006] Embodiments herein disclose a method and apparatus for waste management using an artificial intelligence-based waste object classification engine. The method includes acquiring at least one image and detecting at least one waste object from the at least one acquired image. The method further includes determining that the at least one detected waste object matches a pre-stored waste object and identifying a type of the detected waste object using the pre-stored waste object. The method further includes displaying a type of the detected waste object based on the identification.

Description

[Technical Field]

[0001] The present invention relates to waste management systems, and more particularly to an apparatus and method for automating waste management. [Background technology]

[0002] This section describes the technical field in detail and discusses problems encountered in the technical field. Therefore, statements in this section should not be construed as prior art.

[0003] A typical existing waste disposal system involves unsorted garbage collected from various locations and then manually separated at a waste disposal facility. Manual separation of solid waste not only poses health risks to waste sorters, but is also inefficient, time-consuming, and not entirely feasible due to the large amounts of waste discarded by modern households, businesses, and industries. To make the waste disposal system efficient, an automated waste disposal system is needed to sort, process, crush, compress, and clean waste using identifiers (e.g., barcode identifiers, etc.).

[0004] To make this process efficient, various methods and systems have been introduced in the prior art: US Patent No. 5,929,999 (Kline et al.) discloses a waste recovery and conversion center / power plant that replaces traditional garbage dumps and landfills.

[0005] US Patent No. 6,299,999 (Brunner et al.) discloses mining experimental information to identify patterns from a data measurement database collected from observations.

[0006] US Patent No. 6,299,949 (Kumar et al.) discloses a material sorting system that utilizes a vision and / or x-ray system that implements a machine learning system to identify or classify each of the materials, which are then sorted into separate groups based on such identification or classification.

[0007] US Patent No. 6,299,699 (Horowitz et al.) discloses a system for optical material characterization of waste materials using machine learning. Additionally, US Patent No. 6,299,699 (Parr et al.) discloses a system control for a materials recovery (or recycling) facility. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] U.S. Patent Application Serial No. 10 / 943,897 [Patent Document 2] U.S. Patent No. 7,269,516 [Patent Document 3] U.S. Patent Application Serial No. 15 / 963,755 [Patent Document 4] U.S. Patent Application Serial No. 16 / 177,137 [Patent Document 5] U.S. Patent Application Serial No. 16 / 247,449 Summary of the Invention [Problem to be solved by the invention]

[0009] However, the prior art, dating back decades, lacks automated methods and systems for accurate waste management. Thus, there is a long-felt need for the inventive approach that can overcome the limitations associated with conventional waste management techniques. To solve these problems, the present invention provides automated devices, systems, and methods for fast, accurate, and reliable waste management. [Means for solving the problem]

[0010] The present invention discloses an artificial intelligence-based method for automated waste management.

[0011] In a first aspect of the present invention, a method for waste management is disclosed. The method includes acquiring at least one image. The method further includes detecting at least one waste object from the at least one acquired image. The method further includes determining that the at least one detected waste object matches a pre-stored waste object, identifying a type of the detected waste object using the pre-stored waste object, and displaying the type of the detected waste object based on the identification.

[0012] In one embodiment, the method further comprises notifying a user of the type of waste object detected.

[0013] In an alternative preferred embodiment, the pre-stored waste objects are generated by acquiring a waste object dataset including waste objects having various categories, acquiring portions of images corresponding to the waste objects from the acquired waste object dataset, training the portions of the images corresponding to the waste objects using a machine learning model, and generating the pre-stored waste objects based on the trained portions of the images corresponding to the waste objects.

[0014] In one embodiment, detecting at least one waste object from at least one acquired image includes identifying at least one waste object from the at least one acquired image, extracting the at least one identified waste object from the at least one acquired image by processing a foreground portion of the at least one acquired image and a background portion of the at least one acquired image, determining at least one feature parameter based on the extraction, analyzing one or more pixels corresponding to the at least one identified waste object based on the determined feature parameter, and detecting the at least one waste object from the at least one acquired image based on the analyzed pixels.

[0015] In yet another embodiment, identifying the type of the detected waste object using the pre-stored waste object includes determining whether multiple types of the detected waste object are detected, and performing one of the following: in response to determining that multiple types of waste objects are not detected, identifying the type of the detected waste object using at least one feature parameter, and in response to determining that multiple types of waste objects are detected, determining at least one feature parameter based on the at least one identified waste object, analyzing one or more pixels corresponding to the at least one identified waste object based on the determined feature parameter, and detecting at least one waste object from the at least one acquired image based on the analyzed pixels.

[0016] In an alternative embodiment, the characteristic parameters include the shape of the waste object, the color of the waste object, and the intensity of the waste object.

[0017] In a second aspect of the present invention, an electronic device for automated waste management is disclosed. The electronic device includes a processor coupled to a memory and an artificial intelligence-based waste object classification engine coupled to the processor. The artificial intelligence-based waste object classification engine is configured to acquire at least one image, detect at least one waste object from the at least one acquired image, and determine that the at least one detected waste object matches a pre-stored waste object. The artificial intelligence-based waste object classification engine is also configured to identify a type of the detected waste object using the pre-stored waste object and can display the type of the detected waste object based on the identification.

[0018] Preferred embodiments of the present invention will now be described in conjunction with the accompanying drawings, which are provided to illustrate, but not to limit, the scope of the invention, where like designations refer to like elements and prior art is designated as "Prior Art." [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a block diagram of an electronic device for waste management in accordance with the teachings of the present invention; [Figure 2] System block diagram for waste management [Figure 3] Block diagram of an artificial intelligence-based waste object classification engine included in an electronic device for waste management [Figure 4] Schematic showing the different layers in an artificial intelligence-based waste object classification engine [Figure 5] Flowchart showing a method for waste management [Figure 6] 1 is an exemplary flowchart illustrating various operations for waste management; [Figure 7] A flowchart illustrating the various actions taken to create a machine learning model in conjunction with Figure 5 [Figure 8] A flowchart illustrating various operations for training and maintaining a machine learning model in conjunction with Figure 5. [Figure 9] FIG. 1 is a perspective view of a smart bin waste sorting device of the present invention. [Figure 10] Alternative perspective view of a smart bin waste sorting device [Figure 11] Partial cross-section of a collection canister included in the Smart Bin waste sorting device [Figure 12] A perspective view of a smart bin back panel included in a smart bin waste sorting device. [Figure 13] A perspective view of a smart bin waste sorting device including visual indicators [Figure 14] Schematic diagram of an exemplary system in which a smart bin waste sorting device communicates with a smartphone for waste management. DETAILED DESCRIPTION OF THE INVENTION

[0020] When reading this section (Description of Exemplary Preferred Embodiments Describing Exemplary Embodiments of the Best Mode of the Invention, hereinafter referred to as "Exemplary Embodiments"), the exemplary embodiments should be considered as the best modes for carrying out the invention in accordance with the inventors' contemplation during the application for patent. The exemplary embodiments should not be construed as limiting the invention to one embodiment, as those skilled in the art may recognize substantially equivalent structures or substantially equivalent acts to achieve the same result in the same or different ways.

[0021] Discussion of a species (or particular item) calls out the genus (class of items) to which the species belongs, as well as related species within that genus. Similarly, description of a genus calls out species known in the art. Moreover, as technology evolves, numerous additional alternatives for achieving embodiments of the invention may arise. Such advances should be incorporated within their respective genuses and recognized as functionally equivalent or structurally equivalent to the embodiments shown or described.

[0022] A function or action should be construed to incorporate all modes of performing the function or action unless otherwise specified. For example, sheet drying may be performed by dry heat or moist heat application, or by using microwaves. Thus, use of the term "paper drying" invokes "dry heat" or "moist heat" and all other modes of this term, as well as similar terms such as "pressure heating."

[0023] Unless otherwise expressly stated, conjunctions (e.g., "or," "and," "includes," or "comprises") are to be construed in an inclusive sense rather than an exclusive sense.

[0024] As will be appreciated by those skilled in the art, various structures and devices are shown in block diagram form in order to avoid obscuring the present invention. In the following discussion, acts with similar names are performed in a similar manner unless otherwise specified.

[0025] The foregoing discussion and definitions are offered for the purposes of clarification, not limitation. Terms and phrases are accorded their ordinary and plain meanings unless otherwise indicated.

[0026] In the following detailed description of embodiments of the present invention, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to unnecessarily obscure aspects of the embodiments of the present invention.

[0027] Furthermore, it will be apparent that the present invention is not limited to these embodiments. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art without departing from the spirit and scope of the present invention.

[0028] In a preferred embodiment, the present invention provides an artificial intelligence-based waste object classification engine that, in selected embodiments, is custom-designed (and thus may employ the use of custom-designed, captured training models) and is created from machine learning techniques known as deep learning methods, which allow the artificial intelligence-based waste object classification engine to automatically learn and improve from experience without being explicitly programmed.

[0029] Deep learning methods use networks that can learn in an unsupervised manner from unstructured or unlabeled data. Deep learning methods employ multiple layers of neural networks, allowing the artificial intelligence-based waste object classification engine of the present invention to self-learn through inference and pattern recognition, rather than developing procedural code or explicitly coded software algorithms. The neural network is modeled after the neuronal structure of the mammalian cerebral cortex, with neurons represented as nodes and synapses represented as uniquely weighted pathways between nodes. The nodes are then organized into layers to form the network. The neural network is organized in a layered manner, including an input layer, an intermediate or hidden layer, and an output layer.

[0030] Neural networks improve their learning ability by varying their uniquely weighted pathways based on the inputs they receive. Successive layers in a neural network incorporate learning by modifying their weighted coefficients based on the input patterns they receive. Training a neural network is very similar to how children are taught to recognize objects. The neural network is repeatedly trained from a base data set, and the results from the output layer are continuously compared to the correct classification of the image.

[0031] In an alternative explanation, any machine learning paradigm can be used in place of a neural network in the training and learning process.

[0032] 1 is a block diagram of an electronic device for waste management 100. The electronic device 100 can be, for example, but not limited to, a smart sort artificial intelligence (AI) bin system, a smart bin waste sorter, a smart waste separator, a smartphone, a smart Internet of Things (IOT) device, a smart server, etc.

[0033] In one embodiment, the electronic device includes a processor 102, a communicator 104, a display 106, a memory 108, an artificial intelligence based waste object classification engine 110, an imaging unit 112, and a sensor 114. Although physical connections are not shown, the processor 102 is communicatively coupled to the communicator 104, the display 106, the memory 108, the artificial intelligence based waste object classification engine 110, the imaging unit 112, and the sensor 114 in any manner known in the electronics arts.

[0034] The imaging unit 112 may be, for example, but not limited to, a standalone camera, a digital camera, a video camera, an infrared (IR) or ultraviolet (UV) camera, etc. The sensor 114 may be, for example, but not limited to, a distance sensor, a fill level sensor, an electronic scale, a strain gauge, etc.

[0035] In one embodiment, the imaging unit 112 acquires at least one image and shares the at least one acquired image with the artificial intelligence-based waste object classification engine 110. In one example, the camera captures real-time digital images (e.g., RGB images, etc.) or near real-time two-dimensional digital images or a continuous stream of digital images, geotags the acquired images, and the images may include multiple objects. The multiple objects include a user's hand on a waste object, the waste object on a tray, and a background portion along with the acquired image. In another example, a digital camera captures the waste image, a sensor detects useful feature information from the waste image, and the digital camera and sensor then forward the information to the artificial intelligence-based waste object classification engine 110.

[0036] After receiving the at least one acquired image, the artificial intelligence based waste object classification engine 110 detects at least one waste object from the at least one acquired image. In one example, the artificial intelligence based waste object classification engine 110 processes the continuous stream of digital images or acquired images to generate a properly cropped image that includes the waste object and minimal background for contextual understanding, increasing the probability certainty associated with the waste object.

[0037] In an alternative embodiment, the artificial intelligence-based waste object classification engine 110 is configured to identify at least one waste object from the at least one acquired image. Further, the artificial intelligence-based waste object classification engine 110 is configured to extract at least one identified waste object from the at least one acquired image by processing a foreground portion of the at least one acquired image and a background portion of the at least one acquired image. Based on the extraction, the artificial intelligence-based waste object classification engine 110 is configured to determine at least one characteristic parameter. The characteristic parameter may be, for example, but is not limited to, a shape of the waste object, a color of the waste object, an intensity of the waste object, an IR-detectable or UV-detectable image, texture information of the waste object, etc.

[0038] In one example, the artificial intelligence-based waste object classification engine 110 utilizes connected component information corresponding to the acquired image to segment the image into pixels and detect foreground that is not part of the primary item of interest in the foreground image. This results in a bounding box around the primary waste object to remove portions of other objects in the raw acquired image, and the raw acquired image is processed using an AI algorithm or a vision computer algorithm. Furthermore, the artificial intelligence-based waste object classification engine 110 creates feature values that represent how each pixel corresponds to the AI algorithm or the vision computer algorithm.

[0039] Based on the determined feature parameters, the artificial intelligence based waste object classification engine 110 is configured to analyze one or more pixels corresponding to the at least one identified waste object. Based on the analyzed pixels, the artificial intelligence based waste object classification engine 110 is configured to detect at least one waste object from the at least one acquired image.

[0040] After detecting at least one waste object from at least one acquired image, the artificial intelligence-based waste object classification engine 110 is configured to identify the at least one detected waste object as matching a pre-stored waste object.

[0041] In one embodiment, the pre-stored waste objects are generated by acquiring a waste object dataset including a set of waste objects with various categories, acquiring portions of images corresponding to each set of waste objects from the acquired waste object dataset, training the portions of images corresponding to each set of waste objects using a machine learning model 306, and generating the pre-stored waste objects based on the trained portions of images corresponding to the waste objects. The machine learning model 306 is described in conjunction with FIG. 3.

[0042] By using the pre-stored waste objects, the artificial intelligence-based waste object classification engine 110 is configured to identify the type of the detected waste object. In one example, the artificial intelligence-based waste object classification engine 110 is configured to use a machine learning classifier or filter to identify the type of the detected waste object. The type may be, for example, but not limited to, a recyclable type, a trash type, a compost type, etc. In one example, the images correspond to glass, cardboard, metal, paper, styrofoam, and food, where the recyclable type waste is glass, straw, and aluminum, and the trash type waste is styrofoam and coffee cups.

[0043] In an alternative embodiment, the artificial intelligence based waste object classification engine 110 is configured to determine whether multiple types of detected waste objects are detected. Alternatively, if multiple types of waste objects are not detected, the artificial intelligence based waste object classification engine 110 is configured to identify the type of detected waste object using at least one characteristic parameter.

[0044] In another embodiment, when multiple types of waste objects are detected, the artificial intelligence based waste object classification engine 110 determines at least one characteristic parameter based on the at least one identified waste object, analyzes one or more pixels corresponding to the at least one identified waste object based on the determined characteristic parameter, and detects at least one waste object from the at least one acquired image based on the analyzed pixels.

[0045] Based on identifying the type of the detected waste object, the artificial intelligence based waste object classification engine 110 is configured to display the type of the detected waste object on the display 106. The display 106 may be, for example, but not limited to, an information display, an LED display, an LCD display, etc.

[0046] Additionally, the artificial intelligence-based waste object classification engine 110 is configured to notify the user of the type of the detected waste object using the communicator 104. The communicator 104 may be, for example, but not limited to, a Bluetooth® communicator, a Wireless Fidelity (Wi-Fi) communicator, a Light Fidelity (Li-Fi) communicator, etc. In one example, the notification is provided in the form of an audio and visual alert using a speaker, an LED, and on-screen messaging. In another example, the notification is provided to the user in the form of a push message.

[0047] Additionally, the memory 108 includes stored instructions that, when executed by the at least one processor 102, cause an artificial intelligence based waste object classification engine 110 to perform a function on at least one image. The imaging unit 112 is connected to the processor 102 via a communicator 104, which may include wired or wireless communication means, such as, but not limited to, Bluetooth, near field communication, Wi-Fi, Universal Serial Bus, etc.

[0048] In one embodiment, if the image is a color image, the artificial intelligence-based waste object classification engine 110 utilizes the addition of additional information to aid in more accurate pixel classification. The accuracy of the artificial intelligence-based waste object classification engine 110 is directly proportional to the quality of the image. Image resolution provides the most effective classification of individual pixels and overall objects tested in various lighting conditions, backgrounds, and variable scenarios. Camera image capture must be continuous (i.e., from the point of detection to the point of disposal). Images must be well-illuminated, undistorted, and as unobtrusive as possible.

[0049] Furthermore, the artificial intelligence based waste object classification engine 110 uses multiple techniques including clustering and KNN classifiers, although other classifiers may be used within the scope of the present invention.

[0050] The communicator 104 is configured to communicate with internal units and external devices via one or more networks or a second electronic device (shown in FIG. 2). The memory 108 may include one or more computer-readable storage media. Thus, the memory 108 may include non-volatile storage elements. Examples of such non-volatile storage elements may include magnetic hard disks, optical disks, floppy disks, flash memory, or forms of electrically programmable memory (EPROM) or electrically erasable programmable memory (EEPROM). Furthermore, the memory 108 may, in some examples, be considered a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or propagating signal. However, the term "non-transitory" should not be interpreted as meaning that the memory 108 is non-removable.

[0051] 1 illustrates various units of electronic device 100, those skilled in the art will understand from reading this disclosure that other embodiments are not so limited. In other embodiments, electronic device 100 may include fewer or more various units. Furthermore, the labels or names of the various units are for illustrative purposes only and do not limit the scope of the invention. One or more units may be combined together to perform the same or substantially similar functions for managing waste.

[0052] 2 is a block diagram of a system 200 for waste management. In one embodiment, the system 200 includes a first electronic device 100a and a second electronic device 100b. The first electronic device 100a transfers at least one image to the second electronic device 100b in real time, near real time, or in a recorded format. After receiving the at least one image from the first electronic device 100a, the second electronic device 100b performs various operations for managing waste. The operation and functionality of the second electronic device 100b are as previously described in connection with FIG. 1.

[0053] 2 provides a limited overview of system 200, those skilled in the art will readily understand, after reading this disclosure, that other embodiments are not so limited. Furthermore, system 200 may include any number of hardware or software components in communication with one another.

[0054] 3 is a block diagram of the artificial intelligence-based waste object classification engine 110 included in the electronic device for waste management 100. In one embodiment, the artificial intelligence-based waste object classification engine 110 includes an artificial intelligence model 302, a classifier 304, and a machine learning model 306. Further, the artificial intelligence model 302 includes a box generator 302a and a shape classifier 302b. The classifier 304 may be, for example, but is not limited to, a k-nearest neighbor (KNN) classifier. The machine learning model 306 may be, for example, but is not limited to, a supervised learning and deep learning-based learning model and a multi-layer hybrid deep learning-based learning model.

[0055] In one embodiment, the machine learning model 306 is configured to classify waste objects in raw images into high-level groups, such as metal, glass, cardboard, paper, Styrofoam, food, and plastic, to guide, reward, educate, and match context with content. The artificial intelligence model 302 requires training examples prior to classification, allowing the machine learning model 306 to associate specific combinations of object vectors with specific class types. The result of this stage of the artificial intelligence model 302 during runtime operation is an overall classification of the objects based on the configured categories. The waste objects are deposited based on the classification. Additionally, objects that do not fit into the current classification are fed into machine learning routines, as described in FIGS. 7 and 8, to further train the system and expand the possible classification brackets.

[0056] The box generator 302a outputs a set of bounding boxes associated with the waste information, each bounding box defining the location, size, and category label of a waste object. The box generator 302a generates clear boundaries for the physical characteristics corresponding to the waste object. In one example, icon size and icon share are visually varied based on the intensity of the physical characteristics corresponding to the waste object. The shape generator 302b outputs the predicted shape and intensity of the physical characteristics corresponding to the waste object. The box generator 302a and the shape generator 302b can operate individually or together, either in series or in parallel. The classifier 304 classifies pixel value features into classes using an unsupervised learning model.

[0057] In one embodiment, the framework performs a machine learning procedure to train a classifier 304 using a training image pixel data set. The classifier 304 is applied to the image pixels to identify one or more distinct pixels, which may then be corrected. The artificial intelligence model 302 and the machine learning model 306 receive one or more training image data sets from a reference imaging system. Alternatively, the artificial intelligence model 302 and the machine learning model 306 may use or incorporate a validation architecture, training, and implementation to "teach," correct, and implement the identification of waste materials.

[0058] In another embodiment, the artificial intelligence model 302 uses a convolutional neural network (CNN)-based technique to extract features corresponding to the image and a multi-layer perceptron technique to integrate the image features and classify the waste as recyclable, trash, compost, or other. The multi-layer perceptron technique is trained and validated against manually labeled waste objects. Furthermore, the artificial intelligence model 302 acts as a response center to classify waste objects by aggregating information collected from the imaging unit 112.

[0059] In another embodiment, the machine learning model 306 may be a layer-based neural network (e.g., a four-layer deep learning network, a five-layer neural network, etc.), which can be trained for predictive analysis. For example, a four-layer-based neural network has 32, 16, 10, and 4 nodes at each level to achieve deep learning for waste object prediction. The prediction function passes a feature vector set to the neural network and generates an output, as shown in FIG. 4. As shown in FIG. 4, the layers between the first layer (i.e., input layer) and the last layer (i.e., output layer) are called hidden layers. All layers are used to process and predict waste objects. In another example, a four-layer neural network is used for waste object prediction, and the last layer (i.e., output layer) has six nodes for predicting waste objects. Generally, the neural network has a first layer including 32 nodes, a second layer corresponding to 16 nodes, and a last layer including five or six nodes for predicting waste objects.

[0060] In another embodiment, the machine learning model 306 is created by the TensaFlow library. First, the machine learning model 306 builds a dataset of m sets of waste objects, which are created by acquiring and tagging information on the internet for waste classification. Then, the machine learning model 306 extracts features of the waste objects in the dataset. From the TensaFlow library model, the machine learning model 306 correlates which category the waste object belongs to. In one example, the waste predicted through the machine learning model 306 is recyclable, trash, or compost.

[0061] In one embodiment, the accuracy and speed of the machine learning model 306 varies based on the amount of raw data set on which the machine learning model 306 is trained. In another embodiment, the accuracy and speed of the machine learning model 306 varies based on frame rate, overall CPU power, GPU power, etc.

[0062] 5 is a flowchart 500 illustrating a method for waste management according to one embodiment of the present invention. Operations (502-512) are performed by the artificial intelligence-based waste object classification engine 110.

[0063] In operation 502, the method includes acquiring at least one image. In operation 504, the method includes detecting at least one waste object from the at least one acquired image. Then, in operation 506, the method includes determining that the at least one detected waste object matches a pre-stored waste object. Next, in operation 508, the method includes identifying a type of the detected waste object using the pre-stored waste object. In operation 510, the method includes displaying a type of the detected waste object based on the identification. And, in operation 512, the method includes notifying a user of the type of the detected waste object.

[0064] The proposed method can be used to direct user behavior for waste sorting using AI-based computer vision techniques. The proposed method can be used to evaluate waste and classify it into desired categories, i.e., recyclables, trash, and compost. The proposed method can be implemented in waste disposal in many locations (e.g., office spaces, apartments, recreation areas, stadiums, homes, public places, parks, road sweeping, etc.). The proposed method can be used by users (e.g., technicians, agricultural users, food court employees, pedestrians, etc.).

[0065] The proposed method can be used to capture the visual information of a user carrying a waste object, analyze and classify the waste into the correct stream, provide visual warnings (through LEDs and on-screen messaging) or audio messages to the user, and automatically classify the waste discarded by the user.

[0066] FIG. 6 is an exemplary flowchart 600 illustrating various operations for waste management.

[0067] Beginning with operation 602, the method includes capturing an image and adding geotagging onto the image. By way of example, a camera captures the image and adds geotagging onto the image.

[0068] In operation 604, the method includes detecting and extracting foreground objects from the acquired image. By way of example, the artificial intelligence-based waste object classification engine 110 detects and extracts main objects and sub-images from the acquired raw image and separates background portions from the acquired raw image.

[0069] In operation 606, the method includes calculating feature values corresponding to feature parameters for pixel disambiguation associated with the acquired raw image. In one example, the artificial intelligence based waste object classification engine 110 calculates feature values corresponding to feature parameters for pixel disambiguation associated with the acquired raw image using a shape of the waste object and a color of the waste object.

[0070] Then, in operation 608, the method includes identifying waste sub-parts using the pixel disambiguation. By way of example, the artificial intelligence based waste object classification engine 110 identifies waste sub-parts using the pixel disambiguation.

[0071] Next, in operation 610, the method re-calculates feature values corresponding to the feature parameters for pixel disambiguation. By way of example, the artificial intelligence based waste object classification engine 110 re-calculates feature values corresponding to the feature parameters for pixel disambiguation.

[0072] In query 612, the method may determine whether multiple waste objects are detected. If multiple waste objects are not detected, in operation 614, the method includes classifying the waste objects. By way of example, the artificial intelligence-based waste object classification engine 110 may classify the waste objects.

[0073] Then, in operation 616, the method includes triggering the sensor 114 from the waste classification. As an example, the processor 102 triggers the sensor 114 due to the waste classification.

[0074] Alternatively, if multiple waste objects are detected from query 612, then in operation 618, the method includes performing feature disambiguation corresponding to the feature values for the multiple object detections. In one example, the artificial intelligence based waste object classification engine 110 performs feature disambiguation corresponding to the feature values for the multiple object detections.

[0075] After operation 618, the method proceeds to operation 620, which includes detecting and classifying a plurality of objects based on the feature definition. By way of example, the artificial intelligence based waste object classification engine 110 detects and classifies a plurality of objects based on the feature definition.

[0076] 7 is a flowchart illustrating various operations for creating the machine learning model 306 in conjunction with FIG. 5. Operations (702-706) are performed by the artificial intelligence model 302.

[0077] Method 700 begins at operation 702 with obtaining a raw dataset including a set of waste objects with various categories. Next, at operation 704, the method includes obtaining portions of images corresponding to the waste objects from the raw dataset. Then, at operation 706, the method includes creating a machine learning model by using the obtained waste object information. The machine learning model is trained based on frame rate, overall CPU power, GPU power, etc.

[0078] Figure 8 is a flow chart illustrating various operations for training and maintaining the machine learning model 306 in conjunction with Figure 5, according to one embodiment of the present invention. Operations (802-806) are performed by the artificial intelligence model 302.

[0079] First, in operation 802, the method includes obtaining portions of images corresponding to waste objects from the raw dataset. Next, in operation 804, the method includes labeling primary objects within the waste objects. Then, in operation 806, the method includes training and maintaining a machine learning model based on the labeled primary objects. The labeled primary objects include multiple classes of images corresponding to waste objects.

[0080] The various actions, operations, blocks, steps, etc. in flow diagrams 500-800 may be performed in the order presented, in a different order, or simultaneously. Furthermore, in some embodiments, some of the actions, operations, blocks, steps, etc. may be omitted, added, modified, skipped, etc. without departing from the scope of the present invention.

[0081] Reference is now made simultaneously to Figures 9 and 10, which are perspective views of a smart bin waste sorting device 100c incorporating the above teachings of the present invention. The smart bin waste sorting device 100c is an example of an electronic device 100. In particular, the substantial operation and functionality of the electronic device 100 has already been described in conjunction with Figures 1 through 8.

[0082] As shown in FIGS. 9 and 10, the smart bin waste sorting device (100c) includes a bin housing (116), a smart bin back panel (118), a collection can (120), a bin housing door (122), a bin housing lid (124) having an opening, a digital camera (112a), an information display (106a), a distance sensor (114a), a speaker (126), an optical indicator (128), a fill level sensor (114b), and an electronic scale (130). , strain gauges (114c) (shown in FIG. 11), processor (102) (shown in FIG. 12), power supply (132) (shown in FIG. 12), power distribution board (134) (shown in FIG. 12), mounting plate (136) (shown in FIG. 12), directional visual indicators (138) (shown in FIG. 13), digital camera array (112b) (shown in FIG. 13) for a wider field of view, and widescreen information display (106b) (shown in FIG. 13). The device shown is preferably sized for home or public use, such as in airports, sports facilities (e.g., stadiums, etc.), schools or office locations such as hallways, break rooms, or restrooms.

[0083] The bin housing (116) includes a collection canister (120) for collecting any type of waste. A smart bin back panel (118) is attached to and covers the top of the bin housing (116). A bin housing door (122) is provided with the bin housing (116), which includes a bin housing lid (124) with an opening for accessing and retaining the waste within the collection canister (120).

[0084] A digital camera (112a) captures images of the waste, an information display (106a) displays the type of waste, a distance sensor (114a) measures the distance between the user and the smart bin waste sorting device (100a), and a speaker (126) informs the user of the type of waste.

[0085] 10, an optical indicator (128) indicates the type of waste to the user, and a fill level sensor (114b) measures the level of waste stored within the collection canister (120). An electronic scale (130) is provided at the bottom of the collection canister (120).

[0086] Figure 11 is a partial cross-sectional view of a collection canister 120 included in the smart bin waste sorting apparatus 100c. As shown in Figure 11, strain gauges (114c) measure the weight of waste stored in the collection canister (120). The processor (102) is coupled to various elements within the smart bin waste sorting device (100a), such as the collection canister (120), the bin housing door (122), the bin housing lid with opening (124), the digital camera (112a), the information display (106a), the distance sensor (114a), the speaker (126), the optical indicator (128), the fill level sensor (114b), the electronic scale (130), and the strain gauges (114c).

[0087] Figure 12 is a perspective view of a smart bin back panel 118 included in a smart bin waste sorting apparatus 100c according to one embodiment of the present invention. As shown in Figure 12, a power supply 132 provides power within the smart bin waste sorting apparatus 100a via a power distribution board 134. A mounting plate 136 is provided within the smart bin back panel 118.

[0088] Figure 13 is a perspective view of a smart bin waste sorting device 100c including a visual indicator 138, according to one embodiment of the present invention. As shown in Figure 13, the visual indicator (138) provides direction to the user for waste disposal, and the digital camera array (112b) is used for a wider field of view. The widescreen information display (106b) displays information about the waste.

[0089] FIG. 14 is a schematic diagram of an exemplary system in which a smart bin waste sorting device 100c communicates with a smartphone 100d for waste management, according to one embodiment of the present invention.

[0090] In one embodiment, the system includes a smart bin waste sorting device 100c and a smartphone 100d. The smart bin waste sorting device 100c transfers at least one image to the smartphone 100d in real time, near real time, or in a recorded format. After receiving the at least one image from the smart bin waste sorting device 100c, the smartphone 100d performs various operations to manage the waste. The operation and functionality of the smartphone 100d are substantially as described in conjunction with Figures 1, 2, and 9-13.

[0091] Although the present invention has been described in connection with its preferred embodiments, it should be understood that many other possible modifications and variations can be made without departing from the spirit and scope of the present invention. Upon reading this disclosure, those skilled in the art will be able to make changes, modifications, and substitutions to achieve the same purposes of the present invention. The exemplary embodiments are merely illustrative and are not intended to limit the scope of the present invention. The present invention is intended to encompass all other embodiments that come within the scope of the description and their equivalents.

[0092] The methods and processes described herein may have fewer or additional steps or states, and steps or states may occur in a different order. Not all steps or states need to be reached. The methods and processes described herein may be embodied in, and fully or partially automated through, software code modules executed by one or more general-purpose computers. The code modules may be stored on any type of computer-readable medium or other computer storage device. Some or all of the methods may alternatively be embodied in whole or in part in dedicated computer hardware. The systems described herein may include displays, user input devices (e.g., touch screens, keyboards, mice, voice recognition, etc.), network interfaces, etc., as needed.

[0093] The results of the disclosed methods can be stored in any type of computer data repository, such as a relational database and a flat file system using volatile and / or non-volatile memory (e.g., magnetic disk storage, optical storage, EEPROM and / or solid-state RAM).

[0094] The various illustrative logic blocks, modules, routines, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality. Whether to implement such functionality as hardware or software depends on the particular application and design constraints imposed on the overall system. The described functionality may be implemented in various ways for each particular application, and such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0095] Furthermore, the various illustrative logic blocks and modules described in connection with the embodiments disclosed herein may be implemented or performed by a machine, such as a general-purpose processor device, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. A general-purpose processor device may be a microprocessor, but alternatively, the processor device may be a controller, microcontroller, or state machine, combinations thereof, etc. A processor device may include electrical circuitry configured to process computer-executable instructions. In another embodiment, a processor device includes an FPGA or other programmable device that performs logical operations without processing computer-executable instructions. A processor device may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. While described herein primarily with respect to digital technology, a processor device may also include primarily analog components. The computing environment can include any type of computer system, including, but not limited to, a microprocessor-based computer system, a mainframe computer, a digital signal processor, a portable computing device, a device controller, or a computational engine within an appliance, to name a few.

[0096] Elements of a method, process, routine, or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor device, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of non-transitory computer-readable storage medium. An exemplary storage medium may be coupled to the processor device such that the processor device can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integrated into the processor device. The processor device and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Alternatively, the processor device and the storage medium may reside as discrete components in a user terminal.

[0097] In particular, conditional language used herein, such as "can," "could," "might," "could," "for example," and the like, is generally intended to convey that certain embodiments include certain features, elements, and / or steps, while other embodiments do not, unless otherwise specified or understood within the context in which it is used. Thus, such conditional language is generally not intended to imply that features, elements, and / or steps are somehow required for one or more embodiments, or that one or more embodiments necessarily include logic for determining whether these features, elements, and / or steps are included in or should be performed in any particular embodiment, with or without other input or prompting. Terms such as "comprise," "include," "have," and the like are synonymous and are used in an inclusive, non-limiting manner and do not exclude additional elements, features, acts, operations, etc. Additionally, the term "or" is used in its inclusive sense (not exclusive); for example, when used to connect a list of elements, the term "or" means one, some, or all of the elements in the list.

[0098] Disjunctive language, such as the phrase "at least one of X, Y, Z," is understood differently with the context in which it is generally used to indicate that an item, term, etc. may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z), unless expressly stated otherwise. Thus, such disjunctive language is generally not intended to, and should not imply, that a particular embodiment requires that at least one of X, at least one of Y, or at least one of Z be present.

[0099] The networked electronic devices described herein may be in the form of mobile communication devices (e.g., mobile phones), laptops, tablet computers, interactive televisions, game consoles, media streaming devices, head-mounted displays, virtual or augmented reality devices, network watches, etc. Networked devices may optionally include displays, user input devices (e.g., touch screens, keyboards, mice, voice recognition, etc.), network interfaces, etc.

[0100] While the foregoing detailed description illustrates, describes, and points out novel features applicable to various embodiments, it should be understood that various omissions, substitutions, and changes in the form and details of the illustrated devices or algorithms may be made without departing from the spirit of the present disclosure. As can be recognized, certain embodiments described herein may be embodied in a form that does not provide all of the features and advantages described herein, as some features can be used or practiced separately from other features.

Claims

1. 1. An electronic device for waste management, comprising: memory; a processor coupled to the memory; and an artificial intelligence-based waste object classification engine coupled to said processor; Including, The artificial intelligence based waste object classification engine comprises: acquiring at least one image; Detecting at least one waste object from the at least one acquired image based on a foreground portion of the at least one acquired image and a background portion of the at least one acquired image and deriving at least one feature parameter therefrom; determining a feature value corresponding to the at least one feature parameter for pixel definition associated with the at least one acquired image; determining that the at least one detected waste object matches a pre-stored waste object; Do at least one of the following: Identifying the type of the detected waste object using the pre-stored waste; or providing an option to place the at least one detected waste object in a library and manually create a new classification for the unknown object, or to properly match the at least one detected waste object to a correct classification in the pre-stored waste object, and then add the new classification to the artificial intelligence based waste object classification engine to continue the artificial intelligence training process; displaying a type and a message of the detected waste object based on the identification; notifying a user of the type of waste object detected; Instructing the user to place the waste object in the correct waste container. An electronic device configured to:

2. 1. A method of waste management comprising: acquiring at least one image by an artificial intelligence based waste object classification engine; detecting, by the artificial intelligence based waste object classification engine, at least one waste object from the at least one acquired image based on a foreground portion of the at least one acquired image and a background portion of the at least one acquired image, and deriving at least one feature parameter therefrom; determining, by the artificial intelligence based waste object classification engine, a feature value corresponding to the at least one feature parameter for pixel disambiguation associated with the at least one acquired image; determining, by the artificial intelligence based waste object classification engine, that the at least one detected waste object matches a pre-stored waste object; performing at least one of the following: identifying, by the artificial intelligence based waste object classification engine, the type of waste object detected using the pre-stored waste object; or if the detected waste object is not identified, placing the at least one detected waste object in a library; Manually create new classifications for unknown objects, or properly matching the at least one detected waste object to the correct classification in the pre-stored waste object; and adding new classifications to the artificial intelligence based waste object classification engine to continue the artificial intelligence training process; displaying, by the artificial intelligence based waste object classification engine, the type of waste object detected based on the identification; and notifying a user of the type of the detected waste object by the artificial intelligence based waste object classification engine; instructing the user to place the waste object in the correct waste container; A method comprising:

3. 1. A method of waste management comprising: acquiring at least one image by an artificial intelligence based waste object classification engine; detecting, by the artificial intelligence based waste object classification engine, at least one waste object from the at least one acquired image based on a foreground portion of the at least one acquired image and a background portion of the at least one acquired image, and deriving at least one feature parameter therefrom; determining, by the artificial intelligence based waste object classification engine, a feature value corresponding to the at least one feature parameter for pixel disambiguation associated with the at least one acquired image; determining, by the artificial intelligence based waste object classification engine, that the at least one detected waste object matches a pre-stored waste object; performing at least one of the following: identifying, by the artificial intelligence based waste object classification engine, the type of waste object detected using the pre-stored waste object; or if the detected waste object is not identified, placing the at least one detected waste object in a library; Manually create new classifications for unknown objects, or properly matching the at least one detected waste object to the correct classification in the pre-stored waste object; and adding new classifications to the artificial intelligence based waste object classification engine to continue the artificial intelligence training process; displaying, by the artificial intelligence based waste object classification engine, the type of waste object detected based on the identification; and notifying a user of the type of the detected waste object by the artificial intelligence based waste object classification engine; categorizing the waste objects based on the configured categories; processing the waste objects based on the classifying step; Applying machine learning to train the method to expand classification categories to classify objects that do not belong to the current classification; and Providing the user with an optical or visual indicator of the type or orientation of the waste object to educate the consumer about the proper category and disposal of the waste object. A method comprising:

Images